Medical School Admissions and Deming’s Parable of the Red Beads

pattern_anlysisBy James E. Lewis, PhD 

I used to think I was the only one who thought that the root cause of many problems in the American health care delivery system, particularly those associated with quality, safety, and cost, would be found in the decisions made by medical school admissions committees. I don’t feel so alone now that bloggers and others are suggesting that a lottery for selecting among academically qualified medical school candidates would be as good as thirty-minute interviews, or that pre-application counseling and grooming will improve the overall quality of the students admitted.

“Quality,” of course, has many connotations; academic rank, professionalism, ability to apply knowledge in clinical situations, “doctoring” skills, empathy, independent, teamwork, and leadership are just a few. One school asks about “affinities:” alumni parents, family members, or friends who work at the institution, but, even while assigning an identifying number to students who have them, denies that affinity has any influence on the admission decision. Only one school explicitly encourages legacy applicants, while stretching the relationship well beyond the immediate family. Another school requires visiting applicants to sign a confidentiality agreement. What are they thinking?

In spite of the very high ratio of initial applicants to accepted students at nearly all schools, some schools with actual or intimated requirements that they accept only, or defined numbers of, students from a particular state or geographic area still make recruiting visits to undergraduate colleges and struggle to recruit qualified applicants from the defined geographic area. One school’s ratio of in-state applicants to acceptances is on the order of 7:1, while it receives about 46 applications for each of the first year positions it offers to out-of-state students. Recruiting quality minority and socio-economically disadvantaged students is quite competitive; richer schools strive to achieve their diversity goals by going after top students in states where local schools have fewer recruiting resources and/or historically were less open to minority students.

Does anyone in the admissions process ask medical school applicants if they have read, understand, and subscribe to the vision and mission statements of each of the 14 schools (on average) they apply to? If they don’t, how can they know (or do they care) whether the school and the applicant are a “good fit?” Compare the missions, visions, and values statements for any sample of the 141 schools; the same words and phrases are used so often they are interchangeable.

In the past, virtually all U.S. medical schools sought to admit the most academically qualified applicants possible. The process was not unlike a teaching exercise used by W. Edwards Deming, the guru of continuous quality improvement; he called it the “Parable of the Red Beads.” Participants formed a production team to produce white plastic beads. The feedstock for the production run was a box of beads, all similar in size and shape. The majority were white, but a small number of red beads was mixed in.

Deming, in his role as manager of the company producing white beads, trained the team, showed them the charts that would be used to record the variability (i.e., the number of red beads) in their production runs, and exhorted them to perform at the highest possible level in producing  white beads or risk being fired if the number of red beads in a lot was too high. Of course, the charts for each run showed random variability, and nothing the production team did could change that. The teams would switch their members from one job to another, add production runs, and try various other strategies to improve the ratio of white to red beads in each run. Eventually, every one would realize that the “fault” was to be found in the feedstock. The fewer the red beads in the feedstock, the fewer would be found in each production run.

To me, that is where the medical school admissions process finds itself—using a selection tool that has not been adapted to a vastly changed pool of applicants (feedstock) with women now in the majority; more, but not yet enough, applicants and matriculants from underrepresented minorities; and wider representation of socio-economic groups. Something still seems to be missing.

The list of desired applicant characteristics keeps growing, as does the definition of “diversity.” According to the websites, the latter now includes geography, life and lifestyle goals, past experiences in and out of the health field, age, and other things well beyond the initial concerns for gender, ethnicity, and socio-economics.

About a quarter of the schools claim to have officially adopted “holistic review” of the applicants. As espoused by the AAMC, holistic review is a conscious attempt to give full consideration to applicant interests and experiences beyond GPA and MCAT scores. Several schools conduct multiple mini-interviews rather than a few lengthy ones. These changes broaden the input to the decision process while bringing no real change to the applicant pool.

Applicants are still selected in most cases by an admissions committee, based on their AMCAS packet and a handful of criteria usually beginning with academic qualifications and MCAT scores. Those screened in (the secondary pool) will be invited to submit a supplemental application and perhaps make a campus visit for an interview.

The website verbiage suggests that about a third of the schools are looking for applicants who will be “best for the school.” Phrases like “best fit our mission” show up frequently. About half of the websites indicate that those schools are still committed to selecting and educating white beads, that is,  “undifferentiated” graduates who can take their post-MD careers in any number of directions with the expectation that they can achieve success.

Only one school states clearly and unequivocally that the admissions committee “is concerned solely with the quality and scope of an applicant’s undergraduate educational experience.” Others are searching for the “best student mix,” that is, a student body meeting their definition of diversity, while others are searching for the applicants who will become the “best doctors.”

Holistic review has to be better than searching for “white beads” when it comes to selecting applicants, but applying Deming’s approaches to analyzing and understanding variance among medical graduates three, five, 10, and 20 years post-graduation would provide more useful information for medical school leaders, admissions committees, and faculty who seem to become the work-around correction for weak admission decisions.

Where is the data on post-MD follow-up?  What does it show that could be used to further improve the admissions process for medical schools?

Lewis James—James E. Lewis, Ph.D., is an independent consultant to departments and schools of medicine, teaching hospitals, cardiovascular and cancer research and clinical programs, medical professional associations, disease oriented foundations, consulting firms, pharmaceutical companies, components of the National Institutes of Health, the Centers for Communicable Diseases, and the predecessors of the Center for Medicare and Medicaid Services. Previously he served as Deputy Dean for Operations and Vice President for Academic Administration, The Mount Sinai School of Medicine and Medical Center, New York City, where his academic title was Professor of Medicine and Health Policy; and Senior Executive Officer, Department of Medicine, University of Alabama at Birmingham, where his academic titles were Professor of Medicine and Adjunct Professor of Sociology. He can be reached at greegmountain@gmail.com.

Leave a Reply

Your email address will not be published. Required fields are marked *