Participants Share Lessons Learned From Virtual Match

August 24, 2021, 11:52 a.m. David Mitchell — The AAFP and four other family medicine organizations recently urged residency programs to “strongly consider conducting virtual interviews and virtual visits for all applicants” during the 2021-22 Match season. The 2020-21 Match season was forced online due to the pandemic, and the family medicine organizations called for another year of data to study the virtual format.

AAFP News: Annie, what are you telling students regarding number of applications?

Rutter: This is a question I have been asked since I first started working with medical students almost 10 years ago. I think the question is, did virtual interviewing and COVID change the game? The answer is, we don’t know yet. Anecdotally, Steve mentioned that he did more interviews. I think programs did more interviews because of that fear. I know our program did.

There’s no right number for everybody. Different students should apply to different numbers of programs. To determine what the number is, we have to look at the application as a whole. We have to think about what are their work experiences? What are their life experiences? What are their Step scores? What does their transcript look like? We have to think about geography — where do they want to be? Certain areas of the country are more competitive than others due to location or number of family medicine residencies in that area.

The AAMC talks about the point of diminishing returns, which is variable based on Step 1 scores and specialty. They don’t have data for IMGs, but for U.S. M.D. graduates, the number varies based on Step score. For students who do well on Step 1, we’re talking about 16 programs. If they’re doing not as well on Step 1, it’s about 23. The point of diminishing returns basically means that above that number students are actually not getting an increased likelihood of matching, and they actually may be going down.

I think when students start to hear their number, then they start wrapping their mind around, “OK, I’m not looking at 65 applications,” because family medicine is different. It’s different than your friends who are going into one of whatever number of neurosurgery residencies there are in the country, right? It’s different, and so I think we have to think about family medicine-specific advisors. It doesn’t mean they have to be family docs. It means they have to understand the family medicine Match. The National Residency Matching Program gives us data every year, and for 2020, we can look at how many programs did a student need to put on their rank list in order to successfully match in a family medicine residency. The median number of what they call contiguous ranks — number of ranks on their rank list before they get to a different specialty or their list ends — was 12 for U.S. M.D. seniors, and it was 11 for U. S. D.O. seniors.

The other thing to look at that is interesting is the probability of matching based on number of programs on your list. The probability of a U.S. M.D. senior matching if they have five contiguous family medicine programs ranked is 0.9. The same is true for U.S. D.O. seniors if they have six contiguous ranks. That’s really good.

Now, let’s say a medical student has a blip on their radar from a Step 1 failure. You have this many volunteer opportunities, you have this score on Step 2. Let’s work together as advisor and student to put all that information together and come up with a list that we feel comfortable with. We do this with patient care all the time. We take public health data, and we decide how to apply it to the patient in front of us, right? And so there’s always going to be the person sitting in front of you, and how do you tweak it to make it work for that particular student? You have the conversation, you have the risks and benefits and you figure out what makes the most sense. I think it’s easy as an advisor and as a student to fall in the trap of, “Well, if I just do five more, I’m more likely to match.” And that’s where we see this application fever taking off.

There are limitations to this data. I will point out specifically the limitation to the AAMC data is heavily relying on Step 1 scores. Family medicine programs are not as contingent on Step 1. I think they use it because it’s easy to measure.

AAFP News: What’s the challenge for programs when students apply to too many programs? What needs to happen to change that? And do you think that’s going to be a problem this year?

Brown: I think family medicine program directors always reviewed applications in a holistic way, but I think there’s been a dramatic change to increase that in the last several years. That’s been a huge focus of family medicine education, reviewing people not just from their numbers but what barriers have they overcome? What’s their story? Not so much where are you now, but where did you start and how did you get there? That is hard for program directors because there’s no checkbox to sort people for what their life journey has been.

We would love to holistically review all the applications, but that’s really hard when you get 1,000 applications. You still have to sort in some way, and program directors fear that we’re not spending the time on the students that are really interested in our programs, so you still have to sometimes default to more traditional measures of student excellence. This is a huge problem. How do you match student interest and passions with what a program can give that student?

I still don’t think we have a great resource. It’s hard for students to sort programs based on characteristics. Obviously location is not hard. Suburban, rural, urban is not that hard, but after that the students basically have to search our websites to find the information they need to match their passions, and that’s really challenging when there’s 711 programs. Our specialty has not solved that problem yet. All the things that we can do to improve application fever are not going to work super well until the students feel confident that they can find the programs that are the right fit for them without trying to interview at 100 programs.

AAFP News: How many applications did you get last year?

Brown: Compared to typical years, ours didn’t go up much. I don’t know the exact number, but I think for multiple years we have received between 1,000 and 1,500 applications.

Rutter: I think transparency among programs — what criteria are they using to look at students — is really helpful. That will help curb students from applying if they know, “We don’t have any students with X characteristic,” whether that’s a Step score or some other filterable thing. Then a student is able to assess their competitiveness for a program. When you think of certain undergrad institutions like Stanford or Harvard, you’re like, “They’re really hard to get into,” but there’s nothing like that for family medicine. There’s no communal understanding of competitiveness of programs, other than institutional memory: “Where have we matched students who are highly competitive before?” Or, “Where have we matched students who have had blips on their radar in the past?” That’s not fair to students, because if they don’t have an advisor who has been around for a long time, or they don’t have students ahead of them who has gone through that, then it is nearly impossible.

The other thing, though, falls on the advisors of undergraduate medical education, and I think that’s paring back the applications of students who are highly competitive. Can we encourage students who are going to need fewer interviews at fewer programs to have a successful match to accept fewer interviews to allow their counterparts who actually need more interviews and applications to be successful in the match to get those spots? I don’t have a formula for how that works. I think it’s just something that we should strive toward, guiding students as best we can so that the precious interview slots that a program has available are distributed to students who are appropriately fit for them and students that would be happy there. Nobody wants to see a match be unsuccessful and after a few months an intern decides this is not the right program for them. Nobody wants that. So, how do we strive toward getting people as happy as they can be and positioned to be the family doc they want to be in the program that is the right fit for them?

AAFP News: There are some pretty negative forums online. Are there websites that you advise people to avoid, and what resources do you think students can trust in this process?

Simmons: Reddit and Student Doctor Network are horrible places. It’s miserable, and it really freaks you out. I know that’s much easier said than done because I was on Reddit, too. I’m not here to promote anything, but for my year, especially applying, our GroupMe for Black applicants applying to family medicine was extremely helpful in creating community, and we’re doing it again this year.

Something that was a really good safe haven, and not an ethnicity-based GroupMe, was Twitter. I found Twitter and med Twitter to be extremely helpful throughout the match process creating a community of people who are applying.

Brown: I think Doximity is flawed. There’s some buzz among the program directors that you don’t give your data to Doximity. It’s not judging the right things.

We looked a few years ago at hits to our program website, and the pages that had our residents’ pictures and bios had 10 times more traffic than any other page. So, look at the residents. Where are they from? What are their passions? What are their backgrounds? Would do you feel comfortable there? Obviously, you can’t totally judge a book by its cover, but I think that’s a good way to find out about what kind of students match in that program.

Rutter: Look at graduates, too. Are there people who graduated from this program that are doing what you want to do in five years? You know, I want to be a hospitalist. I want to do OB. I want to do the addiction medicine, whatever your interests are. It’s following those passions.

The AAFP residency directory is really helpful. We send students to that all the time. For prepping for the Match in general, Strolling Through the Match. And I’m not just saying that because I’m in an AAFP group. Those two resources in particular are extremely helpful.

And again, building a community of people going into family medicine is helpful because part of the issue with places like Reddit is it’s not specialty specific. Students get pulled in by their colleagues who are going into dermatology and applying to every derm program. You’re not applying to every family med program; it’s apples and oranges. You just can’t even make that comparison. There are different numbers of programs. Competitiveness is variable. Again, I encourage students to work with family med-specific advisors. It doesn’t have to be a family doc, but it needs to be someone who knows about the family medicine Match. And use resources that are specific to family medicine because that’s where you’re going to get information that is relevant to what you’re trying to accomplish.

Simmons: In the Association (of Black Family Medicine Applicants), a lot of applicants look into whether a program has had a Black person or a person of color in the last five years. And honestly, a lot of times the answer is no. It’s not that you think you can’t go to that program, but it is discouraging. Nobody wants to be the first or only minority. When I was looking at programs’ websites, the first thing I did was look at the residents.

Rutter: Using residency fairs is a great way for students to get a sense of a program, and for one plane ticket or one login you’re able to meet a lot of different people. Students get really excited about the process and start to build a community of family medicine across the country, which may not be present at their medical school. How supported are they in applying to family medicine? Students choose to go into family medicine, but is there someone at their institution that can help them? I’m talking about that as an advisor, but there are schools that that don’t have the ability to have an in-person advisor who can really help them. I think building a network of people, like Dr. Simmons has said, can be really helpful.

AAFP News: Carmen, what was the most fun, or most memorable thing that happened on any of your interviews? How can programs make themselves stand out in a virtual process?

Simmons: I was fortunate to be able to do an away rotation. I was really freaking out about that as one of those students who had some red flags on my application. I was thinking, “How am I going to make myself stand out?” I interviewed there, as well, but I told them, “I 100% saw the same type of excitement and creativity y’all had in the virtual format as when I was at your away rotation. I don’t think anything would’ve changed about how I would have ranked you or how I engaged with the program even if I didn’t see you in person.” They sent boxes to the applicants, and we all opened them together. It was super fun. And that just kind of speaks to the type of program that they are.

For my own personal program, we had a Zoom game night that was fun. Obviously, I’m biased because that’s the program that I really wanted to go to. That was my number one. We had a really good time with that. I feel like a lot of places made things special. And you didn’t have to have a game night, a Zoom party, a gift bag or anything like that. You can tell what programs valued during their virtual format. Some programs want you to come do your work and go home, and that was kind of their culture, which is good for a lot of people. I’m very social. I talk a lot. That’s not kind of the program that I would thrive at. For a lot of people, that is exactly where they need to be. You can see how a game night, for some people, would be like, “OK, this program is doing way too much with a game night. It’s too involved. It’s too much.”

Brown: Cones of Dunshire?  

Simmons: Right. (Laughing.) Hey, we get it. Do we have to spend this much time together? No. I want programs to know that your personality is shining through.

Brown: For virtual interviews, students need to know that, yes, they need to be professional. Yes, students should have professional backgrounds, but also students are going to get a break from programs. If something unexpected happens in your virtual interview, which happens all the time, just roll with it, be positive and laugh about it. I had one applicant who couldn’t be at home for some reason. He went to his school, and the room had a motion sensing light. Every five minutes of his interview he had to wave his arms around. The fact that he was able to laugh about that was a plus. No program is going to be like, “Oh, my gosh, that’s the crazy kid with the motion sensing light.” Give yourself a break and enjoy it, and let your personality shine through.

Rutter: We had a similar story. I also do residency interviews, and we had a student who had a new puppy. We were chatting, and he was totally professional from the waist up. He said, “I’m happy to show you my dog, but you need to understand that if I stand up, I am wearing sweatpants.” It broke the ice. Everybody was laughing.

We thought virtual interviews wouldn’t be impacted by weather, but we had people losing WiFi because there were snowstorms or windstorms when they were trying to call in. We interviewed people from very different time zones, so we had to be thoughtful about it being late at night or early in the morning. There’s a lot that can influence the experience during an in-person interview, and that’s not lost in a virtual format. There are plenty of opportunities for interviews to go off the rails and to be guided back on. It shows you a lot about an applicant’s ability to roll with the punches.