How the Amazon Bar Raiser Interview Process Works — Raising the Bar
Improving the imperfect system of choosing your next co-worker. The Bar Raiser process at Amazon scales one of the worlds most prolific hiring pipelines.
We are terrible at picking which people will become great co-workers in the long run. We review resumes and look for relevant experience, while combing through overstated or poorly explained projects. We attempt to test technical competence which took years to build through a few hours of observation. We ask for references, and pretend that feedback from specific named sources is remotely unbiased.
Beyond attempting to see if a candidate can accomplish tasks, we also need to make certain they have the right leadership qualities for our position.
Me - So what's the main reason you'd like to transfer teams?
Employee - I can't work with co-worker anymore. They're constantly screwing up. They never admit that it's their fault when their code breaks, then they fingers at their co-workers who missed their bugs. In general it's making life on the team miserable.
Employees need to be able to get along with co-workers, graciously admit mistakes, give credit when it's due, and at the very least, not make the work environment worse through their presence.
Finally, we need to accomplish the above without mistakenly bringing personal bias into the process. Beyond illegal bias (e.g. gender, or race), we need to be careful about our biases regarding education, grammar, the types of companies someone has worked at, and someone's choice to spend the previous year surfing instead of working. We are all biased in a thousand ways, and we need to be cautious about which evaluations are valuable, and which are not.
Interviewing is hard. It takes dozens of interviews to feel remotely competent. It takes hundreds of interviews to begin to feel some aspects of mastery. Amazon invented a bar raiser position to bring mastery and experience into every interview loop.
What is a Bar Raiser?
A bar raiser is an experienced person added to an interview loop to ensure that it does not go horribly off the rails.
Stated more optimistically, a bar raiser is responsible for ensuring that a candidate has a great experience, and Amazon makes a great hiring decision.
The bar raiser is an experienced leader. Their common sense and judgment is trusted by their leadership team, and they have the responsibility to correct or modify elements of an interview process. Rather than a checklist of responsibilities, a bar raiser impacts the entire process from end to end. It is not a clear defined role, rather it's a leadership position which requires an understanding of what ownership means.
Pre-Bar Raiser
Prior to a Bar Raiser being involved, a few steps have taken place.
A resume has been screened to start, because generally you would not conduct any type of screen without knowing a candidate's background. Often these days you're getting a better formatted version of someone's LinkedIn profile.
After a resume has been flagged as interesting, someone conducts a screen. The primary purpose of this screen is to validate that a full interview should take place. This is for the benefit of the candidate as well as the interviewers.
Recruiter - Hey, your interview is next, right Dave? You're the third interviewer in the loop?
Me - Yeah, why?
Recruiter - Something went wrong with the initial screen. The candidate did terribly in the first interview, and are bombing the second interview right now. The current interviewer says the candidate has started to cry.
Me - Ok, I'll talk to them and see if they'd like to leave early.
No one wants a candidate to have a terrible time. It's one thing to make a mistake or two while interviewing. We all regret the answers we give in the spur of the moment, and I've received dozens of emails from candidates after interviews explaining that they have a much better answer now. However, interviewing for a position you're clearly not qualified for can be a terribly embarrassing experience. The screen, often done over phone or video chat, is a way of ensuring that the candidate will be prepared.
One way of phrasing the qualification is that we say that a candidate should not have all not-inclined in their full loop. If we believe they have a shot of being hired, and would likely pass one or more in-house interviews, that should be considered a pass. A screen is not a "I think we should hire this person", it's a "I think there's a chance this person could pass an interview" decision.
Pre-Interview
Once we’ve verified someone is generally qualified, we setup an interview. Interviews vary in size, but we often have five interviewers on a loop. One of the interviewers would be the hiring manager, one interviewer would be the bar raiser, and three would be there to help assess the candidate. Each interviewer is assigned what are called "competencies” by the hiring manager to prevent too much overlap as well as ensure we assess as broadly as possible.
As the loop is being set up, the bar raiser surveys the competencies to ensure that they're the right ones for the role. Hiring managers are often not as experienced as the bar raiser on the loop, so the bar raiser often acts as a consultant to help ensure the right things are covered.
Me - I see in your competencies, you assigned "culture fit" to one of the interviewers. Could you explain what you intend by that competency?
Manager - We have a great team culture. We hang out after work together. We sometimes play sports, sometimes go out drinking. In general I want to make certain future team members will fit into our group.
You can't filter hiring based on someone's interest in drinking, how much fun they'd be to hang out with, or if they'd fit into your team culture. The bar raiser is responsible to watch for incorrect or risky competencies being assigned, and help educate the hiring manager on what we do and don't screen for. Even if "culture fit" sounds innocent, it has a number of dangerous side effects.