What's Broken About Tech Interviewing and a Few Crazy Ideas
An analysis of the current tech interviewing process, and some ideas to test.
Like the famous Winston Churchill quote about democracy, I think that the current popular tech interviewing process is the worst form of interviewing, except for all the others.
I've done over a thousand interviews. I was a bar raiser core member at Amazon in multiple groups. I've dedicated a good portion of my career to interviewing candidates and writing about interviewing. Large tech companies need scalable processes, and have a larger risk profile than smaller companies. Therefore innovation tends to lag in the area of interviewing. Thus the proliferation of things like HackerRank. I'd like to take a swing at a few ideas which I think would be worthwhile to explore.
This is my second paid article following the "Dave's opinions" style of article. Please reply to my newsletter to let me know how it's working for you. And apologies in advance, so far these opinion articles are ending up longer than my normal articles.
Existing company processes
Most companies follow (at a high level) a process somewhat like the following:
Discover (in a variety of ways) an attractive resume.
Do a single interview (~1 hour) to decide if you should be serious with this candidate.
Do multiple interviews (~5 hours) to decide if you should hire the candidate.
Make an offer and hire.
For tech interviewing in particular, those multiple interviews often include technical assessments of a candidate's skills. These functional tests can take the form of HackerRank style coding tests, and/or whiteboard design exercises.
What's wrong with this process?
It's stressful to be interviewed so many times (multiply all the companies you're interested in with all the interviews each company requires).
Every interview has a unmeasurable (*mostly - small deep dive below) false positive and false negative rate.
Perhaps most importantly - The interviews don't necessarily measure your ability to be successful doing your real job. Being good at HackerRank style coding tests has very little correlation to being good at building high quality software on a team. It's simply a skill you can practice to defeat the interview process.
So a great process would interview you fewer times than current processes, be accurate enough (whatever that means), and more accurately measure your ability to be successful in the job.
* (from above) How would you measure the false positive and negative rates?
False positive - You hire people who end up failing.
False positives are hard to measure because performance management is already an uncertain science. Managers are inconsistent, and measurement of work quality is based on opinions and personal judgement. However, if you said that employees with >2 years tenure were being fired companywide at a 5% rate, and newly hired employees were being fired at a 10% rate, you could guess that your false positive hiring rate was something like 5%.
False negative - You reject people who would be successful.
False negatives are impossible to measure because you never got to see them perform. The only way to measure a false negative is to actually hire people when they've failed the interview loop, and then compare their rate of underperformance to an employee who passed the interview. A point of which I'll get to lower in this article.
Hiring Process Requirements
You need to have a consistent hiring process at your company, because you're at risk of discrimination if you invent a new process for each hire.
You need to filter out bad hires, because bad hires are terrible for productivity. I've certainly interviewed some people who didn't belong in the software development field. So you do need a process which does some amount of filtering.
You need to keep the interview process short and cheap, because if you're hiring a large number of people, any expense becomes expensive.
Compare potential hires. Unless you're Amazon, you probably can't hire every qualified person who walks through the door. So you need a way of telling if someone is better or worse than your average potential hire.
FYI - Not an actual proposal
I don't think I'm writing a proposal for hiring. Writing a fully formed proposal would suggest that I have more data and answers than I have available. Instead I'm going to provide a list of ideas to test, some of which may or may not lead to better hiring processes. Not perfect, just better.