Question: when hiring engineers, do you look for positive reasons to hire someone (they’ve done something good, have a good capability, a particular strength or good potential), or do you look for reasons not to hire someone (not enough experience, no degree, failed a question you asked)?
On some logical level, it seems a meaningless distinction. People are either good enough for you or not, and whether you find the good ones directly, or implicitly by eliminating the bad ones, wouldn’t seem to matter. In practice, however, there’s a huge difference, and something I believe strongly:
While positive reasons are important, elimination is the primary basis of good hiring
Positive reasons to hire someone tend to be unique to the individual. Evidence of high performance in their past is clearly very specific to them. When you look at people’s strengths, they have unique capabilities or combinations thereof.
This matters because evaluating someone accurately in a short space of time is hard, and one of the most powerful tools at your disposal is relative evaluation. Choosing the best of 4 candidates allows you to be far more confident than deciding whether an individual is good enough in isolation. Performing relative comparisons is best done using standard criteria, and once you have those it becomes easy to eliminate based on them.
Looking for positive reasons to hire someone takes more time. To discover how good a contribution someone made to a past project, you need to spend a lot of time digging (asking them questions, chasing up colleagues, or looking at the code if it was open source).
Evaluating submitted demos takes time most game studios don’t have. Asking open-ended questions that give people a chance to excel, and judging their performance on these questions, takes much longer than setting them problems with right/wrong answers. Attractive places to work get overwhelmed with applicants, and hiring managers have lots of other work to do!
An efficient elimination-based approach involves starting with a large pool of applicants and finding the simplest possible relevant test that can eliminate a large proportion of them. Repeat until left with a small number. At each stage, you can afford to spend a bit more time evaluating the candidates. Eventually, you get down to few enough that you can afford to spend real time to look at their positive qualities, and your tests have heavily reduced the chance of them wasting your time.
#3 Bad hires are disastrous
Hiring the wrong person is like bringing a cancer or a poison into your team. It takes a huge amount of time and effort to fix the problem and they drain energy from your existing good employees in the meantime. Given the choice between two types of hiring mistake: (1) eliminating a candidate that would have been a great hire, and (2) hiring a bad candidate, mistake (1) is far better. Mistake (1) can be fixed, because there are always more candidates of sufficient quality. Your project might be delayed, but it’s generally not disastrous.
All of this means that a good interview process has to be very sensitive to possible bad attributes. You accept that it sometimes eliminates good people, provided it doesn’t let bad people through. Setting up multiple independent elimination hurdles is a great way to do this. Let’s say you have a test that a bad person could pass 10% of the time. That’s no good on its own. But put 6 independent tests in a row with the same 10% failure rate, and the chance of a bad individual getting through them all is 1 in a million. That’s more like it (actually, it’s unrealistically high … but you get the idea) 🙂
Just for the sake of argument, suppose that same test tends to be passed by good candidates 90% of the time. Their chance of getting through all 6 is then about 50%. So you will eliminate quite a few perfectly good candidates, but if you start with a few good applicants, you’ll make a hire.
You often see job applicants get disheartened by this. The classic situation is where someone works their heart out to do something great, advertises this boldly on their CV, and can’t understand why they get eliminated for seemingly niggly reasons. Take this tale for example.
Don’t take elimination personally. Keep the above reasons in mind to understand why the elimination approach is important to the employer. It doesn’t necessarily mean you’re not good enough. There are plenty of jobs to apply for, and most companies won’t have a problem with you re-applying after a while. If you’re good, you’ll get past those elimination stages soon enough, and then you’ll have a chance to show your strengths 🙂