Hiring is Broken And Yours Is Too

I can’t keep track of the number of articles I’ve read about hiring in the past few years. Inevitably, they all follow the exact same format. First, they claim that hiring is broken. Next, they describe the hiring practices used by XYZ company. The present an extremely thorough analysis of every possible way in which such a practice can be gamed, or produce bad results. Finally, they wrap up by saying that hiring should instead be done on the basis of ABC criteria, only for someone else to write an essay the following week attacking ABC as a broken hiring method.

At this point, I’ve seen almost every single combination of ABC and XYZ. As a fun little exercise, I thought I’d take some time to compile every one of them.

I’m sure lots of people have very passionate opinions about the right and wrong way to hire. Quite frankly, I’ve lost all interest in hearing people’s opinions and anecdotal experiences. Until and unless someone does a rigorous scientific study evaluating different interviewing techniques, preferably using a double-blind randomized trial, there’s no point in beating this dead horse further. Everyone’s hiring practices are broken, and yours aren’t any better.

Look at which University they attended

Pros:

  • University admissions departments spend an enormous amount of effort and manpower in identifying and attracting the most talented individuals. Why not leverage this investment and expertise for your own company
  • Top universities generally speaking have top professors, other top students, and challenging course content, all of which will rub off on the candidate
  • Getting admitted into a top university usually takes a lot of hard work and intelligence. This can be a good proxy for job success

Cons:

  • There are idiots present in every university. Yes, even at Stanford
  • University admissions unfairly favors the wealthy and well-connected. By rewarding this, you’re discriminating against the poor, the not-well-connected, and minorities
  • The skills required for getting into Stanford at 17 (extracurriculars, SAT prep etc) do not correlate to job success as a software developer
  • How good a student you were at 17, is not very relevant to who you are at 25

Look at which companies they’ve worked at in the past

Pros:

  • See all the pros listed above, and replace “Stanford” with “Google”
  • Companies are usually better proxies for the technical skills you need in software developers

Cons:

  • Those companies have hiring practices that are broken (hence why you’re writing this post). Therefore, you shouldn’t be relying on their hiring process
  • This doesn’t work for new-college-grads
  • You’re creating a self-perpetuating loop where someone who got rejected by Google the first time, will never get a 2nd chance ever again

Look at their GPA

Pros:

  • It’s a very comprehensive metric, which takes into account performance across a large number of courses and exams, over 4 years, covering a mix of homework, projects and exams
  • It’s a good proxy for how hardworking and dedicated an individual is
  • It’s a reasonable proxy for how smart an individual is

Cons:

Ask them for references

Pros:

  • Gives a view of their actual on-the-job performance

Cons:

Do back-channel references

Pros:

  • Gives an unbiased picture of actual on-the-job performance

Cons:

  • You will be considered evil for doing this
  • Hard to consistently find the right people who can provide accurate references
  • Rewards people with great social/networking skills
  • Punishes people who’ve worked with bad/abusive managers in the past

Take home projects

Pros:

Cons:

Audition for the role as a contractor

Pros:

  • Actual on-the-job work
  • See what they are like to work with
  • See their actual work product

Cons:

  • Depending on their current employment agreement and immigration status, this could be illegal
  • You’re asking them to take a week of holiday, just to work for you. People will hate you for doing this
  • Discriminates against people who have lots of companies showing interest. They can only afford to do this audition at 1 or 2 places
  • Discriminates against people with lives and families

Ask them to describe their past projects

Pros:

Cons:

Ask for source code from significant projects

Pros:

  • Actual work product, that’s a good reflection of their capabilities

Cons:

Quiz them on their knowledge of various technologies, tools and frameworks

Pros:

  • If they know about, and how to use technology X, they will be able to use X to successfully fulfill their job role
  • Shows that they care enough to research this stuff

Cons:

Brain teasers

Pros:

  • Tests their overall intelligence and problem solving skills

Cons:

  • Either you know it or you don’t. Relies on you being lucky enough to get a flash of inspiration, or you having heard it before
  • Not directly related to programming
  • Even Google says it is useless

Live Coding Exercises

Pros:

  • Live test and demonstration of coding skills, that’s not susceptible to the above cons
  • The fact that FizzBuzz is so effective, demonstrates the need for more in-depth live coding tests
  • Completely fair and impartial. If you solve the problem, you’re scored highly. If you don’t, you’re scored poorly. Such meritocracy

Cons:

Pair programming on a “real” problem

Pros:

  • You’re working on a real problem, that resembles the day-to-day responsibilities
  • There is no “known-right” answer. Collaborative exploration

Cons:

  • It takes developers weeks to get ramped up on a specific code-base, domain expertise, and technology stack. Expecting someone to drop in one afternoon and start making an impact, is not very realistic
  • Highly variable. Depending on the day they come in, some candidates can be given very easy issues to work on, involving languages/domains/technologies they’re already familiar with. And others can have the complete opposite experience.
  • In order to assess someone’s potential, you have to give them problems that push them to the limit, even if it isn’t “realistic”. There’s a reason Intel CPUs are evaluated using synthetic benchmarks, and not MS Word
  • If you gave every candidate the same “real issue”, of challenging but not insurmountable difficulty, in a sandboxed environment, requiring minimal prior domain-knowledge, with flexibility on language/technology-choice: Congrats, you’ve reinvented live coding exercises.

The german translation of this essay has been published in golem.de.
Discussion thread on Hacker News.
A variant of this essay was earlier published here.

21 thoughts on “Hiring is Broken And Yours Is Too

  1. Pingback: Is Hiring Broken?
  2. This is a very comprehensive and accurate article. I found it on Hacker News (searching for this month’s hiring thread).

    Let’s not overlook what I believe and fear is a nontrivial component to hiring practices: investigating your online presence. Googling your name, looking for you on Twitter, Linkedin, GitHub, your personal site with your professionally-taken headshot, etc.

    During a previous interview, two of the interviewers asked me how I was acquainted with certain individuals who were in my Linkedin connections. To be fair, I sensed that they were just curious, and weren’t making any kind of decisions based on the question. However, I found it alarming that they looked at my connections at all, as if my legitimacy were tied to the legitimacy of people I’ve met. If they’re looking at such things, other employers are looking at such things, and I must say that I don’t like it.

    I’ve gone out of my way to make myself appear in a Google search as little as possible. But if employers can’t pry into my life, and they can pry into the lives of my competitors, they’re going to go with the candidates about which they can immediately ascertain more, they’re going to go with the candidates who have a pretty face in their avatars. If there year were 1999, they would have nothing to compare except our two resumés.

    One could probably write an entire post on online presence in relation to hiring practices and overall employability.

    Liked by 1 person

  3. Awesome list. Yep, hiring is not easy. The approach we have is a combo of the above things, the times we have been more “rigorous” are the times we good good people. For us what works is , 1) CV scanning 2) phone interview with some technical questions. 3) take home exercise 4) phone/face to face interview discussing take home exercise and more tech questions.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s