Tests Are Better at Hiring Than You

Assessment test

When hiring managers and recruiters use their discretion to overrule the results of assessment tests, their hires do less well than workers selected exclusively on the basis of test scores.

After studying the tenure and performance of 300,000 hires at 15 different companies, a trio of researchers concluded that “firms can improve worker quality by limiting managerial discretion. This is because, when faced with similar applicant pools, managers who exercise more discretion (as measured by their likelihood of overruling job test recommendations) systematically end up with worse hires.”

Last year, when Matthew Jeffery and Andrea Woolley declared in an ERE post that, yes, an algorithm can replace a recruiter, they were describing how the SAP recruiting team democratized its university recruitment. Their provocatively headlined post focused on SAP inviting students everywhere to apply, then using assessment testing to cull without recruiter intervention.

That process created what Jeffrey and Woolley called a “meritocracy of application” from which hiring managers selected their hires.

Now, in a research paper published by the National Bureau of Economic Research, the evidence shows that at least for low-skilled work, assessment testing improves job tenure by 15 percent. Human intervention, the researchers found, was strongly correlated with poorer results.

When managers and recruiters chose candidates based on considerations other than their test results — interview performance or resume qualifications — their hires were more likely to quit or be fired.

Why is that? Bias or poor judgement, say researchers Mitchell Hoffman, of the University of Toronto’s school of management, Lisa B. Kahn, from Yale University’s School of Management, and Danielle Li, from the Harvard Business School. “Managers exercise discretion because they are biased or have poor judgement, not because they are better informed.”

To demonstrate that, the researchers examined the productivity of workers hired out of the pool of top scorers on tests against the productivity of workers who were hired despite their poorer scores. They found no difference in productivity among the different groups.

But when it came to longevity, those hired out of the top of three pools of workers assigned according to test results stayed on the job an average of 29 days longer than those at the bottom. “Better test scores predict longer tenures,” said the researchers, who noted that longer stays lowered costs for the client firms, which provide from a week to two months of paid training.

Managers’ poor hiring decisions, the researchers say, “generates increased turnover in a setting where workers already spend a substantial fraction of their tenure in paid training.”

The research only applies to hiring for such low skilled jobs as data entry, call centers, and standardized test grading.

  • Logan Meece

    Hi John,

    With headlines like that you should be writing political articles…seems a bit disingenuous. Curious, if it is unskilled positions and there is no apparent difference in productivity between the two groups…what exactly are these assessments scoring?

    • http://www.fordyceletter.com John Zappe

      Per hour or per unit productivity was basically the same among the groups. BUT, those who scored in the top tier stayed on the job, on average, a month longer than those who scored in the bottom tier. That translates into significant savings in recruiting and training.

      When a recruiter or hiring manager ignores a top tier scorer in favor of a lower scorer, those hires have a significantly shorter tenure.

  • http://www.automatedats.com Dănuț

    Thank you for sharing this, John.

    Having an IT background and from previously administering an ATS, I can confirm that bias is the number one problem in hiring through human beings. (plus let’s not forget at least one in four applicants “bias” at least once in a resume)

    The advantage of software algorithms is that they structure the screening process (eventually through pre-screening questions) and there is much less space to “bias” there. Automating the full hiring process will never be feasible, for serious jobs, because it will generate high turnover due to incompatibilities in soft skills.

    Now with the mobile shift, there will be more and more pre-screening questions also for pre-screening processes measured by humans. This will hopefully fuel a new trend in evidence based hiring through people analytics.

    Regards,
    Danut Croitor
    http://www.hranalytics.expert

  • http://www.medievalrecruiter.com/ Medieval Recruiter

    Certainly interesting, and I’d have to read the actual study to see if it holds up, but it doesn’t surprise me. It’s been known for a long, long, LONG time that people make decisions based on what would be called irrational and irrelevant criteria. And then confirmation bias and logically fallacious thinking just reinforce their decisions over time. Most people don’t question their decisions, they rationalize them. That the productivity of both groups didn’t differ is interesting, though. But, thankfully, and finally, someone is trying to actually link hiring practices to actual on the job performance and tenure. Maybe next project they can finally put a cork in all this ‘passive candidate’ bs everyone is always buzzing about.

  • http://lensa.com LensaHR

    Good points there. I think that’s really key that the evidence they found is for low-skilled workers. It’s also a great starting point for a discussion.

    For more complex jobs, the use of soft skills assessment tests can be a solution to complement the screenings by the ATS’s, and to provide further information about possible incompatibilities with the company culture to the recruiter.

    So if studies like this imply the need for further automation is not bad at all. Also, I would like to see a similar study for high-skilled workers to reveal how long is the way to go till we’ll have an assessment method that outperforms even the most biased recruiter.

    Aniko Jori-Molnar
    Lensa

  • http://lensa.com LensaHR

    Good points there. I think that’s really key that the evidence they found is for low-skilled workers. It’s also a great starting point for a discussion.

    For more complex jobs, the use of soft skills assessment tests can be a solution to complement the screenings by the ATS’s, and to provide further information about possible incompatibilities with the company culture to the recruiter.

    So if studies like this imply the need for further automation is not bad at all. Also, I would like to see a similar study for high-skilled workers to reveal how long is the way to go till we’ll have an assessment method that outperforms even the most biased recruiter.

    Aniko Jori-Molnar
    Lensa