Interviews: Is it Time to Blow Them Up? (Part 1 in a 2-Part Series)

I’ve always been curious as to why everyone continues to use interviews as a primary means of assessing candidates. Managers don’t like to do them, candidates literally hate them, and as a predictive indicator of performance, they stink!

“Interviews are a terrible predictor of performance.”

That quote isn’t an opinion, it’s a statement of fact based on the past experience of Laszlo Bock, Google’s Vice President for People Operations as quoted by Saul Hansell in the New York Times on January 3, 2007.

The validity of interviews is regularly criticized by corporate professionals and thought leaders who have studied the success of interviews for predicting performance. Despite this criticism, their position as the foundation of most organizations’ assessment system remains strong.

Historically, there is plenty of data to demonstrate that interviews possess a predictive capability akin to that of a Ouija board. Most academic studies these days are not commissioned to test validity, but rather prove validity. As almost any PhD candidate can attest, nearly any research study can be crafted to produce exactly the results vendors with a vested interest in the outcome need to bolster their sales collateral. Rather than diving deep into the problems of studies conducted to prove validity, I’ve decided to show you in some logical detail what makes interviews so weak.

Stuck in the Dark Ages

Interviews are so common that literally millions are conducted each day with little concern for their accuracy or effectiveness. Bloodletting and drowning witches were also once common, but luckily, we learned enough as humans to move past those practices.

Unfortunately, the standard interview has barely changed. As the global economy continues to emerge, and firms adopt new practices that reflect the realities of operating a modern global organization, how they assess talent will become even more critical.

Despite what HR directors trying to cover their behinds say, there are a multitude of cultural influences that affect how candidates from different geographic or ethnic backgrounds interview. In addition, due to decreased cycle times relating to the design, development, and obsolescence of goods and services, organizations today need less homogeneity and more diversity in thinking styles to drive innovation.

If organizations are to continue being successful, interview systems will need to be robustly audited to ensure that they are not consistently screening out innovative individuals who tend to act “differently” during interviews. Unfortunately, only one in 1,000 firms even attempt to gather data on the accuracy or reliability of their interviews.

If you want to stop assuming that yours work, review this detailed list of the top-10 critical issues with interviews:

  1. Some things should not be measured in an interview. The lack of agreement in advance as to what should and shouldn’t be measured via an interview is a serious problem. It turns out that many things just can’t be measured and shouldn’t be measured during an interview. Unfortunately, that fact doesn’t stop “independent-minded” managers from trying to measure inappropriate things, including body language, team skills, and intelligence.
  2. Interview questions are not directly related to needed skills. Each individual question needs to be tied to a specific job skill or knowledge. Unfortunately, most interview questions are developed independently and are not tied to any specific “required” skill or knowledge.
  3. No script or plan. There is no interview “script” prepared for most interviews. As a result, no two interviews follow the same plan, making robust comparison of interview data difficult.
  4. No weights. Interview questions are not weighted, so it is possible that the most important questions do not have the most influence on the final interview score outcome.
  5. No scoring methodology. Without a process to rate candidates’ questions and factors consistently, it is difficult to ensure that decisions are based solely on the factors to be considered.
  6. No agreement on good answers. Without a pre-determined idea of what makes a good answer versus a bad answer, candidates’ similar answers could elicit different “scores” from different interviewers.
  7. No accuracy check. The validity or accuracy of interviews is not checked by comparing whether those who received high scores during the hiring process turn out to be top “on-the-job performers” and vice versa.
  8. Managers are not trained. Managers only receive cursory training; therefore, they don’t know the pitfalls that can lead to bad interviewing and hiring results. Because “mystery shoppers” are not used, HR has no direct way of knowing what might be happening during an individual manager’s interviews.
  9. Interviews are inherently misleading. From the start, the basic foundation of the interview is based on the premise that during the interview, candidates are acting “normally,” which is unlikely because most candidates are scared to death before, during, and after interviews. In addition, we assume that candidates are telling the truth or that we make great lie detectors. I find that premise humorous because we all know that candidates routinely stretch the truth or tell us what they think we want to hear. The interview situation is by definition unreal: what happens during the interview might not be representative of what one would actually do on the job. The entire interview process is ill-conceived and designed to find candidates’ faults, as opposed to gauging a demonstrated ability to do the job.
  10. Subjective factors influence decisions. Numerous subjective factors (i.e., body language, race, sex, age, accent, height, handshake, dress, and physical disabilities) directly impact interview decisions. It’s a fact that individuals from different backgrounds and different demographic categories receive higher interview scores than others.

Common Interview Process Errors

The actual design of the interview process can cause many problems, including:

  1. Behavioral interviews have inherent weaknesses. Asking candidates to describe how they handled a certain situation in the past is problematic. First, the candidate might describe an actual event in which they were involved but did not play the role for which they are taking credit. Second, if their verbal descriptions or their delivery happens to be “clumsy,” their accomplishments will likely be understated (even though they actually did what they described). Third, the past is not always an accurate predictor of the future, and it is probably even less so in our fast-changing world.
  2. The interviewer. An interviewer’s training, sex, age, biases, and experience dramatically impact their assessment of any candidate. All too often, interviewers act like junior psychologists; as a result, they make snap but inaccurate judgments.
  3. Predictability of the questions. Because most corporate interview questions come from behavioral interviewing books or purchased methodologies, candidates can often predict what questions will be asked. Interviews become even more ineffective if candidates can guess and practice the questions in advance.
  4. Illegal questions and notes. Because interviews are not recorded, it’s not unusual for illegal questions to “pop out,” or candidates to inadvertently volunteer illegal or unnecessary information. Finally, the unfettered handwritten notes taken by interviewers can be embarrassing should they see the light of day in a court proceeding.
  5. No written record. Because most interviews are conducted without being taped or even with a written record, there is little evidence (should legal or EEOC issues arise) as to what actually occurred or didn’t occur during interviews.
  6. The time of day. Because multiple candidates are involved at different times of the day, it makes accurately comparing candidates’ interview results difficult. For example, I become more critical in my assessments as the day goes on.
  7. Consistent location. If the location of the interview is not consistent for all candidates, it might influence the candidate’s assessment (i.e., lunch interviews produce different results than conference-room interviews).
  8. Interview length. Interviews are often very short, making realistic assessment difficult. Due to time and business pressures, managers often eagerly make snap “first impression” decisions that are almost always inaccurate.
  9. Fit. Many managers use interviews to measure an individual’s “fit” with the team and the corporate culture. Unfortunately, there is little evidence that untrained managers can accurately assess this in 30 minutes.
  10. Practice makes perfect. Top performers who have been in a job for a long time might be rusty in their interviews skills, which could negatively impact their scores. Unemployed candidates who have recently gone through numerous interviews might actually benefit from their extensive “practice” and do better than the rusty interviewees.
  11. Knockout factors. Many managers seem to make up subjective knockout factors that prematurely and often unfairly screen out qualified candidates.
  12. Which jobs? Interview scores tend to vary based on candidates’ interpersonal and communication skills. Yet many jobs don’t require even average interpersonal skills. This means that some jobs (i.e., receptionist, salesperson, and recruiter) lend themselves to being assessed through interviews, while interviews for other jobs (i.e., welders, artists, and ditch diggers) may be horrible predictors of candidates’ on-the-job success because they work alone.
  13. Lack of technology. Interviews haven’t changed much since the Stone Age. Unfortunately, very few firms have successfully integrated video or online interviewing into their processes.
  14. Panel interviews. Panel or group interviews are often intimidating, making an already nervous candidate more nervous because of the number of people in the room hurling question after question.
  15. The order of the interview. If you are the first among all candidates in the interview process, you’re less likely to be hired than if you are the last candidate. Unfortunately, where you appear in the order of interviews impacts your odds of success.
  16. Hiring only for today and for this job. Hiring managers can be shortsighted and selfish. They frequently interview and hire based on their own short-term needs. Though companies should hire individuals for current and future needs, the reality is that most interview questions are not designed to assess competencies for other positions within the company.

As you have probably begun to realize, interviews can be attacked on many fronts despite all of the logical reasons that exist to use them. While this list could go on and on, it is important to move on and tackle how interviews can become a barrier to the productivity of the recruiting function.

Next week: The conclusion of this two-part series will look at how the interview process routinely fails as a customer-service process, and how all of these issues impact a world-class staffing function.

About the Author

Screen Shot 2015-05-19 at 9.41.40 AM
Dr John Sullivan is an internationally known HR thought-leader from the Silicon Valley who specializes in providing bold and high business impact; strategic Talent Management solutions to large corporations. He’s a prolific author with over 900 articles and 10 books covering all areas of Talent Management. He has written over a dozen white papers, conducted over 50 webinars, dozens of workshops and he has been featured in over 35 videos. He is an engaging corporate speaker who has excited audiences at over 300 corporations / organizations in 30 countries on all 6 continents. His ideas have appeared in every major business source including the Wall Street Journal, Fortune, BusinessWeek, Fast Company, CFO, Inc., NY Times, SmartMoney, USA Today, HBR and the Financial Times. He has been interviewed on CNN and the CBS and ABC nightly news, NPR, as well many local TV and radio outlets. Fast Company called him the "Michael Jordan of Hiring”, Staffing.org called him “the father of HR metrics” and SHRM called him “One of the industries most respected strategists”. He was selected among HR’s “Top 10 Leading Thinkers” and he was ranked #8 among the top 25 online influencers in Talent Management. He served as the Chief Talent Officer of Agilent Technologies, the HP spinoff with 43,000 employees and he was the CEO of the Business Development Center, a minority business consulting firm in Bakersfield, California. He is currently a Professor of Management at San Francisco State (1982 – present). His articles can be found all over the Internet and on his popular website www.drjohnsullivan.com and on www.ERE.Net. He lives in Pacifica, California.
  • Romuald Restout

    I agree that relying only on interviews as a predictor of performance is definitely a bad thing.
    However, were I told to keep one step only in my recruiting process, this would be the one I keep. Not the assessment one. Or any other for that matter.
    Reason #1: Purely personal. As a human being, I would feel really bad that hiring decisions are taken only based on a questionnaire, whatever the quality of this questionnaire is.
    Reason #2: You can have the greatest talent in town, if s/he cannot build a relationship with her/his boss, it is simply wasted talent.
    Reason #3: Interviews build accountability. If you rely only on an assessment to take the hiring decision, you can always blame the process, the algorithm, the assessment designer, the machine, a bug, … anyone, anything but yourself.
    So, are interviews the top mechanism to hire talent. Definitely not. As you point out in your article, there is room (a lot of it) for improvement.
    Blow them up? I don’t think so.

  • Bryan Baldwin

    Agree with vast majority of what you say. However, research has shown that interviews CAN be very good predictors of job performance, if conducted in the right way, implementing all the factors you mention above. Unfortunately this results in a process that is often unpopular with candidate and interviewer alike.

    I believe the problem isn’t so much with interviews as a method, but with how they’re used. If interviews consistently fail to predict performance, it’s because preparing and administering them wasn’t taken seriously enough. Organizations spend months, sometimes years, on creating written tests; many interviews are thought up the morning of the interview. In addition, everyone believes they can interview someone well, but the reality is it’s a skill–and not an easy one.

    Which raises this question: If it’s so difficult for people to conduct good interviews, should we continue to place so much weight on them? Probably not. But is this realistic? What is the alternative?

  • Jonathan Hefferlin

    To the ‘Good Doctor’

    Since the last 5 placements I made (5 different cos) involved only one candidate each, I am a very big fan of prep, deliverables on both side of the table, ‘gut feel’, chemistry, etc. & I’ll match my candidate longevity against anyone’s. I wonder what happened to, ‘If the chemistry is there, it will happen if they have 60-70% of what the perfect candidate needs (skill wise)’? And if the chemistry is wrong, they can have 100% and not get hired. Your system (whatever it is) could do wonders for office morale.

    On your 1st 1-10 list, which I wonder what it is all leading up to, (suspense is great hype for part 2), on #1 are you going to measure chemistry in a tube and deprive interviews with potential bosses and co workers? And I agree with #2-9 and #1-16 on list 2, and wonder where it is all going. I betcha you could make a case that a camel couldn’t make it across our 30 car parking lot in the summer time, in cool Dana Pt, CA.

    I love your analysis, though, & seriously look forward to Part 2. -Jon

  • Alexander Brown

    I think we can all agree that interviews are not optimal but what is the alternative? Interviews are useful when used properly. Just like phone screens, just like references. There is the need to continually tweak the system, think of Microsoft. Hackers constantly try to crash the system for glory, the thrill or boredom. So Microsoft is always trying to guard their system. Talent can become professional interviewers, hide their true nature behind phone screens and they know that references will only give ?basic? information.

    Our jobs as ?professionals? are to find ways to get through the defenses and get to the information. Preparation is key. Know what you want to accomplish, and guide the talent to your goal.

    I think the flaw in interviewing is that we, the interviewer, can become complacent in our approach. There is information that can be derived from a good interview. Interviews are used in law enforcement because it is a way to find facts that are hidden. The talent are on a fact finding mission as much as we are, ?Quid Pro Quo?.

    Preparation, practice, and developing our skills as professionals are the keys to successful interviewing. The honest truth is if you are not a good interviewer, than how can you ever become a great recruiter. The ability to evaluate talent is a key skill.

  • Jim Cargill

    With the advent of consistently more ‘sophisticated’ (i.e., complex and time-consuming) interview models, business has failed to improve the quality of the hire. Rather, hiring decisions take longer, involve more people who are unskilled, and worse yet, uninterested in contributing to the hiring decision, and, lastly, erode the ownership and responsibility of the candidate’s immediate supervisor.

    Interviewing is certainly an inexact science, though handled correctly, it is still the best indicator of potential success of the candidate interviewed. Why not look at today’s interview model as the culprit, rather than interviewing, in general?

    Why do we need the opinions of 3+ peers, 3+ subordinates, and bosses more than two levels above the candidate position? Is there anyone brave enough to say that it’s possible many of these interviewers not only lack interview skills, they may lack objectivity, as well? And while all these people are involved, usually requiring a 100% thumbs up vote to hire, what is the buy-in of the manager? It is no longer their decision to make, nor their butt on the line to ensure that person’s success. When did we stop requiring manager’s to take responsibility for their decisions?

    You want more successful hires? Train basic interview skills to Managers. Make them responsible for the decision they make. For non-Manager positions, have candidates interviewed only by their prospective Manager, that person’s boss, and an HR ‘interview guru’, whose job it is to keep the interview in line, and rate and train interviewing skills. For Manager and Director positions, add someone from outside the department who is typically considered a mentor, and who has demonstrated objective capability.

    Dump the panel interviews. Dump the all-day marathon interviews. Dump the hourly staff interviews (they are not paid to make these decisions for you, and all you do is give them a false sense of power, which you will not support when push comes to shove!)

    Interviewing should be handled with a Capitalistic approach. Those who do it well should be rewarded. Those who do not should be sent back to Remedial Interviewing, or not involved in the process, period.

    When interviewing becomes a social project, as it has in most of corporate America, the result will never equal the expectation.

  • John Aruzian

    Jonathon:

    I just lost a bet with my Associates re how long I could keep my mouth shut and not respond to this informative chit chat (see previous email).

    I was really pleasantly surprised to read that anyone in the Recruiting business had hit a 1 to 1 ratio of candidate submittals to hits!

    In the contract engineering business many years back, when the ratio was 7 to 1 industry wide among CE firms (NOT among ‘permanent placement’ agencies which was more like 15 to 1), our best ratio over a given period of time was, if memory serves, 1.3 to 1. For you to achieve, even for a short period, 1 to 1, is incredible, and shows that you probably (am guessing here) know the technology of the industry & discipline, understand the corporate ‘culture’ (new buzz word I guess), can properly ‘interpert’ those incoherent ‘job descriptions’ from HR (unless you are dealing with real hiring Managers directly), and finally can ‘interpert’ the EEE of your Candidate in real time terms.

    The personal ‘chemistry’ aspect is, unfortunately, a final and important aspect. I am not saying that some chemistry between a hiring Manager and her/his direct report is not a factor, but it can, and does, frequently override the reality….in both directions. I have seen totally unqualified Candidates hired due to chemistry, and totally qualified Candidates not hired due to chemistry. In fact, when HR first became a factor in the process, qualified Candidates were shot down for totally irrelevant reasons.

    As you are based near one of our facilities (S.D.), contact me some time and we’ll discuss our (patent applied for) ‘Relevant Obstacle Survival Technique’ as utilized in Recruiting (it has many other applications).

    I would like to share one example with ya’ll of an extremely sharp Personnel Manager named K.D. McKenna (retired) who was with ‘General Electronics Laboratories’ in the Boston area back in the early 60’s.

    Kenny had a req from Marketing for a Technical Sales type with EEE in electronic components to cold call on systems manufacturers. After screening OUT (our real job in Recruiting) many candidates, Kenny narrowed it down to 3, and he scheduled all 3 for a noon appointment. At 11:45, Kenny had his Receptionist, Secretary, and 3 Assistants go to lunch, leaving their respective office doors open, but shutting his.

    He then waited till the 3 Candidates arrived and took a seat in the reception area. After 10 minutes or so, 1 of the 3 had enough gumption to knock on Ken’s door to find out who was around and what was gong on. You know which 1 he hired!

    By the way, whatever you are doing to achieve a 1 to 1, DON’T CHANGE IT! The number of ‘fees’ is NOT what’s important, it’s the low submittal to hit ratio which counts with the Employer, and the no nonsense, dignified treatment of the prospective Employee which really matters…especially if you want to use the term ‘Professional’ as a Recruiter!!!

    End

  • Jonathan Hefferlin

    John –

    No, its not technology, but 5 hits in a row IS LUCK. You hit streaks where you do nothing different, but it builds confidence and expectations (‘Wow, 2 in a row, lets try 3′)

    Actually it was 6 of 7 (!) – the miss was I deleted my one candidate hired, but it was a PL where 13 had been submitted by another(former)AE here. It was 5,1 & 1 with another (!)hit (we call them ‘Bingos’) with an offer being prepared. Good inventory, and one was, ‘You mentioned you had a guy in Canyon Country a while back’ for a job that I didn’t know was open.

    The key is, and I probably mis-spoke, it was 6 PL where they hired the only candidate THEY AGREED TO INTERIVEW, not presented (sorry). I didn’t count cancelled JOs, or when they hired an internal, lost or delayed the project (its construction), etc. And when they have a ‘planned but unmanned’ project, urgency can contribute to these streaks.

    If you find them, they should interview them. Well, not quite, maybe 2 of 3. Great clients will infact interview every one (I have a couple of those). As an average, over 10 years here its 6:1 JO/PL, 7:1 SO-1/PL, and 4:1/PL SO-2+, more like 3:1, 4:1 and 2:1 for more seasoned folks with over half the candidates presented interviewed.
    Jon, the #s guy

  • John Aruzian

    Jonathon:

    Yes, I know the element of luck…but the record still ‘ain’t half bad’, ya know? I’d take it.

    You must be a sharp and straight up guy to to know how things happen and admit when it isn’t all your wonderful personality, exceptional brains,fantastic experience, etc., but a little dumb luck mixed in. Don’t change!

    Call me when you’re in the Sun Diego or Las Vegas or Austin area.

    JCA

  • Pingback: The Interview Translator : ERE.net()