Google calls time on weird interview questions

In a recent interview with the New York Times, Google’s Senior VP of People Operations Lazlo Bock called time on weird questions. To quote:

“On the hiring side, we found that brainteasers are a complete waste of time. How many golf balls can you fit into an airplane? How many gas stations in Manhattan? A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.”

Now, just because these questions don’t work at Google, doesn’t mean that they won’t work in all companies and with all types of roles.

That said, good on you Google. You experimented, you checked if what you were doing was working, and then you optimised your process. That’s the way it ought to be done.

We talk in the book about the fact that unlike that gleaming new iPad, talent assessment methods and tools require optimizing and should not be expected to work perfectly straight out of the box. And we also talk about the importance of getting interviews right, given that they are the single most frequently-used assessment method. Yet very few firms do what Google have done.

To replicate it, you would need to be using a semi-structured interview format, in which all interviewers asked certain questions and then rated candidates’ answers to them. You’d need to collate those ratings in a central spreadsheet and then maybe nine months later follow up by looking at the relationship between the question ratings and individuals’ performance ratings. Of course, most firms don’t collect such detailed data, so what can they do?

Well, if your interviewers make capability ratings during interviews (and they should be) then you can collect that data and then nine months later check how strong on average the relationship is between each capability and performance ratings (in other words, which capabilities are most predictive of performance, and which the least). You could then adjust your interviews and the capabilities you measure accordingly.

And what if you don’t collect any ratings, just an indication of hire or no-hire? Well, one thing you can do is to check who seem to be the most and least predictive interviewers, through looking at the performance ratings of the people they hire. Or you could ensure that you always follow up the “failure” of new hires with a quick review meeting involving HR and the hiring manager or recruitment lead. During the meeting, the information available at the time of hire can be reviewed to check to see if any potential signs of failure were missed. The idea is not to lay blame but to have a genuinely curious inquiry with the aim of helping people improve their selection skills.

My point here is that this does not need to be just a ‘Google thing’ – something that only they do. Everyone can do this. It doesn’t take a lot of time and it doesn’t take a lot of money. For the most part, it just requires the will to do it.

I have mixed feelings about Lazlo’s comments. On the one hand, it is definitely time to stop with some of the crazier questions. “If you were an invertebrate, what type would you be?” Or “Tell me a joke”. I’m not convinced that these are indicative of anything much, beyond someone’s ability to deal with weird questions. I can see why people might think they are predictive of things like creativity and ability to think on your feet, but I’ve seen no evidence that they really do measure those things, and I am concerned that the weirdness of them interferes with what they are supposed to be testing.

But on the other hand, I am also equally unenthused by “Tell me about a time when…” questions. With some more junior level candidates they may be ok, but they can make interviews feel like oral exams and with more senior candidates that’s just not good enough. One or two of them is fine, but a whole interview based on them gives a poor candidate experience.

So I think there is some middle ground here. I think its fine to get creative, but let’s not get weird. Where’s the line between creative and weird? Well, I recently stumbled across this webpage, which helpfully not just lists example interview questions, but also gives the reasons for asking them. Most of them fall into the weird category, things like:

• What traffic sign would you be and why?

• What would you do if you woke up and found an elephant in your backyard?

• What's your favourite country? Why?

• Who's your favourite author? Why?

Some are borderline, maybe ok, maybe not (would want to research and check to see if they work):

• You've seen the office. What would you change?

• Can you put this deck of cards in order?

·"This is a completely new and untried direction for our company. If we hire you and, through your efforts, nothing happens, the doors get locked, the plant is swallowed up by a sinkhole and no one here is ever heard from again, how would you deal with that kind of responsibility and what might you do to prevent that from happening?"

And some I think are clearly good (check out the page for the reasoning):

• Can you make a function that determines whether a string is a palindrome or not? (technical question for technical staff)

• What was your biggest failure and who was responsible for it?

• I'm not sure you're a fit for the role... (testing resilience and persistence in sales people)