Sunday, 31 May 2015

Hiring: Technical phone screening

Time has arrived to turn to another of my pet subjects: hiring. As any other hiring manager, I went through a long and winding road with varying results, which yielded a few practices, tips and tricks.

As they say at the end of a flight - I realise that you have a wide choice of other hiring guides and recommendations. The one difference here is that I'll be focusing as much on the end result as on how the process was improved on the way.

Why do phone interviews?

It's best to start from the beginning, and in this specific case, it would be the technical phone screening (I'm nimbly sidestepping the various HR phases and the actual procurement of CVs). Now, for most people this stage is taken for granted, but only a decade ago I did not belong to this enlightened group, and scheduled face-to-face meetings entirely based on resumes.

It took a couple dozens of disappointing interviews to realise two recurring points:
  • People can and will tailor keywords to the job spec.
  • Our and candidate's definition of being proficient in something may differ.

Now, before becoming too cynical: very often this misconception does not have any special intent on candidate's part. When they claim that they know C++ they might refer to writing university assignments twelve years ago, and using it in their second workplace eight years back. As I'm looking for a senior engineer, my perception of a C++ skillset is someone who can have a meaningful conversation with an expert and be aware of the latest standard.

(Of course, C++ is just an example - this happened to me with any number of skills, such as Python, Linux, networking, cryptography and so on and so forth)

Both of us are right in our own way, and it's very rare that someone would lie outright on their resume. The problem is that many applicants don't know what they don't know.

Coming to the main point: face-to-face interviews are expensive for the employer, but even more so for the candidate, who has to travel, potentially take a day off at their current job, and spend time on preparation.
Hence, inviting someone without knowing for a fact that they have a decent chance of getting an offer is simply impolite. And, knowing that based on CV alone is impossible.

What phone interviews include?

To understand better what phone screening should consist of, we need to step back and ask ourselves: what do we want to get out of this?

The answer is simple: we want to make sure that the candidate has a reasonable chance of making it through. In other words, if they show up in our office and we realise very quickly that they are not the right fit, then the screening has failed.

Extrapolating this to software engineers (I'll cover QA in a separate post), we need to tick off these points:

  • Can they code?
  • Is there a large discrepancy between what they think they know and what they actually know? 
  • Can we work together, i.e. is there a communication problem?

This is not a huge list, and 95% of the time a candidate that passes these would never be a straightforward rejection later on.

Can they code?

This is the main one. Developers are paid for developing. If they can't do that, then there's no point going much further.

It all sounds painfully obvious, but back in the day I wasn't asked to code even once before being invited over. On the other side of the fence, when we were omitting coding from our phone screening, we were getting candidates who knew theory (or scored a lucky match) but could not put it to practice.

To avoid this kind of situation, some people prefer sending assignments and asking to do those offline ahead of time. My issue with this approach is two-fold:
  1. This method doesn't shed much light on how productive a candidate is; they might have written the assignment in 15 minutes or 15 hours.
  2. If an offline assignment is big, then it's unfair: we ask for substantial time investment before we know there is chance of success. If it is small, then it gives opportunities for StackOverflow- or old-uni-friend-assisted answers, and does not give enough input.
Hence, my preference has been asking to code on a shared document during the call. Naturally, this also has its detriments; a person might freeze under pressure, and logistics become a bit more involved - we need to have a computer at both ends, and there is a slightly uncomfortable wait period, while the candidate is doing their best.

However, my personal experience showed that this method simply works. There was no single case where someone did coding well, and turned out to be an absolute no-go later on. And, vice versa, on a few occasions where I went against my better judgement and invited in people with shaky coding preliminaries, it ended in (figurative) tears.

The coding question is usually very simple, and something that anyone remotely proficient in programming can do within 5 minutes.

This could be reversing a string, determining whether a given list is a palindrome, or find the count of distinct characters. 
(NB: These are not the questions I'm asking today in case you're applying)

For a while, even the infamous FizzBuzz was filtering out a decent subset of candidates, until I decided to do add variety to the proceedings, and raise the bar a tiny bit.

Another benefit of the coding exercise is that it serves as a trampoline to the technical questions. If the code contains an odd construct or algorithm choice (e.g. using a temporary string for reversing), it gives an opportunity to probe theory.

Is there a CV/skillset discrepancy?

First, we need to explore the skills that matter to the job. If, let's say, we do development in Java and the interviewee has a strong Java presence in the CV, then the topic naturally merits a few questions. The trick here is to find questions which are not plainly obvious (i.e. demonstrate experience rather than preparation), and that will give a good insight into candidate's proficiency.

For example, in the Java case, I might ask how its garbage collection works. Here, there can be a wide spectrum of answers: from not knowing what this is to a Wikipedia-level answer covering different JVMs. If it feels like the answer is prepared and I simply played a King to candidate's Ace, I might ask an open question which would push them outside of the straight and narrow (e.g. when do you think your knowledge of garbage collection would be useful during day-to-day work?).

In some cases, even skills not entirely relevant to the job may be fair game. If someone has wide malware analysis experience, and I happen to know it too, we might talk about it. This can work both ways: in worst cases it can uncover CV padding, and in best cases it can show that they do have ability to understand subjects in-depth, and negate shortcomings elsewhere.

Note that discrepancy is not always so simple as know vs. don't know. It happened to me that we went very far in the process with candidates who had decent knowledge of the right technology, but (the crucial "but") they were certain that were far better than "decent", and possessed full mastery, which was simply untrue. This would have created a major problem later on: some people can adjust their self assessment, yet some will inevitably plunge in workplace conflicts when their technical superiority comes under fire.
The key point that it's easier (but not easy!) to acquire new skills than change personal attitude.

For this reason, I used to ask candidates to self-rate themselves on a given subject before drilling in. For example, we would have a problem if someone rated themselves as 8/10 on Python, and would not know what a Global Interpreter Lock is. On the other hand, if the self-rating were 2/10, then it wouldn't be as big an issue. Of course, it's yet another question whether 2/10 is good enough for the role, but at least we are on the same page.

With time, I stepped away a little bit from numerical self-assessment to relieve some of the psychological pressure that inevitably comes with phone interviews, and simply started asking how confident a candidate feels about a specific area. Usually, the certainty and the tone of the answer is a good enough indication.

Can we work together?

Unlike coding or technical questions, this one is very ambiguous. After all, is it even possible to figure out personal compatibility in 30 minutes?

My approach has been asking open-ended questions; someone with data storage in their CV might get "what is your opinion of the NoSQL movement? How does it compare to a full RDBMS?". 
Or, if someone has both Java and Python in the CV: what are your thoughts on strongly typed languages as opposed to duck typing?

(Of course, there's also technical aspect to this, as I'll find out on the way whether they know what NoSQL or duck typing are.)

They do not have a right or wrong answer, and most importantly, they ask for opinion rather than facts. The answers will tell whether the person is opinionated (or not), whether they can present and evaluate multiple positions, and consider someone else's viewpoint. To check the latter, sometimes, even if I agree with what comes on the phone, I might play the devil's advocate.

What do phone interviews exclude?

Now, for the usual flip side of the coin: what's to omit? Phone interviews should be kept reasonably short: both as a courtesy to the candidate, and as a mercy to our own calendar. Recall that we want to have a quick pre-filter, and two hours phone calls are anything but quick.

Here's my list:
  • In-depth questions, such as esoteric corners of a language standard, or a full list of flag names for a UNIX command. There's no point really: we need to figure out if a person is good enough to come over, not whether they are an absolute expert in a specific field. 
  • Review of past experience. Of course, previous jobs matter, but mostly as a path of gathering skills that might be of use for us. Hence, it makes more sense to evaluate the end result rather than the means of acquiring it. In face-to-face interview we might do things differently and go over the CV in detail, but there's not much time for it during a call.
  • HR questions, such as career expectations, main strengths etc. Again, time is short, and in any case: more often than not we'll be receiving stock answers that do not help with filtering the candidate. The right time for this will be the face-to-face stage (especially since these questions work far better when you can see each other).

Interview - not interrogation

Last, but not least, remember: interview is for both parties; the ideal time split between questions is two-thirds company, one-third candidate. As many others stated: phone interview is often the first real impression the candidate will have of your company. If he/she is capable and has the right skills (exactly the type you want to hire), it's very likely that they'll have a choice of employers, so this impression matters a lot.

To that purpose, I tend to follow a number of rules:
  • Never give a verdict during the interview, even if it went great, and the person you talked to on the phone is the living amalgamation of everything you're looking for from a software engineer. Why? You never know what might happen tomorrow: maybe the position will be pulled from underneath you, maybe someone from within the company will apply, and maybe you'll show some of the answers to the team, and they'll notice a gap that you've missed. There are few worse experiences than guaranteeing a face-to-face interface and then reneging on that promise. A special case of that is:
  • Never cut the interview short - even if it's evident after ten minutes that the process won't go much further. This immediately gives a negative answer, and thus violates the previous rule. Also, it is just a matter of basic courtesy.
  • Always close off with the potential next steps and response ETA. As per the previous point: this happens even if deep in my heart I know that we will not be meeting. Regardless, I'll explain how face-to-face interview would run should we get there, and by what date they will hear from us.
  • Always allow interruptions and dialogue. It's not just about satisfying curiosity and giving the interviewee full input (although this matters a lot of course!). In many cases the questions you hear will give a far better insight than the ones you ask.
    For example, years ago, during a senior dev hiring round, I was asked about specific JavaScript sandboxing challenges, which happened to be exactly the task we were tackling ourselves. Needless to say it propelled the candidate up the list.
    Note that you can interrupt too, especially if the candidate goes on a tangent or takes time to impress on topics you haven't asked (this happened to me a number of times). 
  • Never lead off with technical questions. I always prefer to ease in by first taking about the company, the role and why we are looking for this particular profile. This makes the process two-sided, and makes the other party less nervous.

  • Don't sound like a broken record. In other words, when talking about the position, don't say the same words in each call. I know - it's hard, especially once we get going and run a couple of calls each day. However, a brief 5 minute preliminary look at the CV always gives a few pointers and removes that conveyor belt feeling.
    For example, rather than saying "and we use Java for our automation framework" , I might say "and, as with your Initech experience five years ago, we use Java for automation. This is why this part of your career will be particularly interesting for us". 
    This all goes back to first impressions: showing that you carefully looked through the CV is a signal of genuine personal interest, and might be the little gesture that will push the candidate in the right direction.


Good technical phone interviews are the cornerstone of a good hiring strategy. Without them, you either end up wasting your and candidate's time, or rejecting potential matches through CVs alone. Getting them right is hard but rewarding: it consists of laser focus on our pre-filtering points (coding, CV truthfulness, and personal compatibility) and giving the right first impression to the candidate.

No comments :

Post a Comment