Algo interviews are overrated

Google loves algorithms interviews. Palantir loves algorithms interviews. Pretty much every tech company who prides themselves on hiring smart people spends at least half the dev interview time on algorithms questions.

I think algo interviews are overrated. Useful to some extent, but not the only thing and not as important as most engineers think. I think Steve Yegge (mentioned yesterday) would agree.

In my opinion, these are the important things in hiring a new member on a dev team. They are not priority ordered or of equal weight.

1. Ability to translate a problem into components.
2. Ability to reason through a problem and feel the solution.
3. Ability to keep trying and go deeper on problems unlike ones she's seen before.
4. Ability to translate these solutions into code.
5. A bag of tricks (data structure knowledge, technical knowledge) and the intuition that says "I bet there's a better way to solve this problem, I just don't know it yet."
6. Ability to build stuff.
7. Technical flexibility (can a SQL expert adapt to NoSQL? Can a Java person learn Ruby?).
8. Attitude. Is the candidate going to mesh with the team? Are they going to make coworkers' days better or worse? At Google, this was "Googliness".

Good algorithms questions can address points 1 through 5, though algo problems and every day coding problems don't map perfectly. Algo problems are too neat - they're like bowling with lane bumpers whereas everyday coding is like bowling without gutters or even lanes.

The real kicker: algo interviews don't gauge the ability to build stuff. Some companies get this and ask for you to build something for them. Airbnb and RockMelt ask candidates to spend time building something on their own. Square does pair programming so you can build something together (I interviewed at Square and feel that I was at a significant disadvantage because I don't know Ruby.).

Companies are wary about having candidates build something on their own time because it adds hours to the candidate's time load. Companies and candidates alike perceive asking a candidate to spend their own time (as opposed to shared interviewer-candidate time) as arrogant and pushing too much onto the candidate.

I think it's important to gauge how good someone is at building things ahead of hiring, but I haven't figured out a method to do it that isn't burdensome.

Another problem with algorithms interviews: the distribution of questions makes it possible for a candidate to have heard 80% of the questions you're going to ask before the interview. I'm not saying the candidate knows 80% of all the problems asked on interviews, but there's effectively a finite set of questions interviewers ask in a 45 minute interview. Algo questions need:

• Simple set up. The question needs to be intuitive and you can't spend 15 minutes for the problem set up.
• Bounded solutions. There's probably going to be one or two efficient ways to solve it.
• Non-esoteric solutions. Priority queues are awesome, as are tries, but they're rarely used outside of algorithms classes and algo interviews. Questions mostly come down to: divide and conquer, dynamic programming, hashing, merging, binary search, sorting, and iteration.

If you paid attention in your 100 level courses, and solve some example problems, you can probably pass your Google onsite with 10 hours of preparation. Steve Yegge (yep, I have a thing for him today) tells you how. Google has an internal list of banned questions that have become affiliated with Google, but even those are still often asked. Good algo questions are hard to find.

I've concluded that algorithms interviews can be gamed and I've concluded that even if they couldn't be gamed, they aren't the only thing to gauge during the hiring process. So where do we go from here?

• Find a way to have candidates build something during the onsite. I don't have the answer yet, but the closer to real work candidates are doing, the better the interview will be.
• Ask design questions. Not design patterns esoterica, but problems that almost beg the candidate to say "This is like something I did before! We solved it doing ..." and then asking about the other tradeoffs the person has made.
• Ask questions that speak to grey (technical) issues. Ask when they'd choose between GWT and jQuery. Ask when they'd take their HashTable and throw it into MySQL, and when they'd go from MySQL to NoSQL and MapReduces. See if they've gotten the feel for products they've developed before instead of simply hacking at them.
• Get a feel for the candidate. See if it's someone you'd enjoy working with when the server's been down for 3 hours and you still haven't figured out why.
• And yes, still ask some algo questions. See if the candidate has the gut feel of when to use HashSets, or if they're only into brute force solutions. Ask for the runtime - if someone really gets the solution, runtime is trivial.

Too many tech companies are measuring in great detail something that has poor correlation with career success. I haven't noticed it change much in the past half decade.