Hiring for judgment
The interview loop I inherited measured how fast someone could code under observation. The one I run now measures whether they can tell good work from bad, including work a tool produced. What changed, what stayed, and where the candidates actually come from.
I have run a lot of interviews that produced a confident decision about the wrong thing. The candidate reversed the linked list in eleven minutes and we hired him and he could not decide what to build. The candidate froze on the whiteboard and we passed and she went on to lead a platform team somewhere else.
The loop was measuring something. It just was not measuring the job.
What the job actually requires
Before I changed the interview loop, I wrote down what the strongest engineers on my team did that the weaker ones did not. The list was short and none of it was speed.
They could look at a problem and decide what needed to be built and what did not. They could read a change they had not written and find what was wrong with it. They could explain a tradeoff to someone who did not share their context. They noticed when something was off and said so before it was expensive. And they got better at all of this every year, visibly.
That list is the rubric. Every interview in the loop maps to one line of it, and if an interview does not map to any of them, it is gone.
The loop
A take home that looks like our real work, capped at two hours, followed by a conversation about the choices in it. The conversation is the interview. The code is the excuse for having it.
A review exercise. We hand the candidate a pull request with several things wrong with it, some obvious and some subtle, and ask them to review it out loud. This is the interview that predicts the most. People who can find the subtle problem in someone else's change are people who will keep your codebase healthy.
A design conversation with a vague prompt, where the thing we score is the questions they ask before they start drawing.
A conversation about a project they are proud of, and one that went badly, with follow ups about what they would do differently. The second half tells you far more than the first.
Sourcing is a relationship, not a search
The best hires I have made did not come from job boards. They came from people on the team who had worked with them before, from candidates we passed on for one role and kept in touch with for the next, and from a small number of communities where I spent time being useful before I ever needed to hire.
So the sourcing work happens all year. I ask every engineer on the team, once a quarter, who is the best person they have worked with that they would want here. I write those names down. When a role opens, that list is where I start, and it is warmer than any outreach.
I also keep the loop honest about who it excludes. If every hire in a year came through the same three channels, the team is going to look like those channels. The fix is not a quota. It is going where the people we are missing already are.
What changed with AI
When we moved to an AI first way of working, the interview loop needed one more thing. I added a session where the candidate uses an AI tool on a real task, with us watching how they direct it. Not whether they can use it. Almost everyone can. What we score is judgment: do they notice when it produces something wrong, do they check the test it wrote, do they push back on a suggestion that does not fit the codebase, can they explain what the result does.
Candidates who treat the tool's output as finished are a hard pass now, regardless of how fast they were. Candidates who treat it as a draft from a capable junior colleague, to be read carefully and corrected, are the ones who ship in week two.
The decision
Interviewers score independently and in writing before the debrief. I read the scores before I read the names. Concerns have to point at something the candidate said or did. And a lukewarm yes from four people is a no, because the people who turned out to be great were never lukewarm.