Tales from the interview: when the tech test goes wrong

A couple of years ago, when I was last searching for a new role, I was contacted by a recruitment agency regarding a FinTech conveniently located in my home city who were looking for a PHP dev to join their team. I won't name them but they were a matured start-up based around data and analytics - a comparatively small business but earning revenue, profitable as I recall.

The pay and benefits package for this role were excellent, considerably above average. They had the resources to expand and although remaining in the financial sector wouldn't have been my first choice in terms of industry and product, it looked sufficiently intriguing that I gave the go ahead for the agent to send off my CV and put in an application.

The interview

It wasn't long before the agent got back in touch. The business liked my CV and were asking me to take part in a first stage interview with one of the directors. At this point, the agent explained their hiring process to me. Fairly common model - first interview, a take-home tech test, they review the test, second interview, offer or rejection.

I may have touched on the topic of take-home tech tests on my blog before. Broadly, I'm not in favour of them.

I don't believe they're a good way of measuring programming ability, technical acumen or problem-solving capability. I don't think they present realistic simulations of what a candidate will be dealing with on the job or how they will work. Where attempts are made to craft challenges which more closely resemble these things, they become more complex, demand more time and start to look a lot like simply unpaid work.

I don't think this is fair to candidates. When someone is seeking out a new role, they're probably applying to many different prospective employers. They may also still be working full time and juggling family and other responsibilities. If every business to whom they apply is expecting them to spend anything up to eight hours (a full working day) of their own, free time, building complete sample applications, or even churning out a dozen or two leetcode algorithms (which deliver even less value), this quickly adds up to unrealistic expectations.

The inevitable result is that sometimes (often) the submitted works are rushed, incomplete, delivered to a worse standard than what they might do as a paid employee on the company clock and with more than one day to build out a project from start to finish.

The better process, in my considerable experience, is a focused technical interview, where the candidate is asked about problems they've solved, projects they've delivered, challenges they've overcome, their personal interests in technology and their aspirations. Talk about what you do as a business, what both you and the candidate are hoping for in the role, talk about your culture, get into depth about your lifecycle, your infrastructure, your ways of working, etc.

This delivers much better surety and value about the core competencies of a candidate, and it gives them the same about you as a business. It saves both of you a waste of time, money and energy invested in the wrong opportunity. Do you want to hire someone who's jumped through all your hoops only for them to walk three or six months later?

I digress. Much as I disagree with the idea of take-home tech tests, this role - as is unfortunately normal nowadays - included one in the process.

So I have the interview and it's vanilla as far as these things go. We talked a bit about the company, its history, the expectations for the role. I thought it was a little bit pretentious that the interviewer introduced himself by the title Dr (he had a PhD, in a field unrelated to either finance or technology) but other than this slight weirdness, there weren't any obvious red flags.

We get to the topic of the tech test. I politely explained that I'd applied for four other roles, all of which included some kind of technical test or take-home assignment and that while I was happy to complete one, it would have to be something I could complete inside a couple of hours.

Dr Interviewer assured me there was no expectation that I would spend more than two hours on their tech test and this would be enough time for me to do a satisfactory job.

We said a pleasant goodbye and I awaited the details of their test to come by email.

The technical test

Instructions for the technical test were, on arrival, simple enough. The task was to use Symfony to build a simple API with a single endpoint (no worries about access control or other auxiliary configuration) with some particular parameters. I won't go into the details but basically the idea was to take the inputs from the API request, extract some data from a CSV file they supplied and then based on the specifics of the request, perform some calculations or transformations on the data and return the result as a JSON object.

The only real caveat or challenge part of the test was that they wanted you to design this API endpoint in such a way that the parameters supplied as input were flexible and extensible - it was something like the user needed to be able to provide custom expressions with operators in the request and the test wanted to see how you would structure that so new operators could be built later.

I completed the test inside the two hours I'd set aside for it. I built some extensible OperatorInterface and I don't recall loads of the details but you know, I'm not saying my solution was the maximal, theoretical perfect solution but it was entirely reasonable and adequate. It fulfilled the full scope of the test's criteria in a way which, in the part that counted, was done to a professional standard.

But of course constrained by the time I could put into it, for all the parts other than the specific bit of the challenge I described, I cut a lot of corners. I want to stress in these kind of code samples for job applications, this is an entirely normal thing to do - unless we're specifically asked to, we don't worry about boilerplate parts of a system like logging, robust error handling, edge cases, caching, access control and all the rest of it.

It's still important, however, that where we do this as candidates, we make it very clear that any corners haven't been cut because we're not aware of them, or because we don't understand the proper principles of good system design, but just because it's not the bit of code we focused on for the task at hand in the time we had.

So I did what any candidate would do. I annotated the parts of the code where I knew there were things missing or done to a lower standard than we would in a real-world system. I put in comments like the following examples (not real examples, just the gist of the kind of thing I was writing in the code):

/**
 * Parse the CSV file into an array in memory. Obviously in the real world we wouldn't be getting our
 * data by reading and parsing a full CSV file on every single request. We'd be using a data source such as a RDBMS
 * and very likely using an entity mapper to get a list of objects, maybe with a caching layer such as Redis.
 * As this is just a demo project, I haven't bothered with things like error handling, checking correctness / validity
 * of each row, missing headers, missing columns, etc. here, I'm just assuming the whole file is valid...
 */

 ...

/**
 * Extract and tokenize a request expression from a string.
 * For the scope of the sample, I haven't bothered handling error and edge-cases here
 * such as referencing a non-existent record type but we would do this a bit better
 * in a real-world system...for now, we just convert these invalid values to null and ignore them.
 */

I thought the dozen or so comments in the sample application I sent in clearly explained where and why boilerplate-type features were missing. In terms of addressing the core criteria and the design of the particular bit the test was focused on, I was happy with the result so I sent it over.

Feedback: the bombshell

It was the following afternoon I received an email from the recruitment agent. I'm never presumptuous about whether or not I'll be offered a job but I was fully expecting to progress to the second interview. I know what I'm doing with PHP and Symfony, very well, and I was confident they'd be happy with the sample for their test I sent in.

The email in my inbox opened like this:

Hi David

I've just heard back from [XYZ] about your technical test and unfortunately, it's not positive.

I was - and I sincerely don't mean this in an arrogant way - shocked. Blown away. I had to re-read it twice to make sure I was reading it right.

I phoned the agent to discuss the feedback he'd been given and this is where things took an interesting turn.

Apparently, the business had been entirely happy with my code sample as a technical solution to their test criteria. They liked the design, they were satisfied I knew what I was doing with Symfony and in my ability to translate requirements about business rules and data into working code.

What they didn't like were my code comments. The feedback was that because I'd used phrases like "..haven't bothered with proper error handling here", this meant I "didn't demonstrate the attitude they were looking for in an employee" and they wouldn't be progressing my application any further.

I ask you at this point, dear reader, to take another moment to bear in mind the context of those code comments that I've explained above.

I openly invite feedback now, via this blog post, as I ask the question: was I being unreasonable here, or were they?

Because from my point of view, I've sent off a dozen-plus code samples like that in my career with code comments like that and I've never had it be an issue anywhere else, I've never been criticised for it in follow-up interview when tech teams have gone over my submissions with me, let alone been instantly dismissed from an application process over it. As far as I'm concerned, it's just a normal, sensible thing to do, to draw the line between a candidate who's missing pieces because they don't know how to build a real, production-grade system properly versus one who's...just writing a quick code sample.

Bullet dodged

Overall, I feel like this reaction to my tech test was quite valuable. I know my use of language in the comments was relatively casual but it wasn't exactly crass or vulgar either. Whether others would agree I'm right, I don't know, but I took it to be a sign that in this case, the company, the culture and the people I'd have been working with would not have been a good fit for me and that if they'd offered me a position and I'd ended up accepting it, I would have swiftly come to regret my decision.

I'd welcome any reader stories about your craziest interview experience, your worst rejection or your most heartening success. Please feel free to leave a comment using the form below or email me!

---

Comments