A Friday afternoon discussion in a couple of slacks today made me think about "coding tests" that companies like to implement.

While I don't often encounter them myself any more, I certainly have in the past. They're apparently loved by hiring managers, but detested by developers, and I'd like to give some feedback on why they're not helpful (and how to make them better).

There are a few different types of coding test that I've seen in the wild:

Solving a puzzle: ranging from FizzBuzz to complex problems, these are designed to test problem solving skills, and adherence to specifications. The problem? They don't actually show either of those things in any useful manner. Much like the abstract "if you're the size of a penny, how would you get out of a blender" questions that used to be popular, these show skill in puzzle solving, not problem solving. Most problems that you encounter in a software job don't look like puzzles.

The sample project: Usually a basic "payroll" system, or something else that feels like a high-school project. Nothing massive, but that's the problem - because it's not a "real" project, you're not inclined to build it to the same quality that you would build a real production system. It's also a lot of work to undertake - it might take up to a week (usually unpaid) to make something that will never see the light of day. It also produces far too much code to effectively review - basically it's pointless busywork that's way too inefficient to be effective.

The "real" problem: Another approach is to as a prospective candidate to solve a real business problem for you. This shows ability to solve real problems, and ability to create value, but... if the real problem was easy enough to solve that you can ask it as an interview question, then why is still an unsolved problem? It's also difficult to find an appropriate problem like this, and you're effectively getting a prospective hire to work for you for free. A high quality solution would require knowledge of how your systems actually work, which means that you also have to hand over parts of your code base. No matter how you look at this, it's likely to cause a nightmare for someone, and it will probably be you.

The "solved" problem: Like the "real" problem, this is a problem that you've actually encountered in your business. The difference is that you've already solved this one. It means that you can assess the candidate's approach to a problem, and not feel bad about soliciting "free work". The problem here is that your reviewers are going to judge everything against how the problem was already solved. This will actively work against brilliant developers who solve things in a different manner, and again requires you to either hand over access to an (older) version of your code, or re-construct the problem in another scenario. It's possibly one of the better options available, but it's still not great.

So how could you do code tests that are beneficial for all parties?

Open Source

There are very few companies out there that aren't using at least one open source library, if not multiple. They may be maintaining their own open source projects, or providing support to others. No matter what, you'll have some open source tools that your team is familiar with, and this is where you should target coding tests.

Ask your candidate to make a pull request to something open source. It could be a tool that you use, or it could be something that they've already contributed to. Maybe it's a bugfix, or maybe it's a new feature. The scope is up to them. This gives you an opportunity to see what their code is like on a real code base. It lets you see them handle CI processes, talk to community members, communicate, etc. What's more, it's not just contributing to projects that either you or they use, but it's helping them even if they don't get your job. Contributing to open source looks great, and could even get their name noticed by someone else.

You probably won't reap any immediate benefits from the work that they produce. The pull request doesn't even need to be accepted. They could even show you pull requests from their recent past, which they feel adequately shows their current skill level. This gives you the best and clearest idea of how they work, while allowing the candidate to set the scope of work that they're comfortable completing. More than that, it shows that they're competent with modern tooling like version control, CI services, code reviews, etc.

So the next time that you're hiring, and looking for a way to test the mettle of new candidates, don't ask for FizzBuzz - Ask for GitHub.