When to Implement a Programming Challenge: Pre- or Post-Onsite Interview?
The decision to conduct a programming challenge before or after the final onsite interview can significantly impact the hiring process and overall candidate experience. This article explores the pros and cons of each approach, providing insights from real-world experiences and offering practical advice for optimizing your hiring process.
The Importance of a Well-Structured Programming Challenge
A programming challenge serves as a crucial tool in evaluating a candidate's technical skills and problem-solving abilities. However, the timing of when to integrate this challenge can vary based on the size and nature of your organization, the talent pool you are targeting, and the candidate's selling process.
The Google 2010-2011 Case Study
During the period of 2010-2011, Google implemented a coding test/exercise as part of its candidate evaluation process. The goal was to identify high-potential engineers to hire from a pool of candidates across 15 engineering sites in EMEA. The experiment began with 100 candidates, and it was found that there was a positive correlation between the coding exercise score and their subsequent technical interviews. After expanding the experiment to 1500 candidates, the company managed to hire approximately 15 individuals who would have otherwise been rejected.
Continued Success and Adaptation
The success of the initial experiment led to a broader implementation. The coding test was used as an early filter, but it faced challenges. Initially, 30 out of 1500 candidates did not even bother to click on the link sent by a recruiter. However, when the test was moved to after the first technical interview and a VC introduction, the number of candidates who did not take the test reduced to just 12.
The B2B Startup 2015-2016 Case Study
In 2015-2016, a B2B startup in Europe faced a similar challenge. The company was developing sophisticated systems and had to market these to passive candidates, which required significant effort. To address this, the engineering team requested the addition of a coding test as an early filter. The initial results were mixed: 30 candidates did not even attempt the test. Post-implementation, the number of non-completers reduced significantly to just 12, highlighting the importance of aligning the test with the candidate's overall interest and excitement about the role.
Final Thoughts and Recommendations
Given the varying success of these experiments, it is evident that there is no one-size-fits-all solution for implementing programming challenges. However, the following recommendations can help optimize your process:
Focus on Candidate Experience
- Minimize Time Spent: Ensure candidates, especially passive ones, do not have to spend too much time on the test. Jobless coders are a rare breed, and providing a seamless and supportive candidate experience is crucial for retaining potential hires.
- Smooth and Agile Review Process: The candidate's coding exercise should be reviewed efficiently, with minimal delays. This ensures that both candidates and hiring managers are aligned and that the process remains agile and responsive.
Target Specific Candidates
- Sell the Role/Company: Make sure the candidates who take the test are already sold on the role and company. This is particularly important in highly competitive industries, where the candidate's genuine interest can influence their willingness to proceed with the process.
Run Experiments
- Experimentation: Do not take any opinions, including mine, for granted. Conduct experiments to assess the correlation between the coding exercise results and the human feedback. Based on the results, you can determine how many candidates should take the test before the onsite interview and how many after. This data-driven approach will help you make informed decisions and optimize the hiring process.