ChatGPT Should be Allowed in Interviews

Imagine you got a job by using ChatGPT* in both a technical take home project and in technical interviews.

You keep on using ChatGPT at work at people consider you to be a great engineer.

That makes you a great engineer, because generally we rate people on the outcome of their efforts not the method they use.

So, why don’t we let candidates use ChatGP in their interviews? Let us explore the cases for and against the use of these tools during interviews.

The Case For Allowing AI Interview Use

Realistic Interview Setup

Allowing the use of ChatGPT to solve problems is a realistic proxy for a job. 

We might expect ChatGPT to be a choice for candidates to solve problems they might face at work. It is better to test their choice of tools, rather than throwing out a tool that they might use at the first hurdle. 

Assessing Essential skills 

It’s the communication and technical skills of candidates that we should look for, the tools they use should not matter.

It’s down to companies to formulate what exactly they are looking for in an interview (the candidate cannot really influence this), and this is seldom done well.

The fact that companies disbar the use of ChatGPT without considering it to be an essential skill is an interesting strategy. By interesting I mean a euphemism for wrong.

Encourage Efficiency

AI helps engineers work efficiently and concentrate on the core issues with regard to software engineering problems.

Surely, we would want software engineers to focus on the problem and measure their process, rather than getting caught up in the implementation details that can be abstracted away.

The Case Against Allowing AI Interview Use

AI Won’t Replace Us

We can use the same argument as to why we don’t replace developers entirely with AI. For simple problems it works great, but if it does not pick up the full context the answers can be poor. So, although interview questions might fit the problem domain of ChatGPT nicely it does not seem likely that a job will also map to this problem domain.

AI Shouldn’t Be Used At Work At All

There are many instances where companies do not want employees using ChatGPT the same way copy-pasting code from StackOverflow can be damming. That is, there are many problem domains where code without the appropriate license should not be used.

Assess True Skill

Rather than assess a candidates skill using the tools at hand we should measure their true skills. This could mean a whiteboard test which is actually taken on a physical whiteboard.

Accessibility

Some candidates do not have access or are familiar with AI tools. The playing field is already uneven for software engineer candidates. Let’s not make it worse.

It’s Already Happening

Candidates certainly are already giving generative AI answers to interview questions. Colloquially this is around 1 in 3 candidates for some roles.

Sure, you can usually tell when a candidate is reading something. You might have a process where you ask to see a candidate’s hands when they are speaking or even close their eyes when answering a question.

The Truth

I don’t have the skills to cheat during an interview. I’m generally a stuttering mess and trying to multitask using ChatGPT during coding or technical interviews is likely to make my performance worse.

The thing is if you aren’t using AI in your daily workflow you are just slower. AI is amazing as a glorified autocomplete and finding patterns, and to be frank most developers benefit from using the tool. We should be open about the way we work and not create interview procedures that prevent us working in the way we would if we got the job.

But..Mindful AI Implementation Simply Isn’t the Norm

At the point Apple are Introducing (looks at notes) “Apple Intelligence” to each machine you’d think companies should develop policies and ways of working around ChatGPT and AI. 

Unfortunately, this is seldom true as companies seem to be far behind the curve in terms of the generative AI tools appearing on the market. It’s time they caught up.

Conclusion

AI is here to stay. That doesn’t necessarily mean that it should be used in interviews, but it doesn’t mean it should be excluded completely either. 

There should be nuance and this stuff should be thought through.

Is that too much to ask?

*Or any generative AI tool. For brevity, ChatGPT is used as a generic term for whatever generative AI tool you would like.