ChatGPT detects and debugs source code using standard machine learning approaches, reports Analytics Insight.

Its main advantage over other AI methods and models is its unique ability to converse with humans, allowing it to improve the correctness of answers.

“We find that ChatGPT’s bug fixing performance is competitive to the common deep learning approaches CoCoNut and Codex, and significantly better than the results reported for the standard programme repair approaches,”

the researchers write in a new arXiv paper, which New Scientist first spotted.

Although ChatGPT’s ability to solve coding problems is not new, the researchers emphasize that its unique ability to talk to people gives it a potential advantage over other approaches and models.

According to Meta’s Head of Artificial Intelligence Jan Lekun, ChatGPT is built on the Transformer architecture, which was developed by Google.

In the code debugging examples, OpenAI highlights ChatGPT’s dialog capability, where it can ask for clarifications and get hints from the human to arrive at a better answer. Reinforcement learning with human feedback (RLHF) was used to train the large language models that power ChatGPT (GPT-3 and GPT 3.5).

ChatGPT’s ability to discuss may help it arrive at a more correct answer, the researchers note that the quality of its suggestions is not yet known. Therefore, they want to evaluate ChatGPT’s debugging capabilities.

The implications for developers are not yet clear. ChatGPT-generated responses were recently banned from Stack Overflow due to their low-quality but plausible-sounding nature. The Wharton professor finds that ChatGPT can act as a “smart consultant” (one that produces elegant but often incorrect answers) and promote critical thinking in MBA students.