Debug-gym Trains AI to Debug Code Like Human Developers

A new tool called debug-gym is helping AI agents learn to debug code the way human developers do—interactively and iteratively.

While AI coding tools like GitHub Copilot have made code generation faster and more accessible, debugging remains a major bottleneck. Most developers spend more time fixing code than writing it, yet today’s AI tools often fall short when it comes to identifying and correcting bugs that go beyond simple errors. AI is getting better at writing code—but when it comes to debugging, it still struggles to match the reasoning and investigative approach of human developers.

That’s where debug-gym comes in—a research environment designed to simulate how programmers step through code to fix problems. The tool enables code-repairing agents to actively seek information using tools like Python’s debugger (pdb) rather than relying solely on static error messages.

Why Debugging Matters

Industry leaders like GitHub CEO Thomas Dohmke and Y Combinator’s Garry Tan have projected that the majority of code will soon be AI-generated. Yet writing code is only part of the job—maintaining and debugging it is often the greater challenge.

Unlike conventional AI tools that may offer a single guess based on training data, debug-gym allows agents to:

• Set breakpoints

• Print variable values

• Navigate across files

• Use full code repositories

• Form hypotheses and Iterate based on runtime feedback

By interacting with real debugging tools, agents trained in debug-gym generate context-aware, grounded solutions that can be reviewed and approved by human developers.

Key Features of Debug-gym

• Repository-level scope: Agents can access and navigate the full codebase to understand project-wide context, enabling full access to explore and update files.

• Sandboxed execution: All code runs in isolated Docker containers, ensuring security during testing and debugging.

• Text-based interface: Observations and actions are formatted in structured text (e.g., JSON), making it compatible with modern LLM-based agents.

• Extensibility: New tools and features can be added easily, enabling a flexible research framework.

Benchmarks and Early Results

Debug-gym allows researchers and developers to point to any custom code repository by specifying a folder path, enabling them to evaluate how well their debugging agents perform in real-world contexts. It also includes three built-in benchmarks to test agent performance across different levels of complexity:

• Aider for simple function code generation

• Mini-nightmare for compact hand-crafted bugs

• SWE-bench for full-scale GitHub issues that require repo-wide understanding and pull request–style solutions.

Initial tests showed that even simple prompt-based agents performed significantly better with access to debugging tools than without. However, solving complex issues still remains a challenge, especially due to a lack of training data that reflects step-by-step debugging behavior. Still, the notable performance gains—highlighted in the most successful test results—confirm that this is a valuable and promising direction for future research.

To explore debug-gym further or begin training your own debugging agents, check out the technical report and GitHub repositor for full documentation and resources.

Future Work

The team behind debug-gym is now focused on improving the interactive capabilities of AI agents by fine-tuning them with specialized debugging data. Unlike static coding tasks, interactive debugging requires agents to make sequential decisions, respond to live feedback, and gather context before proposing solutions—skills not well-represented in current training datasets.

To address this, the researchers plan to develop and train a smaller, information-seeking model that actively gathers relevant context during the debugging process. This model could then work alongside a larger code generation model, functioning as a lightweight, cost-efficient system for context retrieval—similar in spirit to retrieval-augmented generation (RAG).

The data collected during this training loop will also help improve larger LLMs by exposing them to real-world debugging traces, tool usage, and the decision-making paths human programmers follow.

By open-sourcing debug-gym, the team hopes to catalyze further research and collaboration in building interactive, tool-using AI agents—not just for debugging, but for a broader class of real-world programming challenges.

What This Means

Debug-gym offers a promising foundation for a future where AI not only writes code, but understands and repairs it in real time. For developers and researchers, this tool opens the door to building more reliable and autonomous code-repair agents.

Open-source maintainers and software teams could soon rely on AI agents to triage and suggest fixes for large backlogs of issues, significantly reducing manual overhead. At the same time, AI tool builders have an opportunity to fine-tune these agents for more advanced, information-seeking behavior—going beyond static code suggestions to truly interactive problem-solving.

Ultimately, debug-gym represents an important step toward LLMs that can reason, investigate, and iterate like real developers—not just guess from past patterns.

This is about more than fixing bugs—it’s about teaching AI to understand code through interaction.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.