Skip to content
Free since 2015. Built for the engineers who keep things running.

cat /articles/nvidia-used-20000-training-examples-to-fix-its-ais-mistakes-its-coding-score-jumped.md

AI & MLOps

Nvidia used 20,000 training examples to fix its AI’s mistakes. Its coding score jumped.

Nvidia used Factory's Droid coding agent to produce about 20,000 training rows for its Nemotron 3 Ultra model, lifting its Terminal-Bench 2.0 score from 38.5% to 49%.

The New Stack21 Nov 2026 · 4 min read · summary

article --summary

The summary

This piece explains how Nvidia worked with Factory to capture reproducible failures from the Droid coding agent, covering problems with tool calls, argument formatting, and recovery from errors. Those traces became post-training data for the Nemotron 3 Ultra open model. The reader will also learn how the PivotRL reinforcement learning method, which focuses on critical decisions rather than matching reference examples, fits into the effort. Nvidia guarded against overfitting to one harness by varying Droid's configuration and mixing data from other agent environments such as OpenCode and Codex. The article also contrasts this with Hugging Face's approach of training separate LoRA adapters for different harnesses, which lifted a Qwen model's Terminal-Bench 2.1 score from 39.5% to 55.8%. It notes that neither company has published direct comparisons of the performance and compute trade-offs between these two methods, and that Factory has not released an ablation study for its share of the gains.

Editorial note: This is a summary of an article first published by The New Stack. The full piece is on their site.

Reading now

Topic
AI & MLOps
Source
The New Stack
Published
21 Nov 2026
Format
Curated summary

Have a take on this?

Write it up and submit it. It goes on the list once an editor approves it.

Submit an article

articles --latest

Latest articles

New from the blogs we follow and the community.

All articles

newsletter --weekly

DevOps Weekly, minus the noise.

The best reads, new jobs and upcoming events, in one email a week.