A team of researchers has identified a fundamental weakness in how large language models learn to solve reasoning tasks, and they've developed a new training approach to address it. The discovery suggests that current reinforcement learning methods may inadvertently teach models to rely on superficial features rather than logical problem-solving.
According to arXiv, the researchers conducted experiments by altering prompts in ways that preserved the actual problem and its correct answer but changed irrelevant wording and formatting. This semifactual intervention revealed that language models show inconsistent performance across these variations, despite the underlying mathematics remaining identical. Some individual tokens within the model's output proved far more sensitive to these cosmetic changes than others.
The Core Problem with Current Methods
The analysis exposed limitations in Group Relative Policy Optimization (GRPO), a popular technique for training reasoning models. GRPO assigns the same credit or advantage signal to every token in a response based solely on whether the final answer is correct. This approach treats all tokens equally, which can accidentally reinforce tokens that align with spurious patterns in the training data rather than genuine reasoning steps.
The researchers tested a simple intervention: during model inference, they suppressed tokens that exhibited high sensitivity to meaningless prompt changes. This filtering alone improved reasoning accuracy without any model retraining, demonstrating that the problem stems from the training process itself, not fundamental model architecture limitations.
A More Targeted Solution

Building on these insights, the team introduced Semifactual Credit-Augmented Policy Optimization (SCAPO), which applies more granular credit assignment during training. Rather than rewarding all tokens equally for a correct final answer, SCAPO measures how much each individual token's probability shifts when prompts are altered under semifactual conditions. Tokens that drift substantially get reduced credit early in training, while stable tokens receive appropriate rewards. Importantly, the method does not grant bonus credit simply for stability, avoiding circular incentives.
Testing on Qwen models at two different scales revealed substantial improvements. On the challenging AIME 2024 to 2026 mathematical competition problems, SCAPO outperformed GRPO by 5.63 percentage points on the larger 4 billion parameter model and 4.17 percentage points on the 1.7 billion parameter version. The technique also showed the strongest generalization to out-of-distribution benchmarks across all tested scenarios.
Why This Matters
- Current AI reasoning systems remain fragile and sensitive to formatting changes, limiting real-world reliability.
- The research points to a misalignment between training signals and actual reasoning capability in modern reinforcement learning approaches.
- The proposed fix operates at the token level, offering a pathway to more robust model training without architectural redesigns.
- Improvements on mathematics benchmarks suggest broader applications for any domain requiring logical reasoning and problem decomposition.
The researchers have published their code publicly, enabling other teams to evaluate and build upon the approach. As language models increasingly handle high-stakes reasoning tasks in science, engineering, and other domains, ensuring they rely on genuine logic rather than prompt artifacts becomes increasingly important. SCAPO represents a step toward training systems that reason more reliably regardless of how problems are presented.



