STEP-KTODER: A Framework for Code Preference Optimization Using Function-Level Execution Feedback
Summary:STEP-KTODER is a novel framework for code preference optimization that defines steps as module-level functions in decomposed multi-function programs and assigns binary correctness labels through automatically generated unit tests. This approach combines function-level process supervision with outcome-level feedback on the full program. Evaluations on benchmarks like HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench demonstrate that STEP-KTODER outperforms outcome-only KTO and DPO methods. T
Background and Motivation
In mathematical reasoning, process supervision has significantly improved model performance through chain-of-thought reasoning. However, in code generation, process supervision remains underexplored due to the lack of a standard definition of a 'step'. Supervision targets can be code lines, reasoning traces, or program states, making labeling and optimization complex and ambiguous.
The STEP-KTODER Framework
STEP-KTODER introduces a new approach to code preference optimization with the following key innovations:
- Module-level functions as steps: Defines steps as module-level functions in decomposed multi-function programs.
- Automated unit test generation: Assigns binary correctness labels to each step through automatically generated unit tests.
- Combining process and outcome supervision: Integrates function-level process supervision with program-level outcome feedback for more refined optimization.
Experiments and Results
STEP-KTODER was evaluated on multiple benchmarks, including:
- HumanEval(+)
- MBPP(+)
- BigCodeBench
- LiveCodeBench
The results demonstrate that STEP-KTODER outperforms outcome-only KTO and DPO methods in terms of code generation quality. Additionally, the study reveals that execution-based labels are crucial for optimization, while LLM-as-a-judge annotations systematically degrade performance by over-predicting function failures and corrupting positive step labels.
Technical Highlights
- Innovative supervision mechanism: Achieves more precise supervision through module-level functions and automated unit tests.
- Significant performance improvement: Demonstrates its potential in code generation with strong benchmark results.
- Insight into LLM limitations: Highlights the shortcomings of using LLMs as judges, providing new directions for future research.
Industry Impact and Developer Recommendations
The release of STEP-KTODER brings new ideas and methods to the field of code generation and optimization. Developers can apply this framework to real-world projects to enhance the quality and efficiency of code generation. Additionally, the research underscores the importance of carefully evaluating the limitations of LLM-based automation systems and exploring more reliable supervision mechanisms.
— END —Source: ArXiv AI (cs.AI) (2026-08-27)
Tags: #Code Generation #Process Supervision #AI Optimization #LLMs & Foundation Models #Automation
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments