arXiv Research: How Prompt Style Affects AI Accuracy and Speed in Competitive Programming
Summary:A new arXiv study investigates how different prompt styles affect AI performance in competitive programming using GPT-5.5. The research, conducted across three accuracy tasks and two NP-hard speed tasks from the International Olympiad in Informatics (IOI), found that no single prompt style was universally optimal. The best prompt style depended on the task type and whether results were summarized by the mean or median. While the AI outperformed the average human contestant in one task, it fell s
Background and Motivation
In recent years, large language models (LLMs) have seen increasing use in competitive programming. However, their outputs are highly sensitive to the phrasing of prompts. While prior research has explored the impact of prompt strategies on accuracy, there is a lack of studies on runtime speed and comparisons with human performance.
Methodology
This study tested nine different prompt styles on the GPT-5.5 model across three accuracy tasks and two NP-hard speed tasks from the International Olympiad in Informatics (IOI). The experiments used analysis of variance (ANOVA) to assess the impact of prompt styles on task performance and descriptive statistics to compare AI with human IOI contestants.
Key Findings
- Diversity of Prompt Styles: No single prompt style was universally optimal. The best prompt style depended on the task type and whether results were summarized by the mean or median. For example, concise prompts were beneficial for speed in some tasks, while detailed prompts improved accuracy in others.
- AI vs. Human Performance: The AI outperformed the average human contestant in one task but fell short in the other two, indicating that AI performance in competitive programming is not strictly superior and is contingent on the task and prompt strategy.
- Limitations of Single Metric Evaluation: Many differences in mean scores came from a few outlier runs, suggesting that evaluating prompting by a single metric can obscure how much its effectiveness varies.
Engineering Trade-offs and Recommendations
- Prompt Style Optimization: Developers should tailor prompt styles to specific task objectives rather than adopting a universal approach. For instance, prioritize concise prompts for tasks requiring quick responses and detailed prompts for tasks demanding high accuracy.
- Multi-metric Evaluation: It is advisable to evaluate prompt strategies using multiple metrics, such as accuracy and speed, to avoid biases from single indicators.
- Human-AI Collaboration: AI excels in certain tasks but has room for improvement in others. Developers should consider combining AI with human expertise to leverage AI's strengths and compensate for its weaknesses.
Future Research Directions
Future research could explore the impact of different task types on prompt styles and design more intelligent prompt generation mechanisms. Additionally, studies could extend to other domains, such as natural language processing and image recognition, to determine if the impact of prompt styles on AI performance is consistent across fields.
— END —Source: ArXiv Machine Learning (cs.LG) (2026-10-23)
Tags: #Prompt Engineering #Competitive Programming #GPT-5.5 #AI Performance #Human-AI Comparison
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments