GTAGym Turns GTA V Missions into a Reinforcement Learning Testbed

Summary:GTAGym turns missions from GTA V into an automatically verifiable environment for evaluating and training computer-use agents. Its GTAVBench contains 103 online tasks across nine task families, plus 47 image-based shooting and sniping tasks. Agents interact through screenshots and keyboard-mouse actions, while engine-state checks determine task outcomes. Six frontier LLMs were evaluated on the benchmark. The best configuration reached 70.9% overall accuracy on GTAVBench-AGI, although aerial nav

---

GTAGym Makes an Open-World Game Verifiable and Trainable

GTA V task outcomes have traditionally relied on human judgment, making them difficult to convert into reinforcement-learning rewards. GTAGym reconstructs missions from the game's main campaign, reads engine state, and checks completion conditions automatically. This turns open-world tasks into an environment for parallel agent evaluation and training.

The GTAVBench benchmark contains:

• 103 online tasks spanning nine task families;
• 47 image-based shooting and sniping tasks built from GTA V imagery;
• A web server for online access, evaluation, and parallel testing;
• An agent interface based on screenshots and keyboard-mouse actions;
• Automatic success checks and reward signals derived from engine state.

The main contribution is not merely that models can play a game. It places perception, reasoning, action planning, and control in a single loop while addressing the lack of reliable automatic verification in open-world environments.

Replacing Human Annotation with Engine-Level Checks

Many game-agent evaluations depend on people watching recordings or merely checking whether a model produces a plausible answer. GTAGym instead inspects GTA V engine state to determine whether a target was reached, an object was destroyed, or another task condition was satisfied. This reduces subjective scoring noise.

The architecture supports parallel environment instances. A model receives screenshots and a task instruction, emits keyboard and mouse actions, and receives the next frame and status result after the game advances. Parallel instances improve throughput for evaluation and for collecting long interaction traces.

Automatic checking does not eliminate evaluation bias. A checker that examines only the final state may still accept paths that violate task semantics, including collisions, teleportation, line-of-sight abuse, or other unintended shortcuts. The task generator also samples parameters inside fixed templates. This improves scale and reproducibility, but narrows the distribution of possible tasks.

GTAVBench-AGI More Closely Resembles Real Computer Use

The standard GTAVBench setting provides textual instructions. In GTAVBench-AGI, the instruction appears only in the first screenshot, and a harness automatically compresses interaction history. This is closer to a real computer-use agent: it must extract the goal from visual input and retain it across subsequent actions.

Across six frontier LLMs, the best configuration reached 70.9% overall accuracy on GTAVBench-AGI. Performance nevertheless varied sharply by task type. Aerial navigation, evasion, and interception remained difficult, indicating that a model strong with textual prompts does not necessarily possess reliable visual perception, long-horizon planning, or fine-grained control.

History compression is another important variable. Growing interaction histories consume visual tokens, KV Cache, and inference latency; a compression strategy that discards the initial goal can cause the agent to drift. The 70.9% result must therefore be interpreted alongside prompt format, compression policy, action post-processing, and sampling settings rather than extrapolated to arbitrary open-world environments.

Scalable Task Generation for Reinforcement Learning

GTAGym's generator samples task configurations within user-specified ranges and option sets, then creates batches of 3D online and image-based shooting tasks. Every instance exposes automatic rewards and termination signals, allowing data generation, training, and evaluation to share one interface.

A fixed structure with configurable parameters offers a useful compromise. Tasks remain controllable while environments can be standardized. It supports research on parameter generalization—for example, varying target locations, enemy configurations, routes, or visual conditions to test whether a policy relies on superficial template cues.

In production, many 3D instances consume substantial GPU, CPU, memory, and game-process resources. Image-based tasks are cheaper but resemble static visual policies more closely. Training infrastructure should provision separate pools for these workload types and ensure that screenshot encoding, action injection, reward propagation, and fault recovery do not become serial bottlenecks.

LoRA Reinforcement Learning Produces a Modest but Measurable Gain

The paper applies LoRA-based reinforcement learning to Qwen3.8-27B using generated on-foot navigation tasks. The trained model improves on in-domain tasks, transfers to other task categories, and raises its score on native GTA V tasks from 15% to 23%.

The result indicates that parameter-efficient tuning can improve action alignment between a language model and the game environment. An eight-point gain, however, should not be interpreted as general game intelligence. Updating a small LoRA adapter usually reduces training memory and checkpoint overhead, but does not automatically solve visual recognition errors, long-horizon credit assignment, reward hacking, or distribution shift.

Reproduction should also separate the value of reinforcement learning from the coverage of the generated task distribution. If training and evaluation share maps, visual assets, or template configurations, apparent transfer may be data leakage. Splits should be defined by map, visual asset, template, and parameter combination rather than merely by individual task instance.

Practical Deployment Guidance

1. Validate every success checker: Test whether unintended actions can satisfy completion conditions, and report illegal-action, crash, and timeout rates.
2. Evaluate prompts separately: Compare textual instructions with first-frame visual instructions so prompt effects are not mistaken for model capability.
3. Track end-to-end cost: Measure frames, reasoning tokens, simulated time, and GPU utilization per task, especially for long-horizon aerial navigation.
4. Audit distribution overlap: Split by map, visual material, template, and parameter combination to rule out leakage.
5. Measure compression loss: Preserve critical information before and after history compression and compare full-context and compressed-context success rates.
6. Use workload-specific pools: Schedule expensive 3D training separately from cheaper image-based tasks, then merge or transfer policies according to task requirements.

GTAGym's primary value is that it converts GTA V from a human-operated game into an agent research platform with verifiable state, batch task generation, and automatic rewards. It does not yet demonstrate general-purpose game intelligence, but it provides a substantial engineering foundation for studying visual-language perception-action loops, task generalization, and reinforcement-learning transfer.

---

Source: Hugging Face Daily Papers (2026-11-09)

— END —

Tags: #GTAGym Follow #GTAVBench Follow #Reinforcement Learning Follow #Vision-Language Agents Follow #LoRA Follow