arXiv Releases OpenFrontiers: A Cross-Disciplinary Research Problem Repository and Scientific Rediscovery Benchmark

Summary:arXiv has released OpenFrontiers, a resource designed to support research-level problem solving systematically. OpenFrontiers includes a cross-disciplinary corpus of 12,151 open research problems and a benchmark of 200 mathematical problems, with 110 resolved in 2026 and 90 remaining open. The benchmark uses Lean 4 for formal verification and employs automated probes and independent reviews for quality control. The evaluation protocol tests LLMs under closed-book, frozen-retrieval, and live-web

---

1. Background and Motivation

Large Language Models (LLMs) have shown significant potential in research-level problem solving, but a systematic collection of open research problems is still lacking, and evaluating LLMs on such problems remains challenging. The release of OpenFrontiers aims to fill this gap by providing a cross-disciplinary repository of open research problems and a scientific rediscovery benchmark, thereby advancing the application and development of AI in the research domain.

2. Core Components of OpenFrontiers

• Open Research Problem Corpus: Contains 12,151 problems across disciplines such as mathematics, physics, chemistry, and biology. Each problem includes detailed records of the problem statement, background information, and progress.
• Mathematics Problem Solving Benchmark: Includes 200 problems, with 110 resolved in 2026 and 90 remaining open. All problems are formally verified using Lean 4 and quality-controlled through automated probes and independent reviews.
• HQ-50 Subset: A subset of 50 high-quality problems selected from the benchmark for more detailed evaluation.

3. Evaluation Methodology and Results

The evaluation protocol for OpenFrontiers tests LLMs under the following conditions:

• Closed-Book Condition: Models rely solely on pre-trained knowledge without access to external resources.
• Frozen-Retrieval Condition: Models can access a predefined retrieval corpus but cannot access the real-time web.
• Live-Web Condition: Models can access real-time web resources.

The results on the HQ-50 subset show that:

• Closed-Book Rediscovery: Is rare, with at most 7 problems solved.
• Live-Web Success: Primarily reflects the retrieval of published solutions rather than genuine rediscovery capabilities.

4. Technical Mechanisms and Engineering Trade-offs

• Formal Verification: Uses Lean 4 for problem formalization, ensuring the accuracy and verifiability of problem statements.
• Automated Probing and Independent Review: Combines automated tools and human review to ensure the quality of benchmark problems.
• Multi-Condition Evaluation: Provides a comprehensive evaluation of LLM performance in research problem solving through testing under different conditions.

5. Developer Implementation and Deployment Recommendations

• Research Applications: Researchers are encouraged to use the OpenFrontiers corpus for problem definition and exploration of solution paths.
• Model Training and Evaluation: AI developers should use the OpenFrontiers benchmark to train and evaluate LLMs for research-level problem-solving capabilities.
• Cross-Disciplinary Collaboration: Collaboration among researchers from different disciplines is encouraged to expand and improve OpenFrontiers.

6. Future Outlook

The release of OpenFrontiers provides a new platform and tool for the application of AI in the research domain. As more problems and solutions are added, OpenFrontiers is expected to become an important resource for research problem solving, driving the deep integration of AI and scientific research.

---

Source: ArXiv AI (cs.AI) (2026-10-30)

— END —

Tags: #OpenFrontiers Follow #Large Language Models Follow #Research Problem Solving Follow #Benchmarking Follow #arXiv Follow