arXiv Releases MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor with Systems Engineering Knowledge
Summary:arXiv introduces MARQ, a modular, LLM-agnostic prototype for assessing requirement quality by infusing Systems Engineering (SE) knowledge. MARQ enhances LLM performance using a criterion-specific knowledge stack based on INCOSE standards. Experimental results show significant improvements in F1 scores across models: Claude Opus 5 achieves 0.80, GPT-5.6 Sol achieves 0.62, and Gemma 3 12B achieves 0.53, outperforming baseline models by 45%, 19%, and 195%, respectively. However, the gains are model
1. Background and Motivation
Requirements Engineering (RE) is a cornerstone of Systems Engineering (SE), as defects in requirements can propagate downstream and severely impact development outcomes. While Large Language Models (LLMs) show promise in assessing requirement quality, prior work indicates that off-the-shelf models often miss a significant portion of expert-identified defects, perform unevenly across quality criteria, and exhibit execution variability.
2. MARQ Design and Implementation
MARQ is a modular, LLM-agnostic prototype designed to enhance LLM performance in requirement quality assessment by infusing Systems Engineering (SE) knowledge. Its core component is a criterion-specific knowledge stack based on INCOSE standards. MARQ aims to provide a model-agnostic solution to improve LLM performance in this domain.
3. Experiments and Results
The research team evaluated MARQ using Claude Opus 5, GPT-5.6 Sol, and the open-source Gemma 3 12B models on an expert-labeled NASA requirement set, with 100 executions per model.
- Claude Opus 5: MARQ improved the median F1 score from 0.55 to 0.80, with significant gains in defect detection and false-alarm control.
- GPT-5.6 Sol: MARQ improved the median F1 score from 0.52 to 0.62, with better detection but also more false alarms.
- Gemma 3 12B: MARQ improved the median F1 score from 0.18 to 0.53, approaching the performance of proprietary models, but remains highly variable.
4. Discussion and Future Directions
The results demonstrate that MARQ significantly enhances LLM performance in requirement quality assessment, though the improvements are model-dependent. MARQ underscores the importance of criterion-specific interventions and human-in-the-loop review to build more reliable RE support. Future work includes further optimizing the knowledge stack for additional quality criteria and exploring more efficient human-AI collaboration models.
5. Technical Trade-offs and Engineering Costs
MARQ's modular design and model-agnostic nature offer flexibility for application across different LLMs. However, this comes with additional computational overhead due to the injection of the knowledge stack. Furthermore, MARQ's performance is heavily reliant on the quality of the knowledge stack and the accuracy of the expert-annotated data, which increases data collection and model training costs.
6. Developer Recommendations
For developers, MARQ provides a viable solution to improve LLM performance in requirement quality assessment. It is recommended to customize MARQ according to specific application scenarios and quality criteria and to consider incorporating human review mechanisms to further enhance the reliability of assessment results.
— END —Source: ArXiv AI (cs.AI) (2026-10-28)
Tags: #MARQ #LLMs & Foundation Models #Systems Engineering #Requirement Quality Assessment #arXiv
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments