CMNIE: A Novel Benchmark for Chinese Military News Information Extraction Released

Summary:CMNIE, a novel benchmark for Chinese military news information extraction, has been released. It features joint annotations for event triggers, event arguments, named entities, and entity relations under a unified domain schema, with 13,000 instances covering 7 event types, 10 argument roles, 7 entity types, and 8 relation types. Experimental results show that while zero-shot LLMs can identify relevant semantic units, they struggle with exact span matching. CMNIE aims to provide a standardized p

---

Key Breakthroughs

• First Benchmark for Chinese Military News IE: CMNIE is the first benchmark dataset for information extraction in the Chinese military news domain, filling a critical gap in the field.
• Multi-Task Joint Annotation: The dataset includes annotations for event triggers, event arguments, named entities, and their relations, supporting multi-task joint modeling.
• Large-Scale and Fine-Grained Annotation: With 13,000 instances, it covers 7 event types, 10 argument roles, 7 entity types, and 8 relation types, providing comprehensive and detailed annotations.

Technical Highlights

• Unified Domain Schema: A unified domain schema ensures data consistency and scalability.
• Challenging Evaluation: Experiments demonstrate the challenges existing supervised models and zero-shot LLMs face in relation extraction and event argument span matching, highlighting the dataset's complexity.
• Standardized Platform: CMNIE provides a standardized platform for studying schema adherence, exact span matching, and joint structured extraction in specialized Chinese news.

Industry Impact

• Intelligence Analysis: Offers intelligence analysts a more efficient and accurate tool for information extraction, supporting rapid decision-making.
• Knowledge Base Construction: Helps build a more comprehensive Chinese military knowledge base, enhancing knowledge management and retrieval.
• AI Research: Provides AI researchers with new directions and benchmarks for information extraction tasks in specific domains.

Recommendations for Developers

• Model Optimization: Developers should focus on optimizing models for relation extraction and event argument span matching to improve performance on the CMNIE dataset.
• Multi-Task Learning: Explore multi-task learning approaches to leverage the multi-task joint annotation features of CMNIE.
• Domain Adaptation: Consider domain adaptation training for models to enhance their performance in the Chinese military news domain.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-09-11)

— END —

Tags: #Information Extraction #Chinese Military News #Large Language Models #Benchmark Dataset