arXiv Research: Decoupling Prediction and Parameter Recovery in Knowledge Tracing
Summary:arXiv has published a study on Knowledge Tracing (KT) models, focusing on the decoupling of predictive accuracy and parameter recovery. The research reveals that high predictive performance does not guarantee accurate recovery of item parameters, especially for discrimination and option slopes. Through simulation experiments, the study examines the impact of different encoder-decoder combinations on parameter recovery and proposes a method to optimize recovery by separating sequence representati
Background and Motivation
Knowledge Tracing (KT) models are widely used in the field of educational technology to predict learners' next responses. However, there is a decoupling issue between the models' predictive performance and their ability to recover item parameters (such as difficulty, discrimination, and option slopes). Existing KT models typically rely on improving predictive accuracy, but this does not always translate into accurate recovery of item parameters.
Methodology and Findings
The research team conducted a systematic analysis of different combinations of 2PL, GPCM, and NRM decoders with LSTM, Transformer, and DKVMN encoders. Through simulation experiments, the study found that:
- Decoupling of Prediction and Recovery: Under shuffled sequence order, recovery performance for difficulty, mean threshold, and correct option intercept remains high, but recovery of discrimination and keyed option slope requires wider item representations and degrades further under fixed order.
- Optimization Method for Parameter Recovery: By separating the sequence item representation width ($d_S$) from the item parameter key width ($d_M$) and using a wide key with a narrow sequence representation ($\mathrm{SK}(64,8)$), the method reduces ability distortion under fixed order and improves parameter recovery and cross-order consistency without increasing model size or sacrificing test accuracy.
- Experimental Validation: Experiments on TIMSS and EdNet datasets show that widening $d_M$ consistently improves agreement with offline IRT references, while the trade-off from narrowing $d_S$ depends on the encoder's reliance on sequence width.
Engineering Trade-offs and Practical Applications
- Trade-off Analysis: The method of separating sequence representation and parameter key width enhances parameter recovery while maintaining model compactness, avoiding the computational overhead of increasing model size.
- Practical Application Recommendations: For applications requiring high-precision parameter recovery, it is recommended to adopt a wide key and narrow sequence representation approach and calibrate parameters using offline IRT references. Additionally, developers should consider the sensitivity of different encoders to sequence width to choose the most appropriate model architecture.
Conclusion
This study reveals the decoupling mechanism between prediction and parameter recovery in Knowledge Tracing and proposes an effective optimization method, providing new insights for the further development of KT models.
— END —Source: ArXiv Machine Learning (cs.LG) (2026-10-23)
Tags: #Knowledge Tracing #Parameter Recovery #Deep Learning #Educational Technology #Model Optimization
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments