ArXiv Proposes New Evaluation Protocol to Enhance Performance of Deep Multivariate Imputation Models for Wearable Device

Summary:This research addresses the structured missingness problem in wearable device data by proposing an innovative evaluation protocol. By mining contiguous missing-run templates from training data and injecting them as block masks, the protocol significantly enhances the performance of deep imputation models like BRITS when handling multimodal physiological data. The study also extends BRITS with time-of-day encoding and a circadian harmonic channel, demonstrating the varying performance of differen

---

Background

Wearable device data is crucial for continuous health monitoring, but its quality is often compromised by structured missingness, where features sharing the same physical sensor drop out together. Existing deep imputation methods like BRITS and SAITS have seen limited evaluation on multimodal physiological data, and current benchmarks typically use random-point holdout protocols, incorrectly assuming that missingness is independent across features and time.

Key Contributions

1. Innovative Evaluation Protocol: This study proposes a novel evaluation protocol that mines contiguous missing-run templates from training data and injects them as block masks, preserving the co-missingness structure.
2. Matched Training Protocol: By matching the training protocol, the model is exposed to the same missingness distribution, significantly reducing the Mean Absolute Error (MAE) of the BRITS model in severe missingness scenarios by 43%.
3. Model Extension and Comparison: The study extends the BRITS model with time-of-day encoding and a circadian harmonic channel and compares the performance of different models across various missingness scenarios. The results show that no single model dominates: linear interpolation performs best for slow-moving features over short gaps; the extended BRITS achieves lower MAE on dynamic cardiac features in moderate and severe gaps; and SAITS better preserves the ground-truth distribution by Jensen-Shannon distance despite higher MAE.

Technical Highlights

• Innovative Evaluation and Training Protocol: By preserving the co-missingness structure, the protocol provides a more realistic evaluation method.
• Model Performance Improvement: The matching training protocol significantly enhances the performance of BRITS in handling severe missing data.
• Multi-Model Comparison: The study demonstrates the varying performance of different models across missingness scenarios, emphasizing the impact of evaluation design on model rankings.

Industry Impact

This research provides new insights and methods for imputing missing values in wearable device data. With more accurate evaluation and training protocols, future models can more effectively handle missing values in multimodal physiological data, thereby improving the accuracy and reliability of health monitoring. Additionally, the study offers critical insights for developing imputation strategies for multi-sensor datasets, driving the application of AI in the health sector.

Developer Recommendations

• Adopt the New Evaluation Protocol: Developers are advised to adopt the evaluation protocol proposed in this study to more accurately reflect the model's performance in handling missing data.
• Combine Multi-Model Advantages: Choose the appropriate model based on the specific application scenario, for example, prioritizing the extended BRITS model when dealing with dynamic cardiac features.
• Continuously Optimize the Model: Further enhance the model's performance in handling complex missing patterns by combining methods such as time encoding and circadian harmonic channels.

---

Source: ArXiv cs.AI (2026-08-25)

— END —

Tags: #Wearable Devices #Data Imputation #Deep Learning #Health Monitoring #Model Evaluation