Tactus: Open-Vocabulary Object Recognition from Low-Cost Pressure Arrays
By Mr.Xu
Published: · 8 views
Summary:A new arXiv paper introduces Tactus, an open model for open-vocabulary object recognition from low-cost resistive pressure arrays. It achieves 0.771 top-1 accuracy on the STAG benchmark (27 objects), surpassing the supervised closed-set CNN (0.76) without a trained classifier head. The recipe is small-data: 187 training recordings, masked-autoencoder pretraining on 144k unlabeled frames, and the sensor's calibration affine. Failures are reported with equal precision, and weights, code, and memor
Overview
Tactile sensing is a critical modality in robotics, but tactile representation learning has focused on optical sensors that image a deforming gel. Tactus addresses this gap by performing open-vocabulary object recognition directly from pressure data, answering text queries without a trained classifier head.
Technical Highlights
- Open-Vocabulary Recognition: Tactus matches arbitrary text queries, achieving 0.771 top-1 accuracy (top-3 0.935) on the STAG benchmark, surpassing the supervised closed-set CNN (0.76).
- Small-Data Recipe: Only 187 training recordings, combined with masked-autoencoder pretraining on 144k unlabeled frames and the sensor's calibration affine, yield high performance. The calibration affine recovered more accuracy than all architecture changes combined.
- Robustness Analysis: Errors concentrate in a few contact-ambiguous classes, are uncorrelated with text-target geometry (Spearman rho ≤ 0.05), and survive paraphrased and bare-name queries. Two diverse frames recover 89% of eight-frame accuracy.
- Failure Reporting: Cross-sensor pretraining pooling gave no gain, vision co-training degraded touch, and a mis-normalized input pipeline silently discarded 97% of the sensor's dynamic range. These findings provide important warnings for future research.
Industry Impact and Developer Advice
Tactus demonstrates the potential of low-cost tactile sensors in open-vocabulary recognition, offering new insights for robotic tactile perception. Developers can benefit from its small-data training strategy and the importance of calibration affine, while avoiding pitfalls like cross-sensor and vision co-training. Weights and code are open-sourced for reproducibility.
Source
Based on arXiv paper 2608.04043 (released August 6, 2026).
— END —Source: ArXiv Machine Learning (cs.LG) (2026-08-06)
Tags: #Touch #Open Glossary #Pressure array #Self-supervised learning #Robot
Community Comments