T-Rex: Tactile-Reactive Dexterous Manipulation
UC Berkeley, NVIDIA and Stanford train a tactile-reactive policy that adjusts its grip as it feels contact. It runs on a Dexmate Vega with Sharpa Wave hands, and the 100-hour dataset collected on that robot is public.
- 1UC Berkeley
- 2NVIDIA
- 3Stanford University
- 4Panasonic
- 5Sapienza University of Rome
- 6ItalAI
* Equal contribution
arXiv preprint arXiv:2606.17055
Abstract
The ability to react dynamically to tactile signals has long been considered crucial to agile human-level dexterity. Yet contemporary learning-based Vision-Language-Action (VLA) models for robotic manipulation generally either overlook the tactile modality or are limited to encoders with static cues, due in part to the scarcity of diverse training data and standardized evaluation, architectural constraints in current VLA models, and limitations of static tactile encoders. In this paper, we push the frontier of tactile-reactive manipulation by addressing all of these limitations. We propose a large-scale, 100-hour tactile-rich dataset collected via a novel, data-efficient recipe that prioritizes elementary motor primitives. To effectively exploit naturally high-frequency touch signals without sacrificing the existing capabilities of existing VLAs, we introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder. We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation, achieving over 30% higher average success rate than the strongest baseline.
Citation
@misc{niu2026trextactilereactivedexterousmanipulation, title = {T-Rex: Tactile-Reactive Dexterous Manipulation}, author = {Dantong Niu and Zhuoyang Liu and Zekai Wang and Boning Shao and Zhao-Heng Yin and Anirudh Pai and Yuvan Sharma and Stefano Saravalle and Ruijie Zheng and Jing Wang and Ryan Punamiya and Mengda Xu and Yuqi Xie and Yunfan Jiang and Letian Fu and Konstantinos Kallidromitis and Matteo Gioia and Junyi Zhang and Jiaxin Ge and Haiwen Feng and Fabio Galasso and Wei Zhan and David M. Chan and Yutong Bai and Roei Herzig and Jiahui Lei and Li Fei-Fei and Ken Goldberg and Jitendra Malik and Pieter Abbeel and Yuke Zhu and Danfei Xu and Linxi Fan and Trevor Darrell}, year = {2026}, eprint = {2606.17055}, archivePrefix = {arXiv}, primaryClass = {cs.RO}, url = {https://arxiv.org/abs/2606.17055},}Platform
All real-world experiments use a fixed-base Dexmate Vega with two 22-DoF Sharpa Wave hands. The 100-hour teleoperation dataset was collected on the same robot and released, and other groups have since reused it for pretraining. Vega is available today as Vega 1 Pro and Vega 1 U. See all research published on Dexmate robots.
Related Research
Start Your Physical AI Journey with Us
