Meta Boosts Robotics with New AI Touch and Dexterity Innovations

Meta’s Fundamental AI Research (FAIR) team is taking strides toward advanced machine intelligence (AMI) with the release of new robotics research artifacts and collaborations in tactile sensing. The team has unveiled significant advancements in touch perception, dexterity, and human-robot interaction, partnering with GelSight Inc. and Wonik Robotics to develop and distribute groundbreaking tools for the research community. These advancements aim to strengthen AI’s ability to interact with physical environments and lay foundational technology for more intelligent and responsive robotic systems.

Building a Foundation in Touch Perception and Dexterity

Meta's release includes tools designed to deepen touch perception and enable refined dexterity in AI-driven robots:

• Meta Sparsh: The first general-purpose touch representation that functions across multiple sensors and tasks, enabling AI to perceive details beyond vision.

• Meta Digit 360: A fingertip-shaped tactile sensor with multimodal sensing capabilities similar to human touch, able to detect fine spatial details and forces as minimal as 1 millinewton.

• Meta Digit Plexus: A hardware-software platform that integrates various tactile sensors onto a single robotic hand for coordinated touch perception and motor control.

These innovations hold promise for practical applications in fields like healthcare and manufacturing, where machine dexterity and perception are crucial for complex tasks.

Partnerships to Expand Access to Tactile Sensing Technologies

Meta is collaborating with industry leaders to make these technologies accessible to researchers globally:

GelSight Inc. will manufacture and distribute the Meta Digit 360 sensor, expected to be available next year. Early access will be granted through a Digit 360 call for proposals.

Wonik Robotics will produce a new generation of the Allegro Hand, equipped with Meta Digit Plexus and tactile sensing, also slated for release next year. Researchers can fill out this interest form to stay updated on this progress.

“Partnering with Meta on Digit 360 came from an immediate agreement on vision,” said Youssef Benmokhtar, CEO of GelSight Inc. “We want to encourage researchers and developers to embrace this technology in their research and make tactile sensing ubiquitous.”

Dr. Yonmook Park, Executive Director at Wonik Robotics, added, “Wonik Robotics and Meta FAIR aim to introduce robotic hands to global companies, research institutes, and universities so they can continue developing robotic hand technology that is safe and helpful to humankind.”

PARTNR: A Benchmark for Human-Robot Collaboration

To advance robots' social capabilities, Meta has introduced the Planning And Reasoning Tasks in humaN-Robot collaboration, or PARTNR, benchmark, a new tool for assessing planning and reasoning in human-robot interaction. Built on Habitat 3.0, PARTNR offers a high-speed simulation environment that enables scalable testing of AI-human collaboration in over 100,000 tasks. This benchmark enables systematic testing across various collaboration scenarios, helping developers improve AI planning, perception, and skill execution in social settings.

Vision-Based Tactile Sensing with Sparsh

Meta has also released Sparsh, a new vision-based tactile sensing model capable of processing inputs from various tactile sensors. Trained on a dataset of over 460,000 tactile images, Sparsh outperforms traditional, task-specific models and supports more efficient scaling of touch perception technologies.

Enabling New Frontiers in AI and Robotics Research

Meta FAIR’s commitment to open-source collaboration and its partnerships with GelSight Inc. and Wonik Robotics will broaden the accessibility of robotics technologies, allowing researchers to innovate in fields like medical robotics, virtual reality, and supply chain management.

Looking Ahead

As embodied AI evolves, Meta’s advancements in touch perception and human-robot collaboration are creating a foundation for robots that interact safely and intelligently in human environments. With its expanded research tools and hardware platforms, Meta is working to empower the community to explore new applications for robotics, moving closer to a world where AI systems can be true partners in daily life and industry. For more details about these innovations, please visit Meta AI's blog.