Wearable Device That Changes Perception of Softness Wins Best Paper at UIST 2021

When we say the world is at our fingertips, that’s because they play a major role in our perception of the environment around us. The fingertips are one of the most sensitive places on the human body, with thousands of touch receptors packed into a small stretch of skin and significant real estate in the brain’s sensory cortex. Fingertips are capable of finely-tuned activities such as typing or reading braille, while also detecting small changes in temperature, texture, and softness.

Now, a new wearable technology designed by researchers in the Human Computer Integration laboratory at UChicago CS seeks to fool those fingertips. With a plastic frame that wraps around, but not over, the fingerpad and a small motor, the device can stretch and compress the skin of the fingertip, changing the user’s perception of an object’s softness. A chopstick can feel like a pencil eraser, or touching a hard plastic toy can produce the sensation of pushing into rubber or pressing a button.

The research describing the project, led by predoctoral student Yujie Tao, received a Best Paper Award at the 2021 ACM Symposium on User Interface Software and Technology (UIST), one of the leading human-computer interaction conferences. The project also received the Best Demo Award from the UIST Jury.

Tao’s device, designed with Shan-Yuan Teng and Pedro Lopes, takes advantage of how we naturally sense whether an object feels hard or soft. Hard objects press against our fingerpad, causing the skin to conform to the object’s shape. Soft objects also push back on the fingerpad, but both finger and object conform to each other, increasing the surface area of the contact. Mechanoreceptors inside the skin sense this displacement, and send information about an item’s softness to the brain — the more surface area in contact, the softer the perception.

The device created by Tao and her colleagues tricks this system by pulling on the frame and gently squeezing the edges of the fingerpad, causing it to bulge slightly more than normal and restricting displacement. When the user touches a rigid object with the bulgier fingerpad, it feels softer, due to a larger contact area. But unlike other technologies that have attempted to manipulate the perception of softness by pressing directly on the fingerpad, users can still accurately perceive the other tactile details of what they are touching.

“We wanted our users to still feel the actual texture of the object itself,” Tao said. “Our design goal was to make sure that we are not blocking any existing sensation that people might have when touching physical objects, while augmenting the softness perception in any kind of interactive application.”

Those possible applications include making props for virtual or augmented reality more realistic, making appliances and other objects interactive, or helping designers working with hard 3D-printed objects simulate the feel of a softer material. The researchers used some of these situations to test the effectiveness of the device, including asking blindfolded subjects to touch a series of rubber blocks and use a 3D printed video game controller with rigid buttons, with and without the fingerpad restriction.

The tests revealed that the device could significantly change a person’s perception of softness, from approximately the rigidity of a skateboard wheel to the softness of a pencil eraser, Tao said. Subjects also reported realistic haptic feedback, such as the feeling of pressing down a button on the gamepad even though it was stationary.

“I think what’s interesting about this project is, it all comes from your own body’s perceptions,” Tao said. “Our approach is like augmenting your finger, your own body, it changes your physical experience without really changing the environment.”

Previous efforts to produce this interactive softness have focused on instrumenting the environment, altering the target object instead of the person feeling it, or covering up the user’s fingertips entirely, which only allows it to transform the softness of virtual objects. Instead, Tao’s device directly modifies the subjective experience of the user, without covering their fingerpads — a strategy also used in other Human Computer Integration Lab haptic projects such as Touch&Fold and MagnetIO.

“This idea of providing sensations to our fingerpads without covering them is almost paradoxical, which always puzzled me in a good way!,” Lopes said. “This has been the driving force for a series of projects where we are attempting to engineer a new kind of haptic device that can be useful all the time, rather than just while in virtual reality. In the case of Yujie’s device, this is not just useful for VR but even for interacting with physical matter, prototyping, and much more.”