Input Devices and Interaction Paradigms

In 2015, Google released a demo showing how to control a virtual dial on your screen simply by rubbing your fingers together. There was no glove and no camera, just a tiny radar chip sensing sub-millimeter movements. This was called Project Soli.

Four years later, it shipped in the Pixel 4, only to be abandoned by Google a year later. In this post, I will look at what Project Soli was, why it failed despite its impressive technology, and whether it will make a comeback.

What Is Project Soli?

Soli was born at Google’s Advanced Technology and Projects group (ATAP), led by Ivan Poupyrev. It is a gestural, touchless interface, part of the broader family of Natural User Interfaces (NUIs). NUIs rely on abilities people already have, like moving, speaking or looking, rather than learned tools like a mouse. Soli aimed to make interaction feel more human by using natural hand gestures to control computers.

The team found that with radar at around 60 GHz, they could measure the echoes bouncing off your hand and use tiny changes in distance and velocity to track fine finger motion. Their SIGGRAPH 2016 paper describes how they shrank the whole system onto a single chip, small enough for a watch.

The team assigned specific gestures to actions: tapping thumb to index finger was a button, sliding the thumb along the finger was a slider, and rubbing it was a dial. The fingers touching each other provide physical feedback, something mid-air gestures usually lack. Radar can also work in the dark, through fabric and plastic, and raises fewer privacy concerns than a camera.

Motion Sense

Since Soli’s radar ran at higher power than US rules allowed, Google needed a waiver from the Federal Communications Commission (FCC), which was granted at the end of 2018. The Pixel 4 launched in October 2019 with a Soli chip in its top bezel under the name Motion Sense. Motion Sense actually mixed two paradigms:

1. Explicit interaction: quick gestures, where you deliberately give a command. The user could wave over the phone to skip a song, snooze an alarm, or silence a call.
2. Implicit interaction: presence and reach, where the device reacts to what you’re doing anyway. This is closer to Mark Weiser’s ubiquitous computing, where technology fades into the background. Presence was used to turn the always-on display off when the user walked away. Reach woke the face unlock sensors as the user’s hand approached so the phone was unlocked by the time the user picked it up.

Google did not ship the dials and sliders from the demo, perhaps due to reliability issues. Later models like the Pixel 4a and Pixel 5 dropped Motion Sense entirely. Google cited cost, but I think there were deeper reasons.

Why Did It Fail?

A Non-Existent Problem

On a watch, screen size is limited and using the space around the device makes sense. On a phone, the screen is already sufficiently large. Motion Sense only helped when the phone lay on a table near you, and even then, tapping was barely harder than waving. A new input method has to be clearly better to be worth learning. Waving was, at best, about the same.

Poor Feedback

The research promised precise micro-gestures; the product offered a left or right wave. Reviewers reported swipes going the wrong way or not registering at all. With nothing to touch, you couldn’t tell why a gesture failed. A faint glow at the top of the screen was far less reassuring than a physical click.

The Midas Touch

An always-on sensor must separate commands from everyday movement. Reaching for your coffee or gesturing while talking can look like a wave. Eye tracking research calls this the Midas touch problem: when everything can trigger an action, unwanted actions follow. Making the system stricter means it misses intended gestures instead. Either way, users lose trust.

High Cost

The Soli and face unlock hardware gave the Pixel 4 a thick top bezel while competitors were removing theirs. The 60 GHz band isn’t approved everywhere, so Motion Sense was disabled in several countries, and this was widely reported as the reason the Pixel 4 never launched in India. A feature that adds cost, takes up space and only works in some countries needs to be something people really want. Motion Sense wasn’t.

Conclusions

A Second Life

Soli didn’t disappear. The 2020 Nest Thermostat uses it to light up when someone approaches, and the second-generation Nest Hub (2021) uses it for Sleep Sensing, tracking breathing and movement from the nightstand without a camera or wearable. Both are implicit: nobody learns a gesture, and a radar’s privacy advantage really matters in a bedroom.

Even on the Pixel 4, the best-liked feature was the invisible one: face unlock that was ready before you picked up the phone.

Future Predictions

Soli shows that a technology can work while its interface doesn’t. As an explicit input on a phone, it competed with a touchscreen already in your hand, and lost.

The key lesson is the split between explicit and implicit interaction. Learning new gestures is a big ask, especially without physical feedback. Quietly sensing what people already do is a much easier sell, and that’s exactly where Soli ended up.

Does this mean that micro-gestures are dead? Definitely not. I believe that Soli was just deployed on the wrong device. It has great potential, especially in areas like AR/VR and ubiquitous computing. Current handheld controllers and hand gestures in VR feel exhausting and unnatural. A refined implementation of Soli can be used to accurately control virtual sliders and buttons in Extended Reality (XR).

Watches, smart glasses and headsets, with tiny or no screens, are exactly what Soli was designed for, and muscle-sensing wristbands (e.g., the Meta Neural Band) are now chasing the same goal. I am sure the thumb-against-finger dial will make a comeback one day.

References

1. Lauren Bedal and Leonardo Giusti. 2020. A New Interaction Language. Google Design Library. design.google
2. Jaime Lien, Nicholas Gillian, M. Emre Karagozler, Patrick Amihood, Carsten Schwesig, Erik Olson, Hakim Raja and Ivan Poupyrev. 2016. Soli: Ubiquitous Gesture Sensing with Millimeter Wave Radar. ACM Transactions on Graphics 35(4), SIGGRAPH 2016. doi:10.1145/2897824.2925953
3. Jaime Lien and Nicholas Gillian. 2020. Soli: Radar-based Perception and Interaction in Pixel 4. Google Research Blog. research.google/blog
4. Robert J. K. Jacob. 1990. What You Look at Is What You Get: Eye Movement-Based Interaction Techniques. CHI 1990. doi:10.1145/97243.97246
5. Mark Weiser. 1991. The Computer for the 21st Century. Scientific American 265(3), 94–104. doi:10.1038/scientificamerican0991-94