Interfaces Beyond the Screen

We have grown accustomed to using screens and keyboards as the primary interfaces through which we interact with the digital world. However, as computers become ubiquitous, smaller and more powerful, we need alternative ways to interact with them. Chris Harrison is one of the most prolific researchers working on new interaction technologies. In this post, I briefly introduce Harrison and look at the main strands of his research.

Who Is Chris Harrison?

Currently a professor at Carnegie Mellon University, he directs the Future Interfaces Group (FIG). His lab develops unconventional ways for humans to interact with computers. Harrison gained his PhD at Carnegie Mellon in 2013 with his dissertation The Human Body as an Interactive Computing Platform, a title that sums up much of his career.

While interning at Microsoft Research, he built Skinput (2010), which turned the forearm into a touch surface by listening to the vibrations of a tap, and OmniTouch (2011), which used a shoulder-worn depth camera and projector to turn walls, tables and hands into touchscreens. Another early project, TapSense, told fingertip, nail and knuckle touches apart by their sound. In 2012, he co-founded the startup Qeexo to bring this kind of touch sensing to commercial devices.

His lab builds functional prototypes, often from off-the-shelf parts, and publishes several papers a year at top HCI venues like CHI and UIST.

Current Research

Touch on Any Surface

Modern AR/VR headsets are controlled with handheld controllers or mid-air hand gestures. Gestures have two problems: there’s nothing to touch, so you get no physical feedback, and holding your arms up quickly gets tiring (known as “gorilla arm”). FIG’s answer is to bring back real surfaces.

EgoTouch (UIST 2024) turns your palm and forearm into a touchscreen using only the regular cameras already in the headset. Besides detecting taps, it estimates how hard you press and which finger you’re using, and it works across different lighting conditions and skin tones, and even while walking.

The hard part is telling a touch from a finger hovering just above the surface. EclipseTouch (UIST 2025) uses infrared emitters on the headset to cast a shadow of your finger. As the finger approaches the surface, it meets its shadow, which lets the system detect touches on everyday surfaces like tables and walls with high accuracy. Together with VelociTrack (MobileHCI 2026), which uses high-speed cameras for the same goal, this work points to a future where any desk can become a keyboard or trackpad.

Smarter Eye Tracking

Knowing where someone is looking is very useful, but dedicated eye trackers are expensive and ordinary webcams aren’t very accurate. HiFiGaze (CHI 2026) relies on a neat observation: modern front cameras are sharp enough to see the screen reflected in your eyes. Since the device knows what it’s displaying, it can find that reflection and use it to work out where you’re looking, cutting tracking error by 18% compared to a standard camera-based approach.

Feeling Virtual Worlds

Virtual objects can look convincing but feel like nothing, as your hand passes straight through them. Reel Feel (CHI 2025) attaches thin strings to your fingertips, pulled by small motors in a worn unit. By controlling each string’s tension, one device can simulate a solid wall, something soft, a sudden impact or effects tied to a virtual object, all with less than 10 g of weight on the fingers.

Kinethreads (UIST 2025) scales a similar string-based idea up to a soft, full-body suit that can pull with up to 120 N of force, enough to simulate the weight of a held object. It weighs under 5 kg, costs around $400 to build and takes less than 30 seconds to put on.

Wearables Without Batteries

Charging one smartwatch is fine. Charging a dozen sensors spread across your body isn’t. Power-over-Skin (UIST 2024) uses the body itself as a power cable: a worn transmitter sends radio-frequency energy through the skin, powering battery-free rings, earrings and sensors anywhere from head to toe.

EverRing (ISWC 2025) takes a similar approach for smart rings, which are especially limited by their tiny batteries. It draws power wirelessly from the XR headset you’re already wearing.

Conclusion

One pattern stands out: wherever possible, the lab reuses hardware a device already has instead of adding new sensors. EgoTouch uses the headset’s cameras, HiFiGaze uses the front camera and screen, and EverRing borrows the headset’s power. This keeps the ideas cheap and much closer to real products than many research prototypes.

Challenges remain. Headsets that constantly film your hands raise privacy concerns, strings and motor suits may feel awkward in public, and most of these systems have only been tested with small groups in the lab.

Even so, these prototypes hint at the devices we’ll be using in ten years. Personally, I’m most excited about EclipseTouch, since I find current VR controls exhausting. Turning any physical surface into a touchpad feels like the missing piece for working comfortably in a headset.

References

1. Chris Harrison. Personal website. chrisharrison.net
2. Future Interfaces Group, Carnegie Mellon University. Research. figlab.com/research
3. Chris Harrison, Desney Tan and Dan Morris. 2010. Skinput: Appropriating the Body as an Input Surface. CHI 2010. doi:10.1145/1753326.1753394
4. Chris Harrison, Hrvoje Benko and Andrew D. Wilson. 2011. OmniTouch: Wearable Multitouch Interaction Everywhere. UIST 2011. doi:10.1145/2047196.2047255
5. Chris Harrison, Julia Schwarz and Scott E. Hudson. 2011. TapSense: Enhancing Finger Interaction on Touch Surfaces. UIST 2011. doi:10.1145/2047196.2047279
6. Vimal Mollyn and Chris Harrison. 2024. EgoTouch: On-Body Touch Input Using AR/VR Headset Cameras. UIST 2024. doi:10.1145/3654777.3676455
7. Vimal Mollyn, Nathan DeVrio and Chris Harrison. 2025. EclipseTouch: Touch Segmentation on Ad Hoc Surfaces using Worn Infrared Shadow Casting. UIST 2025. doi:10.1145/3746059.3747743
8. Nathan DeVrio, Vimal Mollyn and Chris Harrison. 2026. Demonstration of VelociTrack: Touch Input On Uninstrumented Surfaces Using High-Speed Headset Cameras. MobileHCI 2026. doi:10.1145/3821581.3833136
9. Taejun Kim, Vimal Mollyn, Riku Arakawa and Chris Harrison. 2026. HiFiGaze: Improving Eye Tracking Accuracy Using Screen Content Knowledge. CHI 2026. doi:10.1145/3772318.3791339
10. Nathan DeVrio and Chris Harrison. 2025. Reel Feel: Rich Haptic XR Experiences Using an Active, Worn, Multi-String Device. CHI 2025. doi:10.1145/3706598.3713615
11. Vivian Shen and Chris Harrison. 2025. Kinethreads: Soft Full-Body Haptic Exosuit using Low-Cost Motor-Pulley Mechanisms. UIST 2025. doi:10.1145/3746059.3747755
12. Andy Kong, Daehwa Kim and Chris Harrison. 2024. Power-over-Skin: Full-Body Wearables Powered By Intra-Body RF Energy. UIST 2024. doi:10.1145/3654777.3676394
13. Nathan DeVrio and Chris Harrison. 2025. EverRing: Powering Battery-Free, Highly-Capable Smart Rings with Headset RF Energy. ISWC 2025. doi:10.1145/3715071.3750420