Leap Forward
The hype has cooled, but the Leap is still exciting technology. Just within frog, there have been incredibly in-depth considerations of structured light, EF sensing, time-of-flight, sonar, or the most likely depth-from-focus.
But in the wider community, there hasn’t been much talk about what we’ll do with these sensors. Many interaction designers are quick to dismiss the potential of unbound hand-waving. After all, it is generally accepted that holding your arms up even to touch a screen can be fatiguing.
I believe, though, there are a host of casual gestures that can augment pointing, and are used rarely enough to assuage these ergonomic concerns.
Pointing and clicking, after all, isn’t everything; keyboard shortcuts are as old as the GUI (if you count Englebert’s mouse-full-o-buttons). With the advent of multitouch we’re finally seeing some more kinetic options such as two-finger scrolling, three-finger swipe between spaces, et. al. I think these are where we’ll see hand-waving and finger-waggling add value. A few examples:
Wave to switch: Brushing-aside motions to move between spaces or full-screen apps would be brilliant with an iPad stand but also useful on MacBooks.
Point to focus: Positioning a cursor is best left to pointer tools or touch, but when targeting a field or window, we don’t need such precision.
Flick to scroll: Window-focus and scrolling are common low-fidelity actions.
Secondary manipulation: E.g. drag up an icon with a pointer, while using another hand to move a window out of the way and uncover the drop destination.
Many of these, of course, also work with touch. But I expect spatial gesturing to require less attention and preserve flow.
There are also gestures like pull in and push out that deserve investigation, perhaps for zoom metaphors where pinching implies too much fluid granularity. (Have you ever used a touch UI where pinching expands or collapses elements with only a binary open/close state? Awkward!)
If we needed mouse-level precision for everything then our iPads would be collecting dust in drawers, and if we only used pointing for constant activities we’d be using chorded keyboards. So let’s not get distracted by silly gaming and “look at my precision” demos or Kinect similes. There are plenty of coarse-grained actions where quick hand motions will be valuable as casual gestures.