Jan 19, 2012

TL;DR - 1:1 manipulation and trigger gestures: thread carefully.


I've been noticing two different implementations of gesture controls on mobile and desktop devices.

The first one, is what it appears to be a gesture trigger, where an action is fired after the gesture reaches a certain threshold.

OSX gestures such as 'Show Desktop', 'Launchpad', 'App Exposé', all implement this pattern.

The second implementation is a 1:1 manipulation of the user interface.

Using the OSX gestures as reference once again, you can see an example of this approach in the 'Switch Space' gesture, where it even allows you to peek into the adjacent desktop and snap back without totally switching between them.

Which one is right?

I think there's room for both, but personally prefer a 1:1 manipulation of the UI.

The gesture trigger is sort of a fallacy in the metaphor, it's a button in disguise:

  • Feedback - triggers don't provide feedback until they're fired, that's bad for discoverability, bad for the metaphor of a natural user interface;
  • Perception - the lack of, or improper feedback affects user perception of the interface, it stops behaving like an object so the user no longer perceives it as such, your app is one step closer to a synthetic contraption.

Still speaking about the NUI metaphor, if you look at real world objects, their state doesn't vary between on or off, there are usually many intermediate steps.

This isn't true for buttons, switches and other man-made objects obviously.

But the real world is a mess!

Yes it can be a mess and I'm not advocating for things like 3D desktops where you can place your files anywhere or pin them to the virtual wall or whatever.

Our medium is digital, and even though we're designing with the real world in mind: masquerading our bits and bytes as flesh and bone, we should take advantage of that medium.

A digital bookshelf should offer the digital benefit of infinite storage, search; while at the same time present a familiar display of data.

Another way to eliminate that mess is to do intention handling and automatically set the state of the UI to a pre-defined one.

Take for instance the 'Switch space' gesture in OSX, or its brothers: the home screen switching in Android or iOS.

All of them allow for 1:1 manipulation of the interface, but once the user stops giving input via the touchscreen, the interface decides between returning to the previous state or jumping to the next screen.

Continuity & Consistency

A gesture should offer continuity.

Wether if it's a trigger or 1:1 manipulation, a gesture should offer continuous effect over the interface.

Don't force the user to stop and resume the control gesture in order to revert the state of the UI.

An example of a trigger gesture where continuity is respected is the 'Launchpad' gesture on OSX, you don't need to lift your hand to hide it, just perform the gesture in the inverse direction.

The same doesn't hapen with the multitask gesture on the iPad, where you have to stop touching the screen in order to hide the app tray.

Granted, these are artificial examples, as in: continuity isn't really required, but it serves to illustrate my point.

I would also say triggers and direct manipulation shouldn't be mixed in the same app, as it breaks the overall metaphor.

Performance

A very, very important factor in your app, if you offer 1:1 manipulation but if your interface lags way behind the user commands, your metaphor is broken.

To sum it up

Gesture implementations should be as well though as animations in your app. Both tell a story to your user and that story can vary between 'Hello, Frankenstein' and 'Hello, World'.

Notes

Another thing that may impact the overall application metaphor, and still talking about consistency, is the way gesture navigation relates to the information hierarchy.

See Path for example, you can drag the main screen to the left in order to reveal the friends list, but after you select a friend and the app scrolls the ui to the left, revealing the new content coming from the right, you would assume it's possible to swipe right in order to push the UI and go back to where you started. That is not the case, you have to resort to buttons in order to get back.

Flipboard isn't immune either. After selecting a topic, direct manipulation is discarded in favor of triggers, which is okay I guess but not consistent with the initial perception, plus, the trigger has a very low movement threshold. I've notived they're doing something smart here, a speed threshold is also in place and slow gestures are ignored.

One idea for Flipboard would be to make the left swipe bring back the latest selected topic when on the main page, this is sort of related to the continuity issue I raised above. Could perhaps be fitted into error handling as the user may accidentally perform a gesture that takes him back to the home screen with no way to get back (remember he'd been drilling down content on a specific topic and we shouldn't assume he keeps track of the topic at all times).

Also, if you look at certain OSX trackpad gestures videos in the System Preferences, like the Mission Control one, you'll notice they've been manipulated to look like 1:1 manipulation of the UI instead of how they really behave.

And yet another note, do play around with iPad multitouch gestures, the 'Send application to background' - 4-finger pinch-in, features 1:1 manipulation and even allows you to revert the action even when the animation has been 100% completed, provided you don't release the screen. Unfortunately, they first thing people try to do, which is a 4-finger pinch-out doesn't bring the app back to foreground.

More articles