Bring on the Gesture Based Operating Systems

I was moving iTunes onto my second monitor just now (placed to the right of my main screen), and the song playing just happened to fade 100% right as I did it. My initial response was, “Oh crap, what did I do?” Almost immediately, though, the sound went back to the left channel and my confusion waned. This wasn’t some newly unearthed OS X gesture, but merely a coincidental alignment of gesture and result. I think the fact that my brain created that causality speaks volumes about gesture-based operating systems in general, though.

Though there are lots of other reasons why I switched to a Mac (the ease of web development being chief among them), but the biggest difference in my mind between Windows and Macs is Cupertino’s love of gesture-based interfaces (which I share). My last Windows machine was a small 8″ tablet running tablet XP–the form factor was perfect for me, but the touch as an interface was way behind the mouse. I build quite a few little interface prototypes in Processing at the time, but there really wasn’t a way to take a small little self-contained java demo and push its conventions onto the OS as a whole (I’m certainly not an OS programmer).

Between the built-in gestures in OS X, the iPhone, multitouch trackpads, and the new multitouch mouse, Apple is kicking Microsoft’s ass on the gesture front. Surface and the Courier are promising, but neither of them are exactly nearing the consumer market at this point. Why not build a touch controller for the XBox? Something small (maybe, uh, Zune sized?) which could act as a secondary display for inventory or controls. There’s nothing stopping them from doing this, and assuming it was available to XNA developers this would instantly get me interested in building more games for the platform (the Zune requirement would mean even less people would buy them, but heck–no one’s buying indie games anyway).

There are rumors swirling that the new Apple tablet will have somewhat of a learning curve, so I’m hoping it’s some kind of new gestural interface. Since it’s not being unveiled for another couple of days, I thought I might as well fantasize a little about my ultimate tablet device:

• An 8″ convertible multi-touch screen with a physical keyboard (I know the Apple won’t have one, but I think an 8″ keyboard is about the smallest still-functional keyboard–and it blows away any virtual keyboard I’ve ever used)
• In lieu of a physical keyboard, a way to dock the thing to a physical keyboard for extended typing.
• An IR emitter with a rich interface for controlling the TV (and a cloud-based Tivo would be nice, too)
• I’d love to be able to just “fling” content from a tablet PC onto a desktop when in blue tooth proximity. Just grab the file, do a little fling gesture, and the file magically lands on the other computer’s desktop. No cords needed. iPhone tethering for internet access on the go–or just toss in 3G to the device itself
• The same compass/accelerometer technology currently used in the iPhone.
• A system for slaving the device to a full computer for use as a tablet-based input device. I can’t count how many times I wished I could plug my tablet XP machine into my full desktop running Photoshop to do a quick sketch. Actually, this is dreaming small–I want any piece of hardware to be able to take control of the thing and use it however it wants. Alarm clock dock? Sure. X-Ray machine? Sure. Car dashboard? Sure.