What exactly is an interface anyway? In its simplest sense, the word refers to software that shapes the interaction between user and computer. The interface serves as a kind of translator, mediating between the two parties, making one sensible to the other. In other words, the relationship governed by the interface is a semantic one, characterized by meaning and expression rather than physical force. Digital computers are “literary machines,” as hypertext guru Ted Nelson calls them. They work with signs and symbols, although this language, in its most elemental form, is almost impossible to understand. A computer thinks—if thinking is the right word for it—in tiny pulses of electricity, representing either an “on” or an “off” state, a zero or a one. Humans think in words, concepts, images, sounds, associations. A computer that does nothing but manipulate sequences of zeros and ones is nothing but an but an exceptionally inefficient adding machine. For the magic of the digital revolution to take place, a computer must also represent itself to the user, in a language that the user understands.
Representing all that information is going to require a new visual language, as complex and meaningful as the great metropolitan narratives of the nineteenth-century novel.
Put simply, the importance of interface design revolves around this apparent paradox: we live in a society that is increasingly shaped by events in cyberspace, and yet cyberspace remains, for all practical purposes, invisible, outside our perceptual grasp. Our only access to this parallel universe of zeros and ones runs through the conduit of the computer interface, which means that the most dynamic and innovative region of the modern world reveals itself only through the anonymous middlemen of interface design.
[Quote adapted from Johnson, S. (1997) Interface Culture. Harper Collins (pp.14-19).]
I like this just because they’ve taken the everyday equation (bigger displays are always better) and turned it on its head. I’m not sure that any, or many, of the applications they show in the video are that useful—and the photo sorting thing really is becoming a bit of a cliché—but one could easily imagine this type of interfacing becoming useful with ’swarms’ of mobile devices.
There have been four posts so far on this blog about multi-touch interfaces: Jefferson Han’s work (along with the iPhone) here and here, Microsoft’s Surface, and Reactable. Why? Firstly, I love ‘em. Secondly, I think they will soon become the norm as far as human-computer interfacing goes.
However, the first commercially available multi-touch interface must surely be the JazzMutant Lemur, released around October 2005. This is an audio-media specific control surface that is “able to track an unlimited number of fingers at once” according to their website. It’ll work with all the major DAWS, and will even interface with Flash. Its controller software includes a whole range of presets objects such as faders, rotary controllers, sliders, pads, scopes, switches, and various readout/LED options, and it will allow you to build almost anything:
There’s loads more info on the JazzMutant site: technical description, image galleries, and some strangely silent videos.
What a great piece of kit, and it’s a shame it’s marooned in the boondocks of the music technology industry. In fact, if I was a venture capitalist I would buy JazzMutant and get this thing out into the mainstream of the computing world now! First off, I’d invest in top-notch and heavy-duty presets for Photoshop, Illustrator, and Flash…. Then Google Maps, iPhoto, etc., etc..
I have previously posted several items about multi-touch interfaces: Jefferson Han’s work here and here, and the Microsoft Surface here. I was therefore quite excited to come across Reactable, described by its developers at the Pompeu Fabra University of Barcelona as a “musical instrument with a tangible user interface”.
However, that’s actually a pretty daft description of it (in oh so many ways): it’s simply a multi-touch interface to a virtual studio built using PD. It’s constructed in the same way as the Microsoft Surface, with the touchscreen positioned above a camera and projector. Here are the Reactable ‘Basic Demos’ 1 & 2:
Pretty neat. I can see there might be major problems using it—e.g. interfacing, playing a tune, remembering patches or sequences, and it’s not exactly portable—but I would love to see something like this as (part of) an interface to a commercial synth or something like Reason.
Back in January I had a good look at the iPhone and Jefferson Han’s work on multi-touch interfaces (here and here). And for those of you looking to gain some kind of contextualization on this fascinating and highly topical area of interface design I’ve recently come across Bill Buxton’s historical overview.
I casually mention these only by way of introduction to Microsoft’s (ahem) “new paradigm in computing”, the Surface computer. Here’s one of Microsoft’s own promotional videos:
For something a little more illuminating, and that briefly includes schematics showing the innards of the Surface:
And for the inner geek, here’s a full 18-minute test-drive of the thing:
Yes, it’s pretty impressive (although the thought of that bog-standard PC running Windows Vista hiding inside is a bit off-putting). Some thoughts:
It’s not clear how the security issues will work. I mean, have you ever transferred data from one device to another without generating security prompts? As these will be public devices it seems inconceivable that security will not be a huge issue, and yet not once do we see anyone even inputting a PIN number in any of the videos. As if!
Will all manufactured objects become ‘tagged’ in the near future to allow interfacing with surface computers?
If so, will there develop a universal tagging language that will be understood by all “surface-compatible” products?
Can we predict a new job description: Surface Designer?
I do think there is a certain inevitability about this type of product.
The ability of the Surface to act as a ‘docking station’ for mobile devices calls to mind one of McLuhan’s Laws of Media: if you push a technology to an extreme it flips over into it’s opposite. In other words, as mobile devices have gotten smaller and yet more powerful, the tendency for miniaturization flips over into single large device that many of them can simultaneously attach to like a Mother Ship.
Doesn’t the Microsoft Surface remind you of those black glass-topped gaming tables you used to find in pubs? Space Invaders, anyone?
Of course there are those who quite rightly question Microsoft’s presumptuous and overblown claims for their product: British multi-touch interface designer Andrew Fentem has a reasonable and well-argued critique of both Microsoft and Jefferson Han here. Fentem’s own Spaceman Technologies website is well worth checking out by multi-touch aficionados.
Finally, irresistibly, if only to puncture the corporate pomposity of Microsoft:
In a recent post (Interfaces of the Future?) I included a video of Jefferson Han’s multi-touch interface. Well I found this one the other day that goes a long way towards answering many of the questions I posed:
It’s real! And notice the mention of “the $100 laptop”? Are we all going to be walking around with these things under our arms in a couple of years?
In a recent Alertbox column Jakob Nielsen looked at the way computer interfacing was represented in films and on TV. I’m not going to reiterate the content of his fine article here, but it did get me thinking about our preconceptions and expectations of the interfaces we use (or wish we used).
There are a whole series of interrelated problems, all based around the fundamental problem of too much information: too much to fit on screen, too much to manipulate, too much to think about. So - the theory goes - there must be some better way to manage this than the current QWERTY keyboard and mouse paradigm that dominates mainstream computing…
How about this:
Impressive huh? Or is it all smoke and mirrors set to a cool soundtrack? Looking at it closely, we can see that it’s simply a touch screen, but one that has multiple simultaneous sensing points. What’s more, there’s a shot of two people working on the machine together, so one can only guess that it recognizes somewhere around 16-20 sensing points at any one time. Apart from the nice-looking but probably-not-all-that-useful lava lamp blobs, examples of programs being used in the video include image manipulation software, word games, text entry, a Missile Command-type game, a real-time music program such as Max/MSP, and some kind of activity using molecular objects. There’s also evidence of a working toolbar. So, yes, impressive actually.
Evidence, then, of a functioning example of an innovative computer interface, albeit in a university research lab. Is it realistic to expect interfaces like this to appear in the near future? Is it Tomorrow’s World or Star Trek? Well:
Sorry, I’m not trying to jump on the “isn’t the iPhone fabulous” bandwagon but it’s interface - which you’ll note features a touch screen that recognizes at least two simultaneous sensing points - is certainly pretty exciting. And it’s on a hand-held device. And it’s coming to a shop near you, very, very soon.
