The Little Man in the Box

Hi from Multitouch Barcelona on Vimeo.

All of us anthropomorphise our machines, perhaps no more so than the car and the computer. Hi, A Real Human Interface from Multitouch Barcelona (an interaction design group that explores natural communication between people and technology) is a charming example of how we think about computers and interfaces from a human perspective.

Whatever we might know about the technology and how it works, we talk about the “server having some trouble” or our computers “having a bad day” or “going crazy”. We’re so biologically programmed for interaction to be with other beings, it’s very hard not to think of the little man in the box.

(Via @LukePittar and all the little people who run messages back and forth in the intertubes.)

Schematic and Public Multitouch Social Interaction

Touchwall Demo from Joel on Vimeo.

Joel Johnson’s exclusive (on Vimeo?) video and interview with the folks at Schematic about their new touchwall shows them dealing with some interesting public multitouch issues. I hate the marketing crap that goes with it and the inevitable Minority Report reference (please, stop making that reference multitouch people), but the idea that what they’re really interested in is “the social interaction in front of the screen” is spot on.

Apart from the fun of playing with what looks like a giant iPhone screen, the key thing about large multitouch screens is that more than one person can use it at once. If it just replicates a bank of individual screens it’s missing the point of having one big one. Connecting people together in social play and interaction can be really engaging and it will be interesting to see what developers and designers explore in this area.

The other issue that they talk about in the video is how to solve the identity problem on such a device so that you don’t have to walk up to it (or “into it” as one of the interviewees says) and type in a log-in. RFID tags come to the rescue, which means the wall knows who you are as soon as you’re close enough to use it.

If we’re going to make comparisons to Minority Report, that screen was an individual experience operated alone by Cruise’s character. By contrast a multi-user multitouch screen feels to me to be much more Star Trek or James Bond to me and about using collaborative workspaces with the added layer of data feeds.

Holographic Worlds and Gestural Interfaces

The Holodeck remains a fantasy for Trekkies and we’re still not yet jacked into The Matrix (or are we? Oooh.). Guys going to enormous lengths to build stuff for their girlfriends, on the other hand, has long been part of the human condition.

World Builder by Bruce Branit is about a guy who builds a holographic world for the woman he loves. There’s a reason it is holographic, which you find out when you get to the ending, so I won’t spoil it here. The film was shot in a day, but then took two years of post-production to finish off. Who says computers make things quicker?

The main reason for blogging it is because of some of the gestural interface elements in it. The overlay buttons and keypads are the usual fare and I remain unconvinced that jabbing at a floating holographic keypad button would be a useful UI approach, although it always looks good on screen. There are also some controls like spreading the fingers to enlarge and object and using the fingertips to rotate a virtual control knob that are already in use in gestural interfaces.

I’m not sure I have seen the idea of being able to pick up things like colours and textures on your fingertips and apply them to objects yet though in an existing multitouch interface. A few desktop applications use that kind of sticky mouse idea and 3D and 2D applications kind of use it with tools and colour/texture chips, but I still haven’t seen it all that smoothly done. Adobe seem to screw this up further and further with every release rather than making it easier. (Does CS really stand for ‘crappy shit’ rather than ‘creative suite’?)

The main issue with a gestural or multitouch interface would be keeping track of the identity of a particular finger tip once it has left the touch panel, it seems to me. But maybe someone has already solved this and it is in use – let me know if you know more.

(Thanks to one of my ex-students, Nico Marzian for mailing me the link).

Code Beneath the Surface

This developer from Infusion is showing off some of his modifications to Microsoft’s Surface at I Live To Code. The table has several cameras underneath instead of just one, so that he can affect the ripples and other interactions on the surface without touching it.

Perhaps the most interesting thing about the demo is the “new gesture” for tilting where he places the palm of his hand on one side of the screen and uses his forefinger and thumb to change the tilt angle. I’ve been trying to think what this is the equivalent too and it feels a bit like adjusting anything on a pedestal or tripod where you have to hold one part still to move the other. I’m not convinced it’s a gesture that is going to catch on because the palm-down hand blocks half the screen.

(Regarding the “Sponsored by Microsoft” link – this is experiment for Playpen too. It’s a sponsored clip by Unruly Media who have a pretty good ethics code. They encourage honest opinions and don’t try to be stealth marketers. I’m not entirely sure I want to have a great deal of sponsorship on Playpen, but the clip interested me anyway, so we’ll see. If you absolutely don’t want to give me an 18 cent kickback, you can watch it on YouTube)

Night of the Living Maps


Apart from last night’s making of history it was a night of interactive maps gone wild.

The BBC’s virtual studio 3D environment was replete with sounds of steel shutters opening and closing as the graphics changed, which gave me flashbacks of The Day Today. Wired have a good selection of other overblown 3D madness. (Can’t believe Wired Gadgets Lab used the term ‘gee-whiz Tech’).

CNN (above) went for multi-touch action with John King zooming in and out of detail and pulling up man-on-the-street video clips. The strange, meta-media, thing here was that the cameras then zoomed into the clip playing on the multitouch screen rather than cutting directly to it. King would then get rid of a clip by tossing it off of the top of the screen. King was keen to show off – quote of the evening: “I want to show you a new feature of the map – let me hit Hispanics here…”

I couldn’t face watching CNN’s ‘Situation Room’ coverage long enough to see if it went wrong (I mean, come on, Situation Room? Pricks.).

Kottke has a gathered a selection of online election maps and it’s a good lesson in information design styles. NetLab’s Kazys Varnelis and Leah Meisterlin have written an in-depth piece on Adobe’s Think Tank site about this shift into intelligent maps and what it means for designers.

“The choice of what to show and how to show not only impacts appearance, it can reframe arguments.”

Of course, maps always have been about framing and re-framing.

The online U.S. Election 2008 map I found clearest and most insightful is from The New York Times.


The Exit Polls Map is particularly good, especially when you set it to size the bars according to size of the electorate revealing just how much minority and female votes had a massive impact. Sliding through the years is enlightening too.

Naturally this trend has brought some good parody too – see Stephen Spielberg Presents John King and a Saturday Night Live skit.

[UPDATE: The Onion just posted How to Understand the Election Map]

But none of them beat Alan Partridge trying to explain the ’94 World Cup system on The Day Today:

The MultiTouch Cell

multitouch-pictures-two-users-two-cells 1.jpg

MultiTouch have just launched the “world’s first modular multi-touch LCD screen that can be used to create large tables and wall screens.” They are LCD screens with multitouch capabilities built-in and can be stacked and configured into many different formats. So far they have built a 6m long wall, but theoretically it be as long as you like providing you have enough machines to power them.

The Cells are also very robust in terms of environment – they can work in bright light (as well as the dark of course) and tracks both fingers and hands for some more complex gestural interaction.

The system also uses their Cornerstone SDK, which should be launched soon which anyone should be use also outside of the Cells.

I would imagine we’ll be seeing quite a few of these around all sorts of public spaces soon. Take a look at the high quality video of it in action. No word on pricing yet though.

(Thanks to John from 3Eyes who has been working on it for the info.)

3M Interface – Reverse Multitouch


My brother, Matt, just e-mailed a link to this interface on the 3M website. Given the multitouch hype at the moment, it’s quite a clever little riff on the theme.

Basically it’s as if you are standing to the rear of a multitouch screen. Your mouse controls the finger movements of the person blurred out in the background and a selection does the old two-finger click-and-drag-larger movement that seems to have become a multitouch standard.

Interactive Gestures Wiki

Multitouch screens of all shapes and forms are really all the rage, but with them come whole new paradigms of interaction. Do you wave like a Wii or do the Minority Report hand swipe popular in many kinds of large-screen set-ups.

Interaction designer, Dan Saffer, who also wrote Designing For Interaction has written a call to arms for interaction designers over at Adaptive Path. He points out some of the issues and dangers if these ideas aren’t documented and resolved. Standards help everyone and sharing knowledge of what you’ve played with, what works and what doesn’t is essential. As Dan says:

And if we wait, well, we’ll simply find individual companies (Apple, Microsoft, Perceptive Pixel, etc. etc.) creating their own standards (as is being done now). And while this isn’t necessarily a bad thing, one can easily imagine having to remember a crazy amount of movements and gestures for common actions. (”Wait, to turn on the lights do I tap the wall, or wave a hand? Is this an iRoom or MS Rume?”) We’ll get a lot of ad hoc solutions – some of which will be great, some not so much. Standards and a pattern library would help.

In order to gather this knowledge together in one place, Dan has started an Interactive Gestures Wiki. It’s already interesting to nose around and see just how many gestures and ideas are already out there, but it needs some filling in from us all.

[tags]multitouch, interactivity, gestures, physical interaction, dan saffer, wiki[/tags]

Where to now with multitouch?

I’ve been doing a bit of catching up with my blog reading recently and noticed Chris’s post on Pixelsumo about the HP giant Multi-Touch screen with the interface created by Darren David. Now that mutltouch has become the dish du jour, it’s time to start working out what to do with it, as Chris points out:

More and more developers are now creating multi-touch screens, without really asking WHY. Now that the technology is open and there are communities available to help, this takes away the initial learning curve. A criticism of all these kinds of projects for me is that the model of interaction doesn’t change. Han, iPhone, Surface and this project all do the two finger drag to stretch a photo, rotate it etc. Who needs to throw a photo around a screen? Unless the interface itself is a toy and a showcase, rather than concentrating on meaningful interaction or function.

Like all new technology, we are just getting to grips with it. It will be interesting to see where it goes next, or if it dies from lack of new creative ideas.

I suspect throwing photos around the screen is probably quite good fun actually, for a while. But he’s right in asking ‘where next?’. I’m really happy to see the cost of these systems falling and people like the NUI Group putting together open-source libraries and research into sensors and multi-touch. The easier it becomes for people to play wit these technologies, the more likely it is that some interesting ideas will be generated and/or found.

Until then, there is the danger that it’s all done for the sake of it, as this brilliant Surface parody shows:

(Thanks to Nic and Iain for that video link).