Sony’s stealthy approach towards interactive augmented reality

Through luck or foresight, Sony appears to be converging on something of a home technology miracle – but to see their approach clearly we should first step back and take a look at the development of 3D.

Do we believe in 3D yet?

We had 3D video content for a long time, it just wasn’t good enough to become more than a novelty. A few years ago I caught a screening of The Creature from the Black Lagoon in the original old-school anaglyphic (red/blue) 3D. While it was an interesting novelty, it was clearly not a compelling  enough experience to beat movies in 2D and colour.

Despite it’s naysayers, the modern 3D cinema experience has gained so much traction that on any given trip to the multiplex you’re almost certain to find at least one new 3D release or another, and the box office takings continue to be respectable; the business case for cinemas to upgrade their projectors conveniently boosted by also including an upgrade to digital, killing two birds with one stone. It seems that audiences are prepared to accept the costs (financial, but also the inconvenience of wearing the glasses, not being able to tilt your head, and a slight reduction in brightness) since the result is (usually) sufficiently impressive. The fact that the conversation has moved on to the quality of the 3D (or lack of it, as seen in the hasty post-production processing 3D of the recent versions of Alice in Wonderland and Clash of the Titans) is surely a good sign for acceptance of the medium. Designer and prescription versions of the glasses also suggest that we are at the next stage of technology adoption.

On other screens, the field is still nascent. Predictably, the first consumer version of autostereoscopic 3D, with it’s look-no-glasses magic, is due to appear on a small screen (to make the cost bearable) designed for a single fixed-position viewer (as is at required by the technology), backed by an experienced player in innovative interfaces: the forthcoming Nintendo 3DS.

In television, active shutter 3D at first seems to be a strange proposition: each viewer must have a pair of active shutter glasses, which will seem expensive in comparison to the well-established polarisation glasses used in cinemas and available for some 3D TVs. On the other hand, the advantage is that many 120Hz televisions are already able to produce active shutter 3D imagery. Despite the perception of being uber-early-adopter territory, 3D televisions are effectively already here.

Then there’s the equally amazing fact that a few months ago Sony rolled out a PS3 upgrade to support 3D, removing another hardware barrier – 3D players are already here, in the form of 38 million PS3 consoles.

Meanwhile, in the console wars

Here’s where things get really interesting. Nintendo, Microsoft and Sony are all pushing for new modes of interaction for the games console. Nintendo took a huge gamble but secured an early lead with the Wii in 2006 (remember how the name first sounded to you and you’ll probably experience a flashback to just how crazy the whole idea seemed at the time).

Microsoft claim to have achieved interface nirvana with the entirely controllerless Kinect. Even the oft-cited screens of Minority Report needed a peripheral to operate, although it remains to be seen if it is as incredible as it seems, and accuracy remains a question.

Given the above, Sony’s decision to back what is widely seen as just a more accurate version of the Wii’s system seems a bit baffling. Being a PS3 owner myself, and curious to understand what Sony is thinking, I recently picked up the Move Starter Pack myself.

The answer became abundantly clear as soon as I tried the demo of Tumble, a very simple stack-em-up knock-em-down game. Your movements of the controller – including depth and rotation, which feels somehow much more impressive than movement in the plane – are mapped to an on-screen version that can pick up each brick (see image at top). It’s an impressive technological trick, but it then immediately demonstrates the next problem to solve: there is no depth perception, and you have to rely on a virtual shadow that indicates exactly which part of the playfield is directly below the object you are holding.

And so it suddenly becomes clear that Sony has brought all the ingredients together for interactive augmented reality. The 3D TVs are already here, the players are already here, and with the Move we suddenly have our 3D controller, which means the hardware for proper augmented reality in the home is pre-installed, just waiting for the right software. The final ingredient is the active shutter glasses, which simply paired with 3D viewing may seem expensive and clunky, but I suspect that image will fall away if you can put them on and then see yourself holding a lightsabre and interacting directly in a 3D virtual environment.

The fact this only works within a field-of-view that includes your TV screen is a limitation, certainly; and the question of whether or not all this can actually be used to create compelling games or usable interfaces remains to be seen – but we can rely on Nintendo to begin exploring this space intelligently with the 3DS, possibly followed by Apple, since the tablet form factor is the natural successor in autostereoscopic 3D.