Messing with Reality

Google is rolling out a tool for Android users that enables them to superimpose realistic-looking and sounding computer generated images of bears, sharks, penguins, cheetahs, snakes – you name it – on to the real world as viewed through the phone’s camera.

You may think this is just another augmented reality (AR) gimmick along the lines of the Pokemon Go game, because, let’s face it, AR for quite some time, has felt like a technology in search of a problem to solve.

But as smartphone cameras and sensors continue to improve, the range of AR applications is expanding rapidly. Retailers are enabling customers to visualise what furniture and accessories would look like in their homes, while car showrooms can give prospective buyers a chance to see what cars would look like with different colours and specifications.

In industry, workers wearing smart glasses with heads-up displays like those used by fighter pilots, are receiving step-by-step instructions on how to identify and fix problems with engines and other equipment.

In short, AR is moving beyond the gimmick stage and into the practical application phase, presenting plenty of opportunities for app developers.

Education is seen as one particularly promising area for development, with apps like Froggipedia (iOS only) giving users the opportunity to dissect and interact with a frog to understand its lifecycle from tadpole to adult.

And the BBC’s Civilisations app (iOS and Android) gives uses the ability to view about 40 historical artworks and artefacts from all angles in 3D, from Egyptian mummies to bronze sculptures. Scanning then digitising real world objects is obviously a lot more time-consuming and technically challenging than creating cartoon characters, but it’s clear to see the potential for AR technology for galleries, museums and tour guides.

Self-learning could also become more interactive and fun using virtual teachers interacting with you in your kitchen or living room. Text books could come to life on the page. And on holiday, wouldn’t it be helpful simply to take a snap of an incomprehensible sign or menu and see the translation instantly pop up on your screen rather than have to stop and consult phrase books and online dictionaries?

The beauty and fashion industries have also been experimenting with digital make-up, hairstyle and clothes overlays using the smartphone’s front-facing camera. Such “try before you buy” services could help reduce the number of online shopping returns – a big issue for retailers.

One thing you really ought to try before you buy – unless you’re on a drunken stag do and beyond help – is a tattoo. An AR app like Ink Hunter (iOS and Android) will let you pick a design and see what it will look like anywhere on your body first before submitting yourself to the inker’s needle.   

The launch of Apple’s ARKit and Google’s ARCore platforms certainly gave a boost to the sector two years ago. Smartphones, making use of their accelerometers, gyroscopes and cameras, were effectively given 3D awareness of their environments.

But while ARKit seems more popular with developers – according to Github at least – the fact remains that Android phones are the most popular around the world. Yet some cheaper Android models don’t have all the required hardware, putting AR capabilities out of reach, and almost all models require ARCore to be downloaded on to the phone first.

Most Apple iOS phones, on the other hand, are ARKit compatible from the off, perhaps explaining this platform’s popularity with developers.

Unity – the firm best-known for its video game development platform – has become the dominant player in AR and VR content creation.

The key to making AR a success is making it as easy to access as possible and providing content that really entertains, informs and educates – to quote Lord Reith’s principles for the BBC. So far, technological challenges and a dearth of content has held AR back.

Blippar, the AR marketing firm that nearly folded last year after it found little appetite for its app that enabled branded products to “come alive” with animated content, has since bounced back under new ownership.

It is now offering a web-based AR product - WebAR – that enables developers to create AR experiences and distribute them over the web without having to resort to an app.

“One of the biggest barriers to AR adoption …has been the need to download a separate app to view the content,” said Ambarish Mitra, founder of Blippar, in a statement.

And perhaps AR won’t really fulfil its potential until we can view the content with both hands free rather than having to hold a device.

This implies AR headsets and glasses. But while the hardware is coming on in leaps and bounds – particularly for business users - we’re still a long way from creating a product that will be light enough, cool enough, powerful enough, cheap enough, or energy efficient enough to appeal to a mass audience.

Some Italian researchers even found that people who used AR headsets or glasses – think of Microsoft’s HoloLens for example - actually performed worse in precision tasks compared to those who had no hi-tech help.

But things are changing fast. Mixed reality will undoubtedly become an everyday part of our lives in ways we probably haven’t even imagined yet.