Augmented reality and artistic experience(s)

Realité augmentée et art

As the popularity of virtual reality continues to soar, augmented reality experiences less gravitas despite providing exciting innovation in terms of technology, namely injecting reality with a virtual world. In terms of creativity, AR is not resting on its laurels. Of course prominent in the visual and digital arts, but also in publishing, architecture and street art, AR offers wonderful things. It proposes a rejuvenation of artistic experiences and is certainly no less interesting than virtual reality. Aren’t the really innovative artistic challenges the ones that often take place in these creative crannies? Let’s take a closer look.

 “Augmented reality and artistic experience(s)”: the choice of title is not down to chance. In effect, if there is one area that indeed proposes to “augment” the artistic experience, both in its creative process – the production - but also in our experience, apprehension and discovery of art, it is virtual reality and its associated technologies, mixed reality and augmented reality. Perhaps even more than its contemporary virtual reality – that has benefitted for two years from disproportionate prominence –augmented reality or AR, appears capable to us, on a daily basis, and very simply (AR only requires a smartphone or a tablet) of transforming both creativity AND artistic experiences. Through detailed examination, Digitalarti will aim to provide support for this theory, but also to offer examples of what is being done, attempting to highlight the pertinence and the originality of the proposed approaches. In short, we will present a panorama of art in augmented reality. 

Giving a virtual dimension to the real world

From its appearance in laboratories fifteen years ago to its use by artists over the last few years augmented reality has undergone many changes. There have been many commercial applications, including within the entertainment sector, for example the augmented reality edition of Esquire magazine, the latest advert for the Citroën C3 Picasso or the use of AR by the brand of ski goggles Oakley (without mentioning the Pokémon Go media and commercial phenomenon this summer). In his 2003 novel, Pattern Recognition , William Gibson described the chance to admire geolocalised virtual works of art with a headset like Google Glass, before it hit the market in 2014. Even if it was not as successful as anticipated, Google Glass nonetheless enabled the general public to envisage the possibility of  “augmented reality through art”. However, Google Glass was something of a flop. Visibly it did not deliver what the general public expected from AR, traditionally defined by the ability to integrate 3D computer graphics, in real time, into someone’s field of vision and convince that they are as real as the physical objects surrounding them. 

Vidéo concept of the project Google Glass by Google (2012)

However, it is interesting to note that current AR technology has abandoned this potential for poetical immersion of the virtual world in reality in favour of practical and pragmatic applications such as finding a local restaurant or to automatically find out the name of a monument that is already displayed on any tourist’s map.

Will augmented reality augment art?

The invention of the Screencatcher series in 2014 by the French artist Justine Emard provided a reasonable picture of the way in which artists, even those from non- technology oriented spheres, took advantage of it to create. A brief summary of the Screencatcher project: to begin with a drawing and exploration project on the theme of abandoned American drive-in theatres, Screencatcher was “augmented” by the use of AR. 

SCREENCATCHER from Justine Emard 

Indeed, after downloading the Screencatcher application for Apple or Android, spectators discovered animations on the artist’s felt-pen drawings, vibrant waterfalls, moving skies or images of films projected on screens that were empty to begin with. A device that relates the explosion of screens (from drive-ins to the multi-screen dimension that is now part of our everyday lives), Screencatcher is also a wonderful reflection on technologies and the increasingly simulated dimension of reality, proposed by a visual artist with a more conventional path.

Hyperplanes of Simultaneity | teaser from adv collective 

The artist Fabio Giampietro, also from the field of visual arts and winner of the Lumen Prize 2016 with Hyperplanes of Simultaneity, focuses on this technology too. Hyperplanes is an illustrated work that comes to life beneath your feet as if you were in a plane thanks to AR technology (even if the participants wear headsets here, the driving force for programming and the means to access Hyperplanes – to augment a pre-existing work- is well and truly the modus operandi of AR). In terms of sculpture, the possibilities are obviously huge, as explained in this article by our contemporary The Creators Project about the Josue Abraham exhibition at Galeria Merida in Mexico, a artist who proposes a series of animated sculptures via tablets and smartphone. 

The painting and drawing takes on a new dimension thanks to AR, and many academically oriented artists make the switch to augmentation. (See this visual arts exhibition using AR technology.

Augmented reality and accessing art

AR will also transform our access to art in its own way and in particular to places that exhibit and present works of art. App developers and producers have targeted museums for a while now. An example of an application that explores the possibilities of augmented reality in museums is the Augmented Reality Museum Mobile Application produced for Merchlar by Sid Lee. 

Sid Lee + Merchlar = Augmented Reality Museum Mobile Application

This type of application enables artists to leave the traditional gallery exhibition or museum format behind and to create 3D immersive experiences for the public. An example that converges with the interests of Muséomix  set up five years ago and whose missions are: “An open museum with a place for everyone, a living-lab museum that evolves with its users, a networked museum in touch with its communities.” To achieve that, and amongst the many ways of addressing access to the museum, the ‘museomixers’ have produced an augmented reality application developed in partnership with the AR and connected objects company Vidinoti.

An initiative that is bearing fruit on an international level and would have a real influence on museum and exhibition attendance according to the Huffington Post. Accessible art is also an option to make works of art appear anywhere. It is the challenge faced by the artist Ivan Toth Depeña with Lapse, proposed at Art Basel, Miami Beach in July 2016. Lapse is presented as “an augmented reality experience that takes you on a journey through Miami. By using your mobile phone and your camera, you will discover virtual art experiences hidden in the city”. 

Lapse : An Augmented Reality Based Public Art Project from Ivan Toth Depeña 

The American artist Aramique Krauthamer had the same principle with Conductar, proposing an augmented reality stroll in the town of Asheville in 2014. The immersive installation Conductar covered the whole of the town and invited visitors to create a visible and audible generative world, thanks to their movements and neurological response to the environment. The installation worked on a mobile application connected to a wave sensor generated by the brain (EEG data). Festival attendees were collectively mobilised to compose music, created thanks to electrical activity in the brain. This music and the corresponding visual world were unique for each user.


What about publishing? Augmented reality books booming

Cultural industry consortiums are the first to be interested in developments proposed by AR thanks to graphic markers. Last October the R&D division of Disney for example, announced its decision to transform colouring books. Dassault Systèmes (DS), editor of digital design software since 1981, also focused on this trend by coproducing La Douce in 2012, the first augmented reality comic strip, produced by the illustrator François Schuiten.

BD réalité augmentée  la douce

A fan of vintage pieces of machinery, Schuiten fell for an old engine called La Douce. Wishing to showcase this heritage, he asked Dassault Systèmes to make this engine come out of his next book. The Passion team for innovation, the technical patronage department at DS, worked on an augmented reality application that gave the impression of a train emerging from the comic book when the cover is placed in front of a webcam. DS has since focused on various other experiences aiming to introduce augmented reality to the general public. Paris, la ville à remonter le temps is an illustrated book about the history of Paris including different elements of augmented reality, also presented in September 2012.

Nathan and Albin Michel also began to use AR technology, particularly for their ranges of books for children (Dokéo and Histoires animées, published 4th May with two titles Chouette ! by Léna Mazilu and Copain ? by Charlotte Gastaut)  

Collection histoires animées

Independent publishing houses took the plunge, and also became immersed in AR technology. Today’s artists Claire Bardainne and Adrien Mondot use many technologies approximating virtual reality in scenic areas. They have just published a lovely book, La neige n'a pas de sens, with Subjectile, displaying many poetic and innovative examples of what AR can offer. 

La neige n'a pas de sens from Adrien M & Claire B 

Augmented reality tornado from Adrien M & Claire B 

Their Pocket Size Tornado in particular opened the eyes of the world of art and its observers. In fact, few artists have for the moment explored the creative face of augmented reality as poetically as studio AM&CB. This pair of artists expresses itself on all media, and also presents great effects that are appreciable on screens, like this 3D virtual sculpture. Keep your eye on them!

Augmented reality and experience of the world

As illustrated, art and access to culture take advantage of techniques and technologies including AR. As a consequence, experiencing art is changing, though not exclusively. Our experience of the world, of society and in a word reality, has changed as equally – if not more – as with the use of virtual reality. Artists propose for example replacing the advertising that we are confronted with daily, with virtual works of art thanks to AR. This is the case with the New York street artist Jordan Seiler, who assembled a collective with about fifty of his peers to create the NO-AD application aiming to replace aggressive advertisements, displayed in the subway in the Big Apple, with living, psychedelic and funny works.

NO AD: NYC from Heavy & Sons 

Of a similar style, see the initiative by Public Ad Campaign, an organisation that opposes outdoor advertising, and has created a mobile AR application that replaces the advertising boards in Times Square with original illustrations. The organisers of this project are hoping to expand the scope of their application to extend this practice to other advertising boards in other public spaces.

Fun and educational applications

The fact that these types of initiatives come from street artists is symptomatic. These reactive street creative types are fans of new technologies. In the style of cyberpunk, they take charge of their daily lives and their world. In this sector, graffiti is particularly pertinent. The approximation of street art to AR is interesting to contemplate presenting reality like the complex media of a socially and technologically coded environment, in which AR combines first hand experience of public space with digital media and creative practices in a hybrid composition. The blend of AR and graffiti offers a philosophical and artistic reflection, around the concept of digitally augmented graffiti.

no ad augmented reality

Key artists active in this domain: The Heavy Projects (Los Angeles); the famous Jordan Seiler from New York; the artists How & Nosm, Aiko, Retna, Ryan McGinness and Momo. Together they presented many AR street art proposals during Art Basel Miami. There is another example, this time in Geneva, where the population can admire a fun, as well as educational, work of art with Digital Neuron, decorating the walls of the new University Medical Centre in Geneva. Visible thanks to the telephone app Heavy Ar, Digital Neuron is a monumental work of art created by the Californian collective The Heavy Projects (them again) on the initiative of Happy City Lab, “a laboratory exploring collective space”. Under the guise of artistic creation, Digital Neuron highlights the importance of neurons in the human ecosystem. 

UniGe Interactive Mural: Teaser

As illustrated, the question is no longer about whether augmented reality will revolutionise artistic creation, or accessing art and artistic experience, it already does so on a daily basis. The fact is that these technologies, through their likeable and spectacular side, create fewer issues than virtual reality for example. AR technology does not involve disconnecting. Better still, it even appears to promote living together and makes our environment more liveable. Let’s wager that this technology, still increasing in popularity, has a bright future ahead.

Maxence Grugier

Photo title:  La neige n'a pas de sens, Adrien M & Claire B