Introduction
This contribution aims at outlining a theoretical framework for the interpretation of some recent trends in museum exhibition design, based on the technological paradigm of Natural Human Computer Interaction. The emergence of gesture-based interfaces and motion tracking technologies in exhibition practice is challenging the museum’s traditional ocularcentricity, which seems to have witnessed the same “bodily turn” that other disciplines have experienced since 1980s. Some digitally augmented museums have moved from being places for mere visual contemplation to being places for dynamic action and production of content and meaning, where the visitor is a moving body, physically interacting with space and objects within responsive environments.
Famous examples in Italy can be identified in projects realized by Studio Azzurro since 1999 and ennezerotre since 2003, as well as in experiences such as the Virtual Archaeological Museum of Ercolano (2011) and the City Museum of Bologna (2012). Outside of Italy, integrated and innate approaches to the use of ICT (Parry and Sawyer 2005) can be identified in the Gallery of Modern London at Museum of London, Cocoon Gallery at Natural History Museum and in the Churchill Museum.
In these scenarios a considerable weight of the exhibition discourse is supported by digital content, which visitors are invited to interact with through new interfaces inspired by the technological paradigm of Natural Human Computer Interaction. Though the term has come to refer to a broad range of meanings and applications, in a general perspective natural interaction represents an approach for designing interfaces that make it possible to interact with the information through dynamics and processes perceived as natural for the user, not necessarily mimicking and simulating real actions. The most recent developments focus more specifically on the bodily experience, as research themes show in their titles, where terms like body driven interaction, gesture based interfaces, kinetic interfaces, whole body interaction are often found.
In museum settings, this kind of interfaces make it possible to combine mental, physical and social interactivity with a strong feeling of immersion, favored by the impression of disappearance of the technological device. The adoption of this design approach to create interactive multimedia exhibits makes the visitor use movement, rather than a traditional desktop system to control a process.
According to Horizon Report 2012 Museum Edition, Natural Interfaces are among those technologies that will be widely adopted by museums over the next five years. “New kinds of sensing technologies — vision, ambient sensors, tangible object based interfaces, etc. — which allow for natural interactions might result in a museum where any surface is the potential home for a virtual interaction. At this time there is probably more utility for NUI’s in science centers, natural history museums, and children’s museums — museums that are essentially experiential and idea-driven rather than object-based”.
According to scholars and practitioners, NHCI enhances the physical environment’s material qualities (Ciolfi and Bannon 2007) and transforms visitors into active orchestrators of their experience (Sparacino 2004). It does not require prior knowledge and leads to surprise effects and intuitive interaction (Hernandez 2009). Furthermore it conveys the same kind of content with aesthetics and presentation modes which suits better to collective use (Kidd, Ntalla and Lyons 2011), makes the final configuration become the result of progressive or simultaneous actions performed by visitors and allows to see more clearly, not the “technology”, but its effects (Rosa 2004).
Everyone is a performer in the age of Natural Interaction
From empirical research, the user’s body may be addressed in different ways in the interactive process, in terms of parts involved by the movement, degrees of freedom and complexity and richness of meaning associated with gestures. The ability of the system to detect continuous or discrete movements can also determine the opportunity to develop expressive movements.
At a general level the body can act as a tool both for implicit interaction, for example, when a sensor detects the appearance of the user, and for explicit interaction, when sensors or cameras track the precise motion of the user to select elements or perform actions. Specifically, the action may involve mere presence, displacement of the body in space, (actual or simulated) touch, air gesture, manipulation of objects (full hand gesture) and multimodal interaction.
In “Mura e Società” multivision display in the exhibition “Il Cinquecento Interattivo” in Bergamo or in the sound installation “The Sound of Art” at ArOs Museum in Aahrus, the visitor’s body detected by sensors or cameras activates immersive projections and audio commentaries.
The body can also be tracked in its displacement through the space, becoming a mouse which can be continuously followed by the system. In sensitive walls the silhouette can be detected by height (“Sand Interactive” at the Sony Wonder Technology Lab in New York, “What else is changing?” and “Me3” at the Science Museum in London). In sensitive floors the user’s shape is observed from the top (“A sparkling sea” in the exhibition “Creatures of Light” at the American Museum of Natural History in New York or “Step into the Painting” in the project Louvre – DNP Museum Lab).
Simulated or actual touch is probably the most common application of natural interaction in museum exhibits, obtained through camera vision systems or sensing surfaces. Well known examples are the interactive tables “Mitigation Simulator” at the Koshland Museum in Washington, “Lifeline” at the Churchill Museum in London, “revolution” at the Senckenberg Natural History Museum in Frankfurt, “Grid” at the Tate Modern in London, the circular pad in the “Settlement Exhibition” at Reykjavik City Museum and the sensitive wall “Climate Change” at the Science Museum in London.
With air gesture (or empty hand gesture) the visitor performs hands-free arm movements in a fixed position in space, generally inside his kinaesphere. Gestures can be static, therefore involving a particular configuration of the hand or of the upper limbs (“Point At” at Palazzo Medici Riccardi in Florence, the kinetic interface at the Swansea Museum) or dynamic, i.e. implying a movement which varies in time (“Xray” in “Enaging Constable” at the Tate Modern in London, “Age Related Movement Changes” at the Koshland Museum in Washington, “Energy Everywhere” and “Climate System” at the Science Museum in London).
With the movement and manipulation of objects (full hand gesture), “concrete” elements can be moved on a surface (the paper tablets in “Multimodal Diverse Travel” at the Archaeological Museum in Thessaloniki, the audioblocks in “Music Mixer” at the Sony Wonder Technology Lab in New York, the chess sets in “The Big Three” at the Churchill Museum in London) or in the air (“Virtual Touch” wand at the Hunt Museum in Limerick, the torches in “Genova del Saper Fare. Lavoro, imprese, tecnologie” in Genoa, the magnifying glass “Polyapton” at the Archaeological Museum in Thessaloniki). They can also be inserted into other structures (the cubes in the interactive table in “Cinquecento Interattivo” in Bergamo), or manipulated while they remain anchored at a point (the screen in “What Causes Infection?” at the Koshland Museum in Washington, the telescopes at the Churchill Museum in London, the rudder in “Sail traffic Instructions” at the German Technology Museum in Berlin, the wheel of “Energy in Everyday Life” at the Science Museum in London).
Finally, within multimodal interaction, forms of interaction can be considered that provide different input from the previous ones, such as voice, breath or gaze. Breath is used for example in “Panoptes” at the Archaeological Museum in Thessaloniki, where a windmill is connected to a slideshow, and in a more sophisticated way in “Talmud book” at the Jewish Museum in Berlin, where the typefaces are projected onto a white surface when wind sensitive sensors detect the user’s breath.
When playing with kinetic interfaces in public spaces the user performs movements which can become at the same time both functional and expressive. Indeed, as researchers point out, “participants are aware of the others awareness and explorations take on a certain aspect of performance” (William, Kabisch and Dourish 2005). Users’ actions are rarely “autistic”, because museums are shared spaces where an audience is somehow always present. It is no coincidence that the same technologies used in museum exhibitions are often adopted in digital performance projects. In these shows the body and its movements represent both aesthetic objects to be watched and controllers of sophisticated audio-visual effects that contribute to the artistic goal (Sparacino, Davenport and Pentland 2000).
As Vom Lehn suggests, the distinction between user and observer, between spectator and performer is not so immediate in context-aware settings: “moving through space, people often become members of the exhibits without knowing it – they can be transformed from unaware users to aware performer” (Vom Lehn et al. 2007, p. 8).
Furthermore, the performative and expressive aspect of interaction can refer both to the process and the product, namely the effects of the interaction.
Extending some reflections developed in the framework of interactive art installations to museum displays, it can be argued that “the content of installations can consist of the users’ very own activity, both in creating novel content and in providing a performance that is watched by others and simultaneously provides the core experience for the active person” (Hornecker and Stifter 2006). Movements are therefore performative both while they are being executed and observed and also subsequently, because they often leave traces. Indeed, in responsive media spaces gestures transform reality, affect the appearance of the environment and produce something new. In this way the meaning of performativity proposed by Austin (1962) becomes particularly relevant. According to the English philosopher, the term performativity is strictly related to the idea of shaping reality and producing something new. Indeed, performative utterances are defined as sentences which not only passively describe a given reality, but actually change the reality they are describing.
As artistic group Studio Azzurro writes
“The involvement of the viewer, who becomes co-author, must be complete, engaging his corporeality and sensory dimension. Thus, we often talk about a progressive closeness of interactive and performance art […]. In order to establish a creative dialogue between man and machine, which can also involve interpersonal relationships, it is necessary that the action gains priority over the procedure, that the act giving rise to the interaction is not just functional and the result of mechanical and predictable behavior, but that it mantains its expressive properties” (Balzola and Rosa 2011, p. 99).
To sum up, in this approach to exhibition design, not only is the relationship between visitors’ movements and digital content implicitly assumed, but also the role of the body is clearly formulated as in a global work of relational art.
Have museums always been “performative”?
Adopting the point of view of those who consider multimediality, interactivity and immersion as intrinsic to museums, regardless of the nature of the devices through which the discourse is organized, these can always be interpreted as performative, because they are conceived as places continuosly traversed by moving bodies. The spectators’ movement through space distinguishes museums and cultural sites from other contexts: here we are turned into “promenading spectators, absorbing images, sounds and texts as our feet shuffle through sometimes crowded, sometimes deserted galleries” (Griffiths 2008, p. 216). Spatialized stories would also require the enactment of a “performance” by visitors (Kirshenblatt-Gimblett 1997).
However, other perspectives on museum studies consider performativity as more strictly related to specific design strategies, both in museological and museographic terms.
From a museological point of view, performativity is commonly associated with discourses of representation, ideology and politics of display. For example, citing linguistics, Hoebink discriminates between constative, object-centered exhibitions, where the objects’ authenticity is crucial, and performative exhibitions, where experience and narration are central. Performative presentations “are not aimed at truths, but at experiences. It is not a question of one true story, but of various perspectives, stories, types of expertise. It is not the objects that are focused on, but the story told with the aid of objects – stage properties. A constative exhibition is characterized by material authenticity, a performative one by a narrative authenticity” (Hoebink 2011).
Similarly, Kirshenblatt-Gimblett (2006) describes contemporary museum practice as performing museology: to appeal to its experience-oriented audience, the museum privileges the processes of display over the particularity of objects to convey information. As Casey (2003) writes, while commercially-oriented presentation techniques can be detected across many museum types, the contemporary cultural history museum has embraced performance strategies more comprehensively than other museum types.
On the other hand, from a museographic perspective, a performative exhibition is described as a sequence of moving images, placed in strong relationship with the path followed by the visitor. Referring to the performative turn, Eckhard Siepmann recognizes a first attempt to create a performative exhibition in Walter Benjamin exposition in 1990 in the Werkbundarchiv Museum der Dinge in Berlin’s Martin Gropius Bau. While not yet using digital technologies, the exhibition aimed at a precise perception of space by the body and was structured so that, thanks to optical effects, the set seemed in motion while the visitor was walking through it. The audience in motion became part not only of the physical but also of the semantic environment, because it was asked to act to make something happen.
Reflections by Studio Azzurro about their exhibition design projects provide a similar understanding of the relationship between proposed narrative and requested performativity. “The hypertext structure of the narrative and documentary fragments evoke the story in a non-linear fashion: fragments are mobile units in an open structure to be recombined also according to your own background. The exhibition space is thus experienced in a narrative way: you move within a film sequence that progresses with the steps of the visitor” (Cirifino, Giardina Papa and Rosa 2011). More specifically the space configuration contains human performativity in a relational aesthetic which shifts the focus from objects to gestures and from things to behaviours (Balzola and Rosa 2011).
The concept of performativity has also been explored from an architectonic perspective: it represents one of the most significant trends in contemporary exhibition design as a new approach to spatial organization and interactivity, which emphasizes the encounter between space, body movements and time – one can identify similarities with the immersive or “inclusive” approach described by Davallon (1992). Performative space is a development of the concept of experience design, which recognizes that the body plays a fundamental role in learning.
To describe some interactive museum exhibits, Dernie (2006) employs precisely the term “performative space”, a place where through physical exploration and gestures a fruitful exchange is activated between objects, contents and visitors.
“The visitor is a body in movement and the contemporary exhibition design can examine patterns of movement to explore communication and exhibition experience. A performative space is one where the movement of the body is considered integral to the structuring of the environment and the landscapes of the artworks, objects or performance. It is a place where the traditional boundaries between observer and object or performance are eroded. […] Visitors become quasi-performer themselves, in a sense, spectators and part of the spectacle, moving through a topography of overlaying sounds and images in an architecture which is constructed by relationships between the moving bodies in the space” (Dernie 2006, p. 14).
There is no reference to a specific technology, rather to a wide range of analogical and digital tools. “Despite the long-standing inclusion of interactive devices in exhibition spaces, such creative shaping of performative spaces is a relatively new tendency. The approach may not fall within the traditional boundaries of exhibition design as established by modern professional criteria, but the variety of experience, movement and activity that invites will be fundamental to creative and inclusive exhibition experiences of the future” (ivi, p. 49).
Devising a new performative museum
“Performance” has become an extremely popular and contested concept in recent years in a wide range of disciplinary fields (Carlson 2004). Considering this, this contribution focuses on some meanings of the terms “performance” and “performativity” developed in the context of theater studies and anthropology, rather than interpreting performativity as an approach to support specific ideological purposes or museographic articulations of space and displays. Performativity is also connected to a peculiar technological paradigm, which emphasizes the corporeal sphere, and to the power of the visitor to choreograph the space temporarily or permanently, giving the museum a character of liveness and contingency.
As Turner suggests, the term performance comes from the old French parfournir, which literally means ‘to provide completely or fully’. To perform means then to produce and to accomplish something, to execute a drama, an order or a project. Rules can ‘frame it’ but the ‘flow’ of action and interaction within this framework can lead to unprecedented insights and also generate new symbols and meanings. (Turner 1982, p. 145)
In art, performance can be defined as an act of behavior that has a kind of spectacular, an action that depends on specific space and time, therefore a creative act with unpredictable consequences and a specific focus on the use of the body as privileged means of expression (Mazali 2007). Performance implies a process of collective construction that involves both the body of the actors and that of the audience.
From a theatrical and anthropological perspective, performing activities are basically processual: there will always be a part of them which is subject to transformation, and subsequently absolutely indefinable. Here verbal language is no longer a priority, instead, there is a polycentricity of communication and the space is strictly relational: “you do not know what it is until you use it” (Schechner 1988, p. 167). In its emphasis on practice, rather than product, and in its refusal of imitation, performance as livineg art has also been set in contrast to representational theater (Kaprow 1993).
From these reflections the performance’s main features emerge: it is unique, it happens in a specific time, it is closely linked to the place and the people involved. In the performance the object is a practice, a process.
Natural interaction shares some similarities with the concept of performance, especially because of the focus on the process and on the body experience and expression and the power attributed to the performer’s gestures, which can transform the real or imaginary space.
Performative museums can therefore be described as cultural institutions that communicate through exhibitions where the discourse is formulated mainly through movement-based technologies and emphasis is placed on the accomplishment of performative acts, aimed at specific cognitive, emotional and communicative goals.
In this type of museums, the role of embodiment is not simply theorized and used as a reference in order to develop the communication project and the relationship between body and displays is not implicitly assumed, but rather actions can significantly transform the appearance and the content of the exposition, since presence and interaction are intercepted and processed.
Parallels can be traced, with relational art, whose goal is to create devices that foster the users’ creativity and social interaction, with the idea that public participation should be a necessary condition for their fulfillment. They are “open works” (Eco 1962) not only in perceptive but also in concrete terms. When the intentions of relational art meet the digital world, new opportunities emerge to experiment with different perceptual experiences and relationships with other people.
As in performance, the museum visit takes on a character of contingency and processuality because visitors are given the elements and structure (the score) to shape their learning and emotional experience, acting in factual and concrete ways and not only conceptually: systems act as the detonator for experiential processes (Rosa 2004).
As Siepmann reminds us, in performative – analogical or digital – exhibitions the focus on the process is associated with a particular attention to sensory experiences. The changing role of the user in these displays can be described in terms of metamorphosis “from observer to participant. […] The processuality of the exhibition corresponds to the perception of it with the whole body. The focus of a pair of eyes on an object is replaced by contact with all the body senses moving within an environment that affects them and is addressed to them” (Siepmann 2003, p. 6). The value attributed to the sensory dimension represents another point of contact between the world of the performance and the dynamics involved in the use of natural and tangible interfaces in the galleries.
A performative museum is thus more specific than an interactive museum: since interaction is intercepted, interactivity is strictly digital, and, since it is based on NHCI, it is also physical.
In the same way that a “narrative museum” (Cirifino, Giardina Papa e Rosa 2011) – as multimedia interactive exhibitions are nowadays often called in Italy – is not necessarily interactive, nor performative, a performative museum may not be “narrative”, since it may not articulate the space like a linear journey through time or it may not present mainly master narratives or short stories.
Compared to the adjectives “interactive” and “narrative”, the word “performative” simply seems better suited to frame the features of new multimedia installations developed with intuitive interaction and the “disappearance” of the technological device in mind. Furthermore, it especially emphasizes the centrality of human activity. From the performative perspective, by adopting a user-centered approach, the focus is placed on the user rather than on the communication project designed by the curator, on the exhibits themselves or the arrangement of the displays, which seem more in keeping with the temr “narrative”.
Summing up, the subject of the verb “to perform” is not so much the museum itself, as is suggested in some interpretations from museum studies (Casey 2003, Hoebink 2011), but is the visitor: the museum offers the conditions (a structure, a plot) for the visitor to be an active participant, proposing exhibitions which provide the opportunity for perfromativity.
References
Austin, J. L. (1962), How to do Things with Words, Cambridge, Harvard University Press
Balzola, A., & Rosa, P. (2011), L’arte fuori di sé. Un manifesto per l’età post-tecnologica, Milano, Feltrinelli
Carlson, M. (2004), Performance: A Critical Introduction, London and New York, Routledge
Casey, V. (2003), “The Museum Effect”, online at http://www.valcasey.com/thesis/thesis_effect.html
Ciolfi, L., & Bannon, L. (2007), “Designing hybrid places: merging interaction design, ubiquitous technologies and geographies of the museum space”, in CoDesign, vol. 3, n. 3, September 2007, pp. 159–180
Cirifino, F., Giardina Papa, E., & Rosa, P. (eds.) (2011), Musei di narrazione. Percorsi interattivi e affreschi multimediali, Milano, Silvana Editoriale
Davallon, J. (1992), “Le musée est-il vraiment un média?”, Musées, n. 2, Lyon
Eco, U. (1962), Opera aperta. Forma e indeterminazione nelle poetiche contemporanee, Milano, Bompiani
Griffiths, A. (2008), Shivers down Your Spine. Cinema, Museum and the Immersive View, New York, Columbia University Press
Hernandez Ibanez, L., Lopez Mihura, R., & Barneche Naya, V. (2009), “Nuevas tecnologías en los museum: de la contemplación a la experiencia”, in Rico, J. C. (ed.), ¿Cómo se cuelga un cuadro virtual? Las exposiciones en la era digital, Gijón, Ediciones Trea
Hoebink, D. (2011), “Closereading van ‘Een monument vol verhalen”, Bulletin KNOB 110 5, online at http://bulletin.knob.nl/article/view/81
Hornecker, E., & Stifter, M. (2006), “Learning from Interactive Museum Installations About Interaction Design for Public Settings”, in Proc. of OzCHI 2006, ACM, pp. 135-142
Kaprow, A. (1993), Essays on the Blurring of Art and Life, Berkeley, University of California Press
Kidd, J., Ntalla, I., & Lyons, W. (2011), “Multi-touch interfaces in museum spaces: reporting preliminary findings on the nature of interaction”, in Ciolfi, L., Scott, K., Barbieri, S. (eds.), Re-thinking Technology in Museums: Emerging Experiences, University of Limerick
Kirshenblatt-Gimblett, B. (1997), Destination culture: tourism, museums and heritage, Berkeley, University of California Press
Kirshenblatt-Gimblett, B. (2006), “Exhibitionary complexes”, in Karp, I. Kratz, C.A., Szwaja, L., and Ybarra-Frausto, T. (eds.), Museum frictions: Public cultures/global transformations, Durham & London, Duke University Press, pp. 35-45
Johnson, L., Adams, S., & Witchey, H. (2012), The NMC Horizon Report: 2012 Museum Edition, Austin, Texas, The New Media Consortium
Parry, R., & Sawyer, A. (2005), “Space and the machine: adaptive museums, pervasive technology and the new gallery environment”, in MacLeod, S. (ed.), Reshaping museum space: architecture, design, exhibitions, New York- London, Routledge
Rosa, P. (2004), “Rapporto confidenziale su un’esperienza interattiva”, in Vassallo, S., & Di Brino, A. (eds.), Arte tra azione e contemplazione. L’interattività nelle ricerche artistiche, Pisa, ETS
Sparacino, F., Davenport, G., & Pentland, A. (2000), ”Media in performance: Interactive spaces for dance, theater, circus, and museum exhibits”, in IBM Systems Journal, vol. 39, n.3-4, pp. 479-510
Sparacino, F. (2004), “Scenographies of the past and museums of the future: from the wunderkammer to body-driven interactive narrative spaces”, in Proc. Of ACM Multimedia 2004, New York, 10-16 October 2004
Schechner, R. (1988), Performance Theory, London-New York, Routledge
Siepmann, E. (2003), “Ein Raumverhaltnis, das sich durch Bewegung herstellt. Die performative Wende erreicht das Museum”, online at http://www.kunsttexte.de/zusamm.php?artaus=1
Turner, V. (1982), From ritual to theater: The human seriousness of play, New York, PAJ Publications
Vom Lehn, D., Hindmarsh, J., Luff, P., & Heath, C. (2007), “Engaging Constable: Revealing art with new technology”, in Proceedings of CHI 2007, ACM, pp. 1485-94
Williams, A., Kabisch, E., Dourish, P. (2005), “From Interaction to Participation: Configuring Space through Embodied Interaction”, in Proceedings of the Ubicomp 2005, Tokyo, pp. 287-304
