‘The Viewer/Observer/User: an archaeology of Multimedia’, paper delivered at the Queensland College of Art, and Queensland University of Technology, September, 1995

The promise of multimedia is that it will profoundly change spectatorship. We will no longer be passive viewers, or distant observers, instead we will be users, or players, or the neologism ‘interactors’. The history of technologies of vision and display over the last several centuries has been seen to be one of a continual seduction, a gradual attractive force which the image has exerted on the viewer, drawing the viewer ever closer, until now we seem to be on the brink of being drawn into the image itself. Key scenes from films such as Videodrome or Poltergeist described this fatal seduction even before William Gibson’s final pioneering break through into cyberspace. The imbrication of the body of the spectator and the technology of the display has produced a multimedia/multisensory ensemble of machine and body, a perceptual cyborg. Of course the most dramatic image of this visual prosthesis is the person in the VR suit, but the ensemble is just as complete, if less anthropomorphic, in the computer based ‘multimedia interactive’. From the outside this cyborg looks fairly prosaic, nothing more than somebody siting in a chair in front of a computer with their arm outstretched to a mouse on a mouse pad. People have trouble making this ensemble look as interesting as it is, so in TV shows like the X Files computers are always used in dark rooms so they cast an exaggerated glow onto the face of the users, some of the computer screens seem to be also equipped with lenses that actually project the letters and words of the screen onto the face of the user. The promise of this immersion in the image can have both positive or negative connotations, there has been a spate of ads lately where people drive their cars or burst with their skateboards through billboards into a better life within the image, conversely the fear of cyberporn has gripped the imagination of the press, with some articles even reporting on the physical spiriting away of hapless children by strangers on the Net.

The homunculus of this new cyborg is simple too, a closed circuit system of screen/eyes/hand/mouse/cursor/screen/eyes/hand/mouse/cursor/screen etc. But within this charmed, circular apparatus, of course, there are wonders: an infinite, fluid space without boundaries because it folds in on itself; a world without gravity, or friction, or acceleration, or deceleration, a world whose only horizon seems to be the temporal lacunae when the cursor is replaced by a watch symbol and you can hear the disc spinning in the machine. A world without perspective, where distance is contained within the limits of a given resolution.

The viewer is immersed, but they also travel: they fly over terrains, follow branching pathways, move through rooms, navigate through labyrinths, explore rhizomes, etc. The viewer interacts. The new multisensory computer/user cyborg is dependent on immersion and interaction.

In this talk I want to do two things. First of all want to examine the precise nature of the ‘newness’ of multimedia. I want to historicise its newness. Secondly I want to examine the relationship, and possible tensions between immersion and interaction in multimedia.

In the 17th and 18th centuries the figure of the camera obscura was used in a very similar way to the ways in which multimedia or new technology are used today. The camera obscura was certainly a particular technology, a series of objects which were developed and refined during those centuries, the camera obscura also defined a series of specific cultural and social practices, artistic styles and ways of looking, but the camera obscura was also a philosophical model, a scientific and metaphysical metaphor for states of being and ways of knowing. The camera obscura was an assemblage, both a technical object and a discursive model. It is the fact that multimedia is also such an assemblage that makes it fascinating.

At the beginning of the 17th century Kepler used the camera obscura as a model for the functioning of the eyeball. The lens was like the pupil, and the retina was the piece of white paper or ground glass screen. He couldn’t work out, however, how the two images made their way from the retinas into the mind to produce a single image of the world. Later Descartes developed Kepler’s model of vision into a model of visual cognition by introducing intellectual understanding as a crucial complement to the cold image projection of the iris, the perceiver read the mini-movies that were being continually projected inside the skull in an equivalent way to the way they read the other signs of the world which were brought by the other senses, or which could be induced by thought. Descartes therefore elevated the camera obscura to a model of understanding itself. But in the camera obscura the viewer’s body is bracketed out, because the viewer is inside the machine the machine can take no regard of the viewer, all that matters is the punctal lens and the objective image which the viewer perceives with sober, detached scrutiny. As Descartes said “perception, or the action by which we perceive, is not a vision … but is solely an inspection by the mind.”(Crary 43)

Jonathan Crary, in his book Techniques of the Observer, claims that there was a monumental shift to this paradigm in the early 19th century. The body, which was bracketed out of Cartesian perception, became the very site of perception for 19th century scientists. The eye, which had been a cold dead optical instrument in the 16th and 17th centuries suddenly flowered into a febrile, quivering organ in the 19th century. Scientists and philosophers like Goethe, Schopenhauer introduced a temporal dimension into perception with their investigation of the phenomena of the persistence of vision (on which the apparatus of the cinema depends), others desperately tried to draw and catalogue the various varieties of afterimage before the faded, one blinded himself staring at the sun, other scientists furiously rubbed their eyes to create impressions of light and colour with no optical stimulus, others discovered the blind spot, other scientists discovered that any given nerve will give the same sensation no matter what the stimulus—optical, electrical, chemical or physical

Perception is not the sober inspection of some inner cinema, instead inside and outside mix so intimately that the cannot be distinguished. Perception is incarnated.

A whole range of apparatuses were developed as simultaneous amusements, tools of scientific investigation and epistemological models. Examples like the Thaumatrope, phenakisttiscope, zootrope and kaleidoscope are well known. I want to concentrate on the stereoscope and the phantasmogaoria.

There is no stereoscopic image, it is never stably projected anywhere for sober contemplation, instead it is formed by an exertion of the mind, nor is the stereoscope a replication of natural vision, it is an artificial simulacra of vision. It gives the temporary illusion of b binocular vision, not binocular vision itself. The virtual stereoscopic image is like a stage set, the planes appear recede like flat disjointed cardboard cut outs with a vacuum between. The stereoscope also announced for the first time the prosthetic interlocking of body and machine, though artificial, it is a thoroughly corporealised vision. But it also had the effect of taking vision out of the body, which is left behind, and producing a kind of phantom, body which travels through a virtual space.

Oliver Wendell Holmes writing in 1859 described this new form delirious navigation through virtual reality: “The first effect of looking at a good photograph through the stereoscope is a surprise such as no painting ever produced. The mind feels its way into the very depth of the picture. … Oh infinite volumes of poems that I treasure in this small library of glass and pasteboard! I creep over the vast feature of Rameses, on the face of his rockhewn Nubian temple; I scale the huge mountain-crystal that calls itself the pyramid of Cheops. I pace the length of the three titanic stones of the wall of Baalbec — mightiest masses of quarried rock that man has lifted into the air; and then I dive into some mass of foliage with my microscope, and trace the veinings of a leaf so delicately wrought in the painting not made with hands, that I can almost see its down and the green aphis that sucks its juices. I look into the eyes of the caged tiger, and on the scaly train of the crocodile, stretched on the sands of the river that has mirrored a hundred dynasties, I stroll through Rhenish vineyards, I sit under Roman arches, I walk the streets of once buried cities, I look into the chasms of Alpine glaciers, and on the rush of wasteful cataracts. I pass, in a moment, from the banks of the Charles to the ford of the Jordan, and leave my outward frame in the armchair at my table, while in spirit I am looking down upon Jerusalem from the Mount of Olives” (The Stereoscope and the Stereograph)

Stereoscopy, not surprisingly, quickly became associated with pornography in a similar way in which the Net and VR have instantaneously suggested porn to today’s press. Baudelaire railed against it in his famous diatribe against photography of the same year: “The love of pornography, which is no less deep rooted in the natural heart of man than the love of himself, was not to let slip so fine an opportunity of self-satisfaction. And do not imagine that it was only children on their way back from school who took pleasure in these follies; the world was infatuated with them. I was once present when some friends were discretely concealing some such pictures from a beautiful woman, a woman of high society, not of mine—they were taking upon themselves some feeling of delicacy in her presence; but ‘no’ she replied. ‘Give them to me! Nothing is too strong for me.’ I swear that I heard that but who will believe me?”(Salon of 1859)

The stereoscope was a kind of corporeal/visual labour on the part of the user, who is no longer a passive cartesian viewer, but an active observer, quite aware of the nature of the apparatus, which was transparent to them, and willingly submitting themselves to the surprise of the illusion, the production, within their corporeal selves, of a artificial shock, as Baudelaire’s scandalous lady say ‘nothing is too strong for me!”

The stereoscope was primarily a domestic technology, for armchair travellers or masturbators. Other technologies produced corporeal illusions in a public space, to observers who were logged into the cybernetic visual system, not through stereoscopes attached to their faces, but through having their bodies immobilised in auditorium chairs. These were dioramas, wax works, phantasmogaorias,  magic shows and early pre-narrative cinema.

It is one of the founding myths of cinema that the first audience for the Lumieré’s Arrival of a Train at a Station in 1896 caused panic amongst its audience because they thought they were about to be run over by an actual train. However recent film historians such as Tom Gunning have disputed this seductive myth. For one thing, audience in Europe were used to being astonished by mechanical and optical apparatuses. They were not a naive audience who assumed the apparatus to be transparent, Gunning claims that the astonishment of early audience was at the apparatus itself, and not at what it purported to represent. The apparatus tested the limits of an intellectual disavowal: I know, but yet I see, and that was its pleasure. This pleasure is an aesthetics of astonishment, where the viewer does not get lost in the fictional world of the film’s dram, but remains aware of the act of looking. Films with great titles like The Railroad Smash-Up, Photographing a Female Crook, Demolishing a Wall, and Electrocuting an Elephant, were exactly what the said they were. Safely ensconced in their seats the audience experience the pleasure of a discontinuous series of bodily shocks, which they know is being brought to them by the technology of cinema—a cinema, which in its structural logic is similar to the machinery of the factory, or the experience of a tram-ride through city streets. Critics in the 1920s and 30s, such as Benjamin, Kracauer and Jünger saw these shocks as a kind of training of the body, a hardening of it, to be able to accept the increasing jolts of modernity

This cinema of attractions persists throughout the history of cinema, even when another kind of psychological absorption into the diegetic narrative of the cinema is developed by DW Griffith and Hollywood through the combination of 19th century narrative modes and shot/reverse shot psychological identification with characters. Again, pornography is the most obvious examples of the sub-subterranean persistence of the cinema of attractions, but even Hollywood cinema retains elements of it. Recent mainstream films contain elements of the Ride Film, where the viewer’s body is subject to a series of vertiginous affects, optically and directly, not through psychological identification. Often, as in batman, these films are associated with real theme park rides, and the technologies of actual transport of the audience through real space, along a roller-coaster, and their virtual transport, via projected imagery, have been merging. Again this has precedents in the dioramas and panoramas of the 19th century where audience seating were mechanically moved from scene to scene.

My reason for elaborating all of these precedents to multimedia immersion is not simply to point out that it is all not such a new thing after all, and definitely not to celebrate multimedia as the inevitable culmination of centuries of striving for a ‘better’ vision. Rather I want to shift attention from the newness of the particular technologies—computers and their programs—to the ensemble of viewer/machine. I want to look at what is happening in the two feet or so of space between the user and the screen, and for that a kind of archaeology of the interface may be useful.

Within today’s interactive user of a computer interface there sits another historical figure, the corporealised observer of astonishing optical phenomena which are produced, or perhaps incarnated, within the observer themself, and within that figure lies another figure, the decorporealised cartesian viewer, soberly understanding the language of the outside world. I think we need to call upon all three—viewer, observer and user—to understand what is going on.

One of the most popular films of the ‘cinema of attractions’ was the phantom train ride where, instead of the train threatening to come out of the screen at the audience, a camera was mounted on the front of the engine and the audience was endlessly plunged through space along the camera’s central perspectival axis. The audience seemed to be carried forward not by the train, which was invisible, but by the thrust of vision itself.

Today, of course, the connection between lines of sight and ballistic trajectories has been firmly established by smart bombs. And in today’s VR environments or multimedia interfaces, it is not the train which is a phantom, but the user themself has a phantom double, which is recognised by the machine. The key rupture between old and new immersive environments is that in new immersive environments, the apparatus recognise us, as much as we recognise it. This can extend all the way from the computer responding with a new window when we point and click at a button with our simple cursor arrow on the screen, which is our punctal double in the program’s interface, to the shoot-em-up VR games where if your opponent sees you before you see them…they gleefully shoot you. Between are all the possibilities of interactivity, and all the possible architectures of the virtual environment: the hierarchical, the labyrinthine, the rhizomatic.

However the two qualities of today’s multimedia seem to have an inverse relationship to each other. Maximum effects of immersion rely on minimal interactive choices, more complex choices and hypermedia pathways rely on dense and complex interfaces which demand more of a sober Cartesian intellection than a spasmodic corporeal reaction. For instance most flight simulators, although being highly immersive, only allow the viewer to gradually inflect the trajectory, the paths are quite tightly constrained, and the machine needs time to process new data. In any case, how can more complex, meditative, cartesian decisions be made in a complex labyrinthine interface when the body is hurtling through virtual space. When there is no split between mind and body, when they are vertiginously thrust together by the G forces of immersion, can cartesian ‘choice’ be exercised in the same way? In real situations the body’s reactions are a matter of the somatic memory of relentless training and habitual corporeal drilling.

Most information rich multimedia actively work against the natural immersive seductions of the medium, interfaces which are discussed in terms of being user friendly, or ‘transparent’ seem to me to be on the contrary to act as speed humps. The provide a series of already recognised objects signifying sober selection, books on the shelf, doors along a hallway, pictures on the wall, on which the user must click. Acting like a virtual sphinx these interfaces say, pause before you choose.

There may be a very fundamental metaphoric flaw here. All the hypertext/hypermedia hype rotates around the metaphoric figure of exploration, the breath taking discovery of a new fact. The sudden, unprecedented surprise. The instantaneous enlightenment of dark continents. The Eureka! But all our post-colonial writers have told us that exploration was never like that, frontiers never moved across the face of the globe with the inexorable certainity of the dawn. Exploration was always preceded by speculation, myth, hypothesis, the new was always read through established models of what otherness should be. And of course, on the other side, the side of darkness there is always resistance, evasion, dissimilation and mimicry. Quite simply exploration is never simply spatial or territorial, it is also textual, incremental, layered, accumulative and a process of exchange, however out of balance.

I think that the tension between immersion and interactivity must be a very delicate one to balance, because, from my position solely as a user, multimedia producers seem to very often get it wrong. Astonishment fails, information exchange does not take place between computer and user, the user skims over the interface, checking to see that something, it doesn’t seem to matter what, happens as buttons are clicked. They interact with the interface, rather than the program. The multimedia exhausts itself when its interface, rather than the program itself, has ‘played itself out’. Boredom and impatience are spectres that haunt every multimedia developer.

Is this why the most psychologically engaging immersive interactivity practices are text based, not image based. MUDs and MOOs have a low data bandwidth, but a high informational bandwidth, courtesy of the English language. To my knowledge the only reported psychological trauma from a rape in cyberspace was textual and occurred in LAMDAMOO (?).