Last year we had the honor and wonder of collaborating with Raja Feather Kelly (choreographer for the 2022 Tony Award winning musical The Strange Loop ) on his first VR work. If you are in Ashland for OSF, June 29 – July 14, 2022, stop by the Black Swan Theater and check it out.
I published this as part of a peer reviewed conference on Forms of Apocalypse at University Paris 8 in 2017. It holds up and speaks to why considerations of embodiment are critical in XR. Also, warning: Heidegger ahead..
The word “apocalypse” comes from the Greek apokaluptein meaning to uncover or reveal. In keeping with the word’s origins, I approach the concept of apocalypse not as the destruction of the material world, per se, but, epistemologically, as the end of the world as we have known it. If not the primary agent of this epistemic cataclysm, computer technology, which increasingly mediates human perception of reality is increasingly reflective of it. Unsurprisingly, these technologies have also dramatically changed how we communicate about the world. In his essay, “Database as a Symbolic Form.” new media theorist Lev Manovich describes this as a move from “narrative” to “database” storytelling.
As a cultural form, the database represents the world as a list of items and it refuses to order this list. In contrast, a narrative creates a cause-and-effect trajectory of seemingly unordered items (events). Therefore, database and narrative are natural enemies. Competing for the same territory of human culture, each claims an exclusive right to make meaning out of the world. (Manovich, 1999 “Database as a Symbolic Form”)
Manovich concludes his essay by praising the Soviet film director, Dziga Vertov whose Man With a Movie Camera, he argues, successfully creates a new kind of narrative by ordering a database of images around the “kino-eye” of the filmmaker. This process of discovery is film’s main narrative and it is told through a catalog of discoveries being made. Thus, in the hands of Vertov, a database, this normally static and “objective” form, becomes dynamic and subjective. More importantly, Vertov is able to achieve something which new media designers still have to learn — how to merge database and narrative into a new form (Manovich, 1999, “Database as a Symbolic Form”).
Whereas, even today, both the novel and cinema remain mostly on the side of conventional narrative, many new media forms resist the linear arrangement of information. Of these, virtual reality (VR) most directly and completely mediates visual sense perception. In the VR headset, the world disappears, replaced by an alternative reality generated by a computer. Here, the user is no longer a passive consumer of someone else’s database. By exploring and interacting with the environment, she orients herself according to her own interests. One might even go so far as to say that she becomes a kind of film director framing close-ups and long-shots.
Philosopher Vilém Flusser has characterized the time we live in as an epistemic crisis brought about by the recognition that, “scientific research is not the gesture of a transcendent intellect.” What Flusser calls “the gesture of searching,” for meaning or knowledge, has shifted from “a digging down for reasons” to an examination of the aesthetic and relational qualities of things and an exploration of how humans respond and attend to their environments as subjects and as objects (Flusser, 2014, “The Gesture of Searching”). Perhaps unsurprisingly, Flusser’s description of this gesture serves as an apt description of being in virtual reality, a reality which has become, in essence, a database […] the researcher is embedded in an environment that interests (matters to) him, both at close range and at a distance. There are aspects of the environment that interest him intensely and others that hardly touch him. The more an aspect of the environment interests the researcher, the more ‘real’ it is for him (Flusser, 2014, “The Gesture of Searching”).
In this essay, I will build upon Manovich’s observation that, “cinema language, which was originally an interface to narrative taking place in 3- D space, is now becoming an interface to all types of computer data and media” (Manovich, 2001, 326). I will argue that whatever the language of VR turns out to be, it will rely far less on symbols and signs (content) and more on a procedural code (form) that elicits both the memory of the body’s movements and affects as well as a shared cultural memory, not of the lived world, but of cinema, itself.
What determines the authenticity or veracity of this kind of storytelling? Here, it is useful to consider one of the earliest, most widely distributed experiences in VR. The Tuscan Villa (https://youtu.be/wuFTd5TVhHw), was provided free with the developer kit for the Oculus Rift DK2 head mounted display. It offers users the chance to meander through a country villa and surrounding landscape. A generic looking computer-generated farm house, its stones visually coded as “old,” suffices to invoke a nostalgic memory—one that will be even more poignant and believable to a viewer who has never actually stayed in a Tuscan villa. In other words, the Tuscan villa is meant to exist outside of history. In this work, the visual and auditory codes are perceived as “authentic” if they successfully activate a process of false memory and effect a change in the viewer’s affective state. Significantly, the experience does not provide a traditional narrative. Rather, it offers the vague sense of remembering a dream or retrieving something that has been lost which never actually existed.
Kitsch is nothing if not a suspended memory whose elusiveness is made ever more keen by its extreme iconicity […] kitsch is not an active commodity naively infused with a wish image, but rather a failed commodity […] a virtual image, existing in the impossibility of fully being. Here, for a second or even a few minutes, there reigns an illusion of completeness, a universe devoid of past and future […]. (Olalquiaga, 1998, 28).
In The Language of New Media, Manovich points out that as cinema becomes more and more reliant on software manipulation in postproduction, it functions less as “a record of perception,” and more like a computer screen— “as a record of memory” (Manovich, 2001, 325). In the case of the Tuscan Villa, cinema no longer serves an indexical function vis a vis reality, but rather, it registers the trace of a shared cinematic memory. The kitsch experience of the villa and garden offers a momentary illusion or simulacrum of “realness”. For this, it sacrifices the passage of time and complexity of meaning.
Flusser suggests that gesture, by which he means the “reality” of affective states made manifest through aesthetics, should be judged in terms of art or kitsch rather than whether the gesture is true or false. “The scale of values we use to evaluate (gesture) may not oscillate between truth and error or between truth and lies but must move between truth (authenticity) and kitsch (Flusser, 2014, The Gesture of Searching).
It is not artifice in itself that renders the Tuscan villa demo artless. To understand this, it is instructive to compare the Oculus demo to Maria Menken’s decidedly un-kitsch 1957 film Glimpse of the Garden (https://youtu.be/_fGg7D1naIs?t=2m10s). Even though both The Tuscan Villa demo and Menken’s film privilege surface effects over depth of meaning, the result is decidedly different. In her film, movement of the camera, a unifying looped soundtrack of birdsong, and color filters take precedence over content, which consists mostly of shots of cultivated nature. Despite the amateurish quality of the images, the result is strangely magical. Menken’s editing, which juxtaposes different scales, points of view, and speeds of movement, resists the Tuscan villa illusion of completeness. Thus, it is due to the authenticity of the filmmaking gesture itself, not its content, that Menken’s film can be called art whereas the Oculus demo remains kitsch.
If there is any redeeming gesture in the Villa, it is in its ability to invoke an uncanny sense of bodily presence, a being there in a digital world that is generated by the user’s wandering or, in Flusser’s terminology, “searching” through the environment. According to neuroscientists, presence is a primordial mechanism by which organisms establish self and other, create a motor map of the universe and use memory to plan actions and interactions. In so far as the Tuscan Villa succeeds in creating this sense of presence, it sets up a queer dynamic in the user—an oscillation between interior and exterior, self and other, the “real” and the virtual. In this way it, perhaps inadvertently, succeeds in making the familiar strange. In other words, the odd and unresolvable experience of being in two places at once creates a kind of digital estrangement conceptually related to Bertolt Brecht’s Verfremdungseffekt, distancing effect, and Russian Formalist critic Viktor Shklovsky’s “making strange” (Russian: приём остранения priyom ostraneniya) whereby certain devices are used to disrupt the audience’s easy identification with an artwork.
To see how this might be productively used in VR, I turn to filmmaker Karen Cytter’s work for a contemporary example of estrangement. Cytter employs familiar, even kitsch codes of cinema, especially melodrama. But, rather than offer an illusion of completeness, she uses what Thomas LaMarre refers to as “internal montage” (LaMarre, 2009,125). LaMarre suggests that computer graphic imagery and digital effects have created a new emphasis on the animation technique of compositing —the creation of an image composed of multiple layers. If an artist suppresses the space between layers to create an illusion of wholeness, the result is either a unification in depth, a hyper- Cartesianism with a well defined vanishing point (e.g. Pixar type animations), or a flattening of all layers into a single “superflat” plane in which multiple frames of reference and sight lines coexist. Cytter’s films demonstrate a remarkable compositing of space, dramatic events, and sound. It is effectively a jumbled cinematic database in which voice, visual form, emotion, and plot nearly float free of each other on the level of meaning. Surface effects are at least as important as content here. Through a process of making the familiar strange, she astutely reveals the underlying game of seduction that takes place in cinema. In Cytter’s work we are left with empty codes, empty signs which nevertheless incite emotional responses. Devoid of linear narrative, in Rose Garden (https://vimeo.com/87553434), the mournful sound of the flute is not only a wry reference to Sergio Leone’s Once Upon A Time in the West, it codes the psychological tenor of the piece. Although there are no clear “characters,” as with avatars in video games, the human body remains the basis for emotional engagement. For instance, at the end of the film, when a boy is shot in the back, though we know nothing about him as a character, the violent collapse of a child’s body is still shocking and emotionally wrenching.
Rose Garden’s flattening out of time and space, and its move from depth to surface, which Flusser relates to the gesture of searching, resonates with Hiroki Azuma’s description of database narrativity. In his book, Otaku: Japan’s Database Animals, he notes a move from “grand narratives” to “grand non-narratives” or database (Azuma, 2009, 55). This is associated with the proliferation of small narratives the purpose of which is not to refer back to some greater or universal theme, but to invoke strong feeling or fulfill consumer preferences. Whereas, in the modernist era, personal fictions could be related in depth to a grand story about the nature or meaning of human life, now, there is a lateral move in which multiple small stories are spawned from the same database as in Cytter’s film. Likewise, in virtual reality, narrative unfolds as movements through space. However, distance is no longer measured in terms of meters or any other universal objective standard, but, as both Flusser and Azuma suggest, in terms of the user’s desires and interests interacting with the environment.
In Mark Amerika’s cell phone film Immobilité, one of the characters remarks that the world is disappearing before her eyes. Indeed, the opening scenes, a shaky, looped walk down a tree-canopied path, reveal just how the world will end— not in some fiery apocalypse, but rather by our recognition that the world was only ever virtual. In Immobilité, what remains of embodied experience, what Amerika calls “muscle memory,” finds its expression not in meaning, but in form. In effect, Immobilité — a film that remixes some of Flusser’s thought into the subtitle track — consists not so much as a series of images, but as a series of film gestures.
In Immobilité, the quality of the image is sacrificed to movement of the camera just as content is sacrificed to surface effects. Depth, in the form of philosophic musings, is separated from the images and brought to the surface as subtitle. What remains on the screen are vibrations, rhythms, and frequencies that resonate with or cancel each other out. In fact, in many parts of the film, the camera is decidedly unsteady or even appears to vibrate thereby creating a surface that destroys a Cartesian perspective and the static perceiving subject. “And then I would begin to lose myself, to play uncontrollably, becoming something like music,“ the unreliable narrator texts us from the beyond.
Heidegger’s concept of “profound boredom” operates here. This is state in which all beings “recede” into indifference and the one who is bored comes face to face with the experience of time itself (Heidegger, 2001, 80). Heidegger describes profound boredom as the ground for all the multifarious variations of being. Profound boredom leads us to suspect that reality is neither static (immobile) nor completely knowable. Concomitant with this loss of a fixed self, language fails to signify. Rather, it functions procedurally to carve up pure duration into pieces: life and death, existence and nonexistence, chaos and order. This resonates with Henri Bergson’s suggestion in Matter and Memory that homogeneous and universal perceptions of space and time are a trick of the mind, a refraction of pure duration into space. For Bergson, space and time do not exist anterior to perception and action. We do not act through space and time, rather we create these symbolic diagrams because we need to “divide the continuous, fix the becoming, and provide our activity with points to which it can be applied (Bergson, 1990. 212). Bergson suggests that our knowledge of the world is delimited by the concerns of the body and its possible actions and, perhaps, even more importantly, by our memories. If that is true, we are like Plato’s cave dwellers: always living in a virtual reality of our own making.
The key dynamic in Amerika’s film, one that will likely dominate in this time of apocalyptic narrative, is a tension between overt aesthetics and conventional notions of truth. Although Immobilité’s low production values and distinct lack of artful montage lend the film a “real-life” documentary feel, Amerika’s overt use of glitch, repetition, and jerky camera movement insists upon a kind of self-conscious performativity or aesthetics. When the narrator asks, “Was I authentic?” — the question is equally what does “authentic” mean if everything is virtual and, ultimately, data, and, furthermore, who is “I”? If the world is disappearing, so is the singular author, that figure of the auteur which haunts French New Wave cinema, to which Amerika pays homage in his use of subtitles. In Immobilité’, Amerika, like Flusser, suggests that the question of “authenticity” lies not in a “true” identity but in the aesthetic value of the gesture itself.
Antonioni’s masterwork, Zabriskie Point, also operates by disavowing conventional notions of authenticity. In Zabriskie Point all is surface: from the setting, an almost abstract stretch of ancient desert, to the way images are composed in multiple layers without depth. There is also a strange flatness to the main characters. They seem to have no ambitions and no particular future. With exquisite conciseness, Antonioni destroys the epic march of history with the first line spoken by the male protagonist: “I’m willing to die (for the revolution), but not out of boredom.” For the characters in Zabriskie Point, time operates as it does in profound boredom as a pure duration.
Even on a personal level, events have no apparent consequence. Though one of the young lovers dies violently, there seems to be little emotional repercussion for the lover that remains. This flatness of character is matched by the physical beauty of the actors. It does not matter that beneath they are voids, the viewer can still be seduced into watching. In the end, it is up to the viewer to decide if Zabriskie Point’s aesthetics, which privilege form over content, offer any meaning.
The last scene of the film, which is overtly apocalyptic, masterfully captures this dynamic. Here, the female lover imagines blowing up the ultra-modern house of her boss, a slick real estate developer who wants to produce a housing development in the desert. Antonioni’s slow motion explosion which shows mostly intact objects floating across a horizonless, flat blue surface of sky disrupts conventional Cartesian perspective as well as scale. Moreover, using slow motion and repeating the moment of the explosion from multiple points of view, Antonioni disrupts the conventional flow of time and a singular perceiving subject. However, as with the rest of the movie, nothing actually transpires. The film concludes with the woman, a virtual terrorist, driving away from the intact house without a clear destination. The strange physics of Antonioni’s explosion brings to mind the procedural physics operative in the software used to create VR experiences. These are the rules by which objects behave in a virtual 3D landscape, rules of interaction put into play by an algorithm in a game engine. As Baudrillard notes in his odd little book Seduction, games are
subversive precisely because they allow for a reversibility and fluidity of meaning that operates outside of law (Baudrillard, 1991,131-157). Because procedural physics operates outside of natural law, it offers the potential for radically new forms of narrative.
As the quote below from Donald MacKay, one of the key figures at the Macy Conferences on Cybernetics, points out: meaning, itself, acts as an “organizing function” for a perceiving subject. Significantly, MacKay defines information according to its functional impact, without respect to whether it is “true” or “false,” an idea which resonates with Flusser’s insistence that aesthetics determines the truth of a gesture.
INFORMATION can now be defined as that which does logical work on the organism’s orientation (whether correctly or not, and whether by adding to, replacing, or confirming the functional linkages of the orienting system). Thus, we leave open the question whether the information is true, false, fresh, corrective or confirmatory, and so on… The MEANING of an indicative item of information to the organism may now be defined as its selective function on the range of the organism’s possible states of orientation (Wilden, 1982, 236).
Discussing filmmaker Peter Greenaway’s move from film to installation, Lev Manovich suggests that spatialization allows Greenaway to construct pure database narrative, something which film does not allow. No longer having to conform to the linear medium of film, the elements of a database are spatialized within a museum or even the whole city. This move can be read as the desire to create a database at its most pure form: the set of elements not ordered in any way. If the elements exist in one dimension (time of a film, list on a page), they will be inevitably ordered. So the only way to create a pure database is to spatialize it, distributing the elements in space. (Manovich, “Database as a Symbolic Form”).
Following Bergson’s emphasis on the body as the starting point for perception, I would counter that this does not constitute pure database. At least in the “real world,” the narrative structure of installation arises out of the interaction of bodies with the architectural environment. As Lakoff and Johnson, and others have shown, human language also revolves around the body. However, in the virtual world, this is not necessarily so. Here, even gravity need not operate according to natural law and, furthermore, reading text can often induce nausea. Yet, even in virtual reality, meaning-making requires some form of structure.
One way that virtual reality narratives might recoup meaning without resorting to conventional cause and effect is by using procedural physics to structure narrative and to create asymmetries in the environment (database) which allow for meaning-making according to MacKay’s definition.
Artist Rachel Rossin’s virtual reality works illustrate this possibility of a new kind of narrative, one that speaks to and through virtualized, cyborg bodies. In an early work n=7/The Wake of Heat in Collapse, Rossin uses the procedural physics of gravity itself to shape narrative. Here, users progress through three contemporary Dantesque landscapes. Gravity in the landscape decreases algorithmically as the user descends. Invoking the virtual body of the user, Rossin notes:
In video games you have a certain type of hermeneutics…what you understand as the sort of “body language” or instinct that come with the world. Gravity – especially – falling off of things usually means death. We get this sad little feeling when we watch ourselves die in a video game and there is a pause that actually feels like critical distance and so I used that as a loose Virgil… the thing that pulls you through the layers of the separate worlds (Rossin, personal email correspondence December, 2015).
In a more recent work, I Came and Went as a Ghost Hand, Rossin scanned intimate photos of her domestic surroundings then manipulated these and used them to create a virtual reality landscape. As a result, the viewer sees familiar things differently, not just because of the data loss, which reduces some objects to their raw qualities – color, shape, volume – and which causes erasures, but because of the unsettling experience itself: moving through objects, speeding up, slowing down, torque. In lieu of a narrator, Rossin offers a reassuringly banal, reoccurring icon of a white gloved hand, a kitsch gesture that acts as a point of continuity for viewers as they traverse the disorienting, horizonless landscape.
In his essay, “Envisioning the Virtual,” Brian Massumi suggests that the virtual should be considered as the formative or potential dimension of reality, not as something which is in opposition to actuality. Analyzing an optical illusion where “Pac-man” circles cause a virtual triangle to appear, Massumi proposes that it is the tension between different forms of perception which produces “a field of intensity” within which the “pressure for resolution” acts as “a formative force.”
The tensions are between modes of existence proposing themselves to the experience[…] The modes do not add up to a form. They are tensely, incommensurably different. Their incommensurability exerts a differential pressure. Something has to give (Massumi, 2014, 56).
Massumi is describing a situation in which different ways of looking at the world seek to coexist in time. Cinema attempted to resolve this with montage, VR offers a different solution: the spatialization of time itself. How humans will organize that time-space in a meaningful way with respect to virtual bodies is evolving. Moreover, the political and social implications of this are as yet unknown. However, as I have shown, the memory of the real, preserved as a visual code borrowed from shared cinematic experience and the memory of the body, preserved as codified gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms might be revealed and uncovered with the aid of human-computer interfaces remains to be seen.
AZUMA, Hiroki, Otaku: Japan’s Database Animals, translated from the Japanese by Jonathan E. Abel and Shion Kono, Minneapolis (MN), University of Minnesota Press, 2009.
BAUDRILLARD, Jean, Seduction, translated from the French by Brian Singer, New York, Palgrave Macmillan,1991.
BERGSON, Henri, Matter and Memory, translated from the French by N.M. Paul and W.S. Palmer, Zone Books, Cambridge (Ma), MIT Press, 1990.
FLUSSER, Vilém, Gestures, Kindle edition, translated from the German by Nancy Ann Roth, Minneapolis (MN), University of Minnesota Press, 2014.
HEIDEGGER, Martin, The Fundamental Concepts of Metaphysics: World, Finitude, Solitude, translated from the German by William MacNeill and Nicholas Walker, Bloomington (IN), Indiana University Press, 2001.
LAKOFF, George and Mark Johnson, Metaphors We Live By, Chicago (IL), University of Chicago Press, 2008.
LAMARRE, Thomas, The Anime Machine: A Media Theory of Animation, Minneapolis (MN), University of Minnesota Press, 2009.
MANOVICH, Lev, “Database as a Symbolic Form,” http://www.mfj- online.org/journalpages/Mfj34/Manovich_database_frameset.html, accessed online 22/01/2017., The Language of New Media, Cambridge (Ma), MIT Press, 2001. MASSUMI, Brian, “Envisioning the Virtual,” Oxford Handbook of
Virtuality, Oxford, Oxford University Press, 2014.
OLALQUIAGA, Celeste, The Artificial Kingdom: On the Kitsch Experience,
Minneapolis (MN), University of Minnesota Press, 2002.
WILDEN, Anthony, System and Structure: Essays in Communication and
Exchange, New York, Tavistock Publications, 1980.
In a clinic in Mali, in the late 1980’s, Sebastian, a young gay American physician discovers that to live with his new diagnosis of AIDS, he first needs to change the story he tells about himself.
With FLY ANGEL SOUL, we utilize virtual production methods to radically alter the process of filmmaking. Here, liveness rests less in the performance of the actors (captured prior to filming with volumetric video) and more in the way the cinematographer responds to the mis-en-scene (spatial sound, captured performance, architecture, lighting, and objects) as it changes in real time. In so far as the mis-en-scene responds to the presence and actions of the cinematographer and vice versa. Filmmaking is a documentation of a call and response between the story”machine” and the cinematographer. As such, the set is critical to the filmmaking–not only to provide context for the story, but also to provide an environment for the cinematographer to inhabit and interact with. In such an environment, rules of the game stand in for “natural” laws. We are so pleased to have found Paolo Barlascini, an intuitive and intrepid artist who is acting as production designer/environmental architect.
Michel Foucault suggests that the contemporary organization of space resembles how data is organized on a computer in that it takes the form of “relations among sites” rather than a fixed local. For him, “the heterotopia is capable of juxtaposing in a single real place, several spaces, several sites that are in themselves incompatible.”
Paolo Barlascini, Artist and Production Designer working on the architecture for FLY ANGEL SOUL. VR is the ultimate heterotopia.
For FLY ANGEL SOUL we create a heterotopic building and environment. On the outside, the medical clinic references the soft edges and earth colors of Malian mud buildings. However, the interior itself is rigid. Whereas, the exterior displays the passage of time as erosion, inside time is regulated, represented by a rhythm of repeated arches.
Interestingly, Foucault suggests:
“the last trait of heterotopias is that they have a function in relation to all the space that remains….their role is to create a space that is other, another real space, as perfect, as meticulous, as well arranged as ours is messy, ill constructed, and jumbled. This latter type would be the heterotopia, not of illusion, but of compensation. “
Indeed, in FLY ANGEL SOUL, the massive scale of the waiting room suggests a kind of over-compensation that tends towards the mythic (or the pathological… )
Here, the regimentation of the interior reflects both society’s attempt to “straighten” him as well as his own attempt to regulate his frenetic emotional state (exposed through rapid fire speech and large erratic gestures in scene 1. In this space, memory also complicates and unmakes the order of the architectural space. The uniformity of the space is disrupted by videos of other spaces from Sebastian’s past life in LA. These videos offer the tantalizing possibility of escape from the space Sebastian and the cinematographer finds herself in. It is as if the film audience could actually step into another film entirely.
In the second scene, the doctor’s office is designed as a long hall lined with windows, at the end of which is a massive wooden desk. Here, the the ceiling slopes down and the width of the room narrows to something less regal and more domestic as the doctor as king becomes doctor as father figure. The windows look onto a garden, an almost impossible oasis in the desert, which suggests that underneath all this sand, there is water, life. The shadow play of flora and fauna (birds and the eponymous flies) on the floor of the clinic brings the outside in. These arabesques queer and enliven the space, complicating the linear swaths of light and shadow that divide the room. In the same way, in his conversation with the doctor, Sebastian begins to loosen his rigid conception of good and bad which he has used to damn himself with all of his life.
In the last scene, both shadow and light disrupt the linearity and regularity of the waiting room space. The architecture itself has not changed, but now the interior resembles the exterior more. Similarly, the exit which mirrors the entrance, appears worn away by the passage of time, sand has built up in the corners, a little plant grows. Although Sebastian will die, his perspective has changed. With the rigidity of God, the father subsumed by the home/womb of the mother, there is the possibility that for the time that remains, Sebastian can find a new way to live.
Here, again, is Foucault:
The space in which we live, which draws us out of ourselves, in which
the erosion of our lives, our time and our history occurs, the space that claws and
gnaws at us, is also, in itself, a heterogeneous space.
FAS tells the story of Sebastian, a young gay physician, estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart that we might “rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” FAS will be created using Unity’s Cinemachine software with three networked cameras adopting the p.o.v of the eponymous characters. What each camera “hears and sees and how each moves depends in part upon the actions of the other cameras in real time. The cameras function as the “players,” both in the video game sense and in a theatrical sense. The cameras’ real-time “performance” is the material for the 2D film. Thus, in FAS, “liveness” resides in the “embodied” cameras as much as in the actors whose performance is pre-recorded with volumetric video. Thus, the film, itself, is a poetic documentation of both human and computer machinations. Montage, as such, will not come through a post-production editing process, but occurs and becomes manifest as a result of the procedural logic of the game engine + the incommensurable logic of the human operator/performer. In keeping with Eckhart’s intent, the final film will display all the p.o.v’s on one screen, a tripartate montage of images and sounds, not created in post, but recorded “live” in real time.
Even though we work with cutting edge technology, I start projects with paper and scissors and glue. Today, Cyril, who is now used to this came over to discuss what I’d come up with. (queerskins a novel queerskins.com began with a 100 page pile of collages that I showed Cyril at lunch one day. They were made on children’s colored construction paper as I’d work late at night after my two young kids were asleep and that was all I had available at the time I started.)
It’s wild to see how your mind builds on things–the delving into architecture and windows for viewing in our In My Own Skin project last year is taken up ten levels. In this project, architectural space and light actual become as integral to the film as the actors.
We will build a set based on radial forms –see the original plan for St. Peter’s Basilica in Rome above.
In the final scene, I note that the Angel which manifests to the human as light is powerful force inspiring awe and terror.. Cyril and I spent part of our time today discussing “how does an angel see?” So, I look for this brain worm, later and find it on my shelf in Rilke’s Duino Elegies
“Who, if I cried out, would hear me among the angelic orders? And, even if one of them pressed me suddenly to his heart: I’d be consume in his stronger existence. For beauty is nothing but the beginning of terror, which we can just barely endure, and we stand in awe of it as it coolly disdains to destroy us. Every angel is terrifying. ” So, I guess this has been living in my mind all these years, waiting to emerge!
Fly Angel Soul is a short experimental narrative film shot within virtual reality which explores the potential for virtual production techniques to expand two dimensional cinematic language. FAS tells the story of Sebastian, a young gay physician, estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart that we might “rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” FAS will be created using Unity’s Cinemachine software with three networked cameras adopting the p.o.v of the eponymous characters. What each camera “hears and sees and how each moves depends in part upon the actions of the other cameras in real time. These positions are not characters, per se. They function as aspects of Sebastian’s interior milieu. Although all three p.o.v. will be equally represented in the final film, using a split screen, FAS unapologetically privileges the ineffable workings of the human heart as the driving and unprogrammable logic of the film. The human camera is the only one operated by an actual living being. The angel and fly exist as state machine, pre-programmed virtual entities.
We are not advocating an impossible return to a pre-technical state of “nature”, rather, we are asking what suffering means in our technologically embedded existence.If as Jean-Luc Godard famously stated, “The tracking shots are a matter of morality,” the use of virtual cameras in agile film production brings up pressing ethical questions which have yet to be confronted. In FAS, a “simple” and universal story of human suffering–a diagnosis of terminal illness–AIDS at the beginning of the epidemic, invites viewers to contemplate how suffering is mediated using digital technologies. Our intent is to construct the film in a way that reasserts an embodied, participatory perspective, one that acknowledges the primacy of a “human” perspective while, at the same time, offering the audience alternative, perhaps transcendent computer-mediated ways of seeing, hearing and moving through the same story.
In FAS, all three cameras function as the “players,” both in the video game sense and in a theatrical sense. The cameras’ real-time “performance” is the material for the 2D film. Thus, in FAS, “liveness” resides in the “embodied” cameras as much as in the actors whose performance is pre-recorded with volumetric video. Thus, the film, itself, is a poetic documentation of both human and computer machinations. Montage, as such, will not come through a post-production editing process, but occurs and becomes manifest as a result of the procedural logic of the game engine + the incommensurable logic of the human operator/performer. In keeping with Eckhart’s intent, the final film will display all the p.o.v’s on one screen, a tripartite ever changing montage of images and sounds, not created in post, but recorded “live” in real time.
We had the great pleasure of working with rising star Raja Feather Kelly to realize his first VR work Ordinary Gesture. For the first time, I had the opportunity to act as both creative producer and co-director for a work that was not my own, and I loved it. The team (Raja, Me, Cyril Tsiboulski working as Art and Technical Director and Lead Developer and Christoph Mateka doing Sound Design and Score composition) collaborated with an ease and respect and openness that nourished all of our creativity. Thanks to Artizen and the Oregon Shakespeare Festival who commissioned this work.
From Raja: “Ordinary Gesture is a Virtual Reality Theatrical experience that intersects theatre, meditation, and movement. The experience seeks to surrealize the experience of empathy by situating the player in 5 scenes that expand from their body to space-time (the universe) and back again. Inspired by the movies Magnolia, Melancholia, Waking Life, the poem You Are Never Ready by Nicole Blackman, and the writing of cultural anthropologist Ernest Becker, Ordinary Gesture asks the player to contemplate existence, suffering, compassion, and gesture as both ingredients to create theatre and a means to perhaps better understand empathy.”
We have plans to further develop this work and find additional venue/arts institution partners so that audiences outside of the VR festival world can experience these.
Really happy with the response to my talk today.
” Wonderful, especially drawing inspiration from Bergson, Lakoff & Johnson, Damasio, Baudrillard!” was one comment.
Also, Hilma af Klint, Sarah Ahmed, Jakob von Uexkull and Marshall McLuhan…Since I have no tech or art background, I feel free to draw on and seek inspiration from a range of sources. I really wish I could teach at the university level, but my M.D. doesn’t go far in academia and institutions are, often, unfortunately, very resistant to new ideas. The thing is, I love teaching and students really enjoy my style and mind (I taught narrative VR design at Independent Film Producers Center in NYC, the student clapped after almost every class…) So, if you are game, hit me up!
I’m going to post my slides and a draft of my paper (warning–this was not for publication, so expect spelling mistakes and grammatical errors) and eventually, I’m sure G4C will post the talk on YouTube. Thanks to all who attended. If you are free, please stop by the Expo (free, but you need to register https://www.gamesforchange.org )–I’ll be taking your questions from 11-12:30 tomorrow at the Queerskins: Ark virtual booth. And from 1:30 -2PM I will be in wild conversation with Queerskins’ actor Michael DeBartolo talking queerness, dance, VR, bodies and more. He’ll do some live readings from Sebastian’s diary, too. Join the conversation!
I’m also excited that Queerskins: ARK is garnering interest in academia! It’s really great when people think your work is important enough to actually do an in-depth analysis and publish a paper on it. Thank you!
Oculus decided that for Pride Month Queerskins is worthy of mention! We wish it were not just for Pride, but, we will take it. The good news is that it means is that YOU can see a 360˚ video version of Queerskins: ARK on your Quest on Oculus TV . Beautiful performances by Hadley Boyd, Michael DeBartolo and Christopher Vo. It is not the same as the interactive obviously, but as far as I know, the only pas de deux between two men in VR–intimate, sexy, joyful, sublime. Exquisite score by Wilbert Roget, II. Carl Nassib, this one is for you. Visibility matters. Love wins!
You could go to the Unity for Humanity panel Monday July 12th at 1:30 PM at the Games for Change Festival or you could geek out on my talk where I’ll argue that aesthetics are critical to empathy and may actually be the nature of reality. Think Bergson, queer phenomenology, Kojeve’s end of history, Heidegger’s broken hammer and mirror neurons. I’ll be expanding on points in our recent paper published in The International Journal of New Media, Technology and the Arts. Join me!!