Tracking shots are a question of morality.” This was Jean Luc Godard’s response to a shot in the film Kapo, in which the viewer is brought up close to the body of a Holocaust victim who has just killed herself on an electric fence. He is pointing out that although technology makes this shot possible, it is a perspective that is not human(e). All technologies have the potential to alter how humans perceive, organize, and value their world. One of these, virtual reality (VR), is especially effective in increasing empathy. Whether or not virtual production methods might also be used as a “moral” agent in filmmaking is yet to be determined.Meant to provoke discourse and encourage future artistic and scientific experimentation, this paper describes the artistic process behind creating our 2D short film, Fly Angel Soul shot entirely in a responsive virtual mis-en-scene utilizing three types of virtual cameras: a fly, an angel, and a human/soul, each of which sees and moves differently. We created a novel cinematic language at the intersection of theater, film, and video games. Our intention was to heighten empathy for the main character, Sebastian, a gay man who, having lived a peripatetic life, receives a diagnosis of AIDS in Mali at the beginning of the epidemic. At the same time, the film offers viewers the opportunity to consider their own embodied (and mortal) state. To contextualize the hypothetical potential for a VR-informed cinematic language to promote empathy, we also review key philosophical and neuropsychological bases for VR’s pro-social effects and the neuropsychological bases for empathy in film. We suggest that cinema informed by spatial computing might also allow audiences to experiment with and rehearse new pro-social forms of embodiment and new ways of relating in our computer-mediated world. https://revistas-fonseca.com/index.php/2172-9077
“Inspired by the words of Meister Eckhart, “let us pray to God that we be empty of God, and that we rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,’ ‘FLY ANGEL SOUL’ transports viewers into a world where spirituality, love, and technology converge.
The story unfolds as a devoutly Catholic mother, drawn by this enigmatic quotation in her estranged son’s diary, envisions an alternate past. In this vision, she is present when her son, Sebastian, receives his AIDS diagnosis at the height of the epidemic. Employing cutting-edge technologies, the film invites us to explore the tension between the constraints of the physical world and the human pursuit of something greater.
‘FLY ANGEL SOUL’ introduces us to a groundbreaking narrative approach through three distinct eponymous virtual cameras: fly, angel and soul, each offering a unique perspective. This isn’t traditional cinema; it’s a journey into the uncharted territory of computer-encoded viewpoints and the unscripted depths of the human heart. The ‘soul cam,’ masterfully embodied by Clémence Debaig, a choreographer, dancer, and creative technologist, who “plays” the role of Sebastian’s mother, acts as a cinematic avatar, allowing viewers to actively participate in the narrative., as she navigates a world where past and present intertwine.
With FLY ANGEL SOUL, we utilize virtual production methods to radically alter the process of filmmaking. Here, liveness rests as much in how Debaig responded to the mis-en-scene (spatial sound, captured performance, architecture, lighting, and objects) as it changed in real-time as in the actors’ performance (captured prior to filming with volumetric video). This sense of liveness is augmented by Jarrah Gurrie’s dynamic editing.
Soul Cam scene 2
Soul Cam perspective Scene 1
Incorporating an immersive first-person perspective and weaving in audio monologues drawn from Sebastian’s fragmented diary entries and those who shared his life, viewers are not just spectators but integral participants in the narrative. Together, yet maintaining our individual viewpoints, we delve into the themes of human suffering, love, and faith. Meticulously designed scenes guide us through a dynamic journey, shifting from expansive computer-generated landscapes to intricately hand-painted cathedral-like interiors. These transitions parallel Sebastian’s impending diagnosis and the consequent erosion of his freedom, mirroring our own progression from discovery to intimate engagement.
Fly cam perspective Scene 1
In a pivotal moment, The Doctor challenges Sebastian’s self-condemning logic, setting the stage for a powerful transformation. As our story culminates, an initially imposing interior yields to the embrace of nature. The sun’s harsh glare softens into a starlit night sky, symbolizing both Sebastian’s realization that he, too, is worthy of love and the realization of his mother’s provocative assertion that her son has “returned to God.”
Here is my talk on XR and Theater. If you are interested in discussing further, please get in touch! I love thinking! SLIDES
SLIDE 1: I’m Illya Szilak. I’m a writer, artist, director and also, a medical doctor.
SLIDE 2 : I’m going to start with a little movie from a hundred years ago. This is really the beginning of film as an art form. In this film, which is only about 2 min long, you will see what looks like a cosmos, generated by Man Ray be sprinkling salt and pepper on film and it will end with a body, of sorts. Man Ray was a surrealist. To some extent, World War 1, change everything. Suddenly nothing made sense and human progress, those enlightenment ideals of reason, were shown to be if not utterly bankrupt, not easily achieved. So, Man Ray made this. As you watch this, I want you to think about who or what is the actor? How does “liveness”, the quality of being alive and having agency arise in this work? And, is it reason that allows you to make sense of what you see? What organizes and connects information in the scenes of the film? Is the film as a whole a kind of performer? These questions of performativity and liveness are ones to which we will return.
PLAY SLIDE 2
SLIDE 3
To begin this talk, I am going to start with some very basic assumptions that I hope you will agree with. First, is that storytelling is perhaps the quintessentially human way of organizing information. Second, technology allows us to access information that is otherwise inaccessible to us through our senses. Third, the repercussions of this are not entirely benign. • In a world where information is being created so quickly, as you see on this slide, most of it unstructured that is to say it’s not part of a predefined data set or model, the critical question is how do we choose what information to pay attention to and value? How much does technology itself determine what kinds of information we pay attention to and value?
In this talk, I’m going to delve into some philosophical and neuroscientific perspectives on this. I’ll argue that although all technologies from language on up have influenced how humans see and organize their world, the impact of computer technologies is unprecedented. I will argue that in this Anthropocene age, we must find a way to relate to each other and to the planet differently if we are to survive and thrive as technologically extended humans. Theater is a way of playing out, rehearsing ways of being in the world and also seeing ourselves through the technologies that we use. XR Theater is especially powerful as it allows us to inhabit two positions, self and other at once. It allows us to play with new forms of liveness, and in a lived, as opposed to purely theoretical way, explore what we machine extended beings should value. Do we value the body? Do we value gravity and work (in all its meanings). Which ways of perceiving time and space do we privilege? How do we represent ourselves? How do we relate to, represent, and acknowledge (or not) gender, race, age, species, ability in machine extended environments.
To start, I’m going to take you on a short foray into how philosophers of the machine age have thought about how we organize information.
SLIDE 4: BERGSON We won’t start too far back. Let’s start when media technology really started to take off telephone, photography, phonograph, film. The year is 1894. This is when French philosopher Henri Bergson published an amazing book which all of you should read. In Matter and Memory, he suggests that the universe is made up of an infinite number of images: sound images, thought images, visual images, etc. and the one’s that we pay attention to are ones related to two things: our bodies and memory.
SLIDE 5: UXEKILL Jump ahead thirty years and we have Jacob von Uxekill publishes a little book in which he creates a thought experiment imagining how the world to different species including the deaf and blind tick which perceives the world in terms of and responds to levels of butyric acid radiating off of a mammal’s skin. These worlds he referred to as “Umvelt” a lived-world of experience.
SLIDE 6: MCLUHAN Now jump to 1967 where Marshall McLuhan coins the phrase “the medium is the message,” arguing that content per se could not be separated from the medium. Media acts as a kind of sense organ for humans, how we now construct our “Umwelt” is not simply through our native senses but through all our media extended senses. One might even argue that with most of us living through our devices much of the time, the human “Umwelt” has undergone an apocalyptic change.
SLIDE 6: VARELA A century after Bergson, Francisco Varela and his colleagues published this book which created a revolution in thinking about how humans process information. They argued that the idea that human reason3Zxq ZXCGVHJIL;’’DF could some how be separated from the body, an idea which really took hold during the Enlightenment and became valorized with the advent of computing machines, was just plain wrong. We don’t just passively receive information from out there and translate these into abstract symbols which we then translate into actions. We enact our world.
SLIDE 9
Moving on to 2000, philosopher Villem Flusser argues for something even more provocative, technology , he suggests, doesn’t just alter the message, as McLuhan suggests, it alters the way human consciousness works, the way we organize information. The ‘apparatus’ — the camera is a tool has its own ‘program,” one which supercedes human control. Photographers essentially are playing a kind of game according to the camera’s preprogrammed rules. It is a menu of choices that arise from the apparatus itself and not from within the human.
SLIDE 10: The issue is that the images no longer help us orient ourselves into the world because their basis is an exploration of the program and potential of the apparatus, not of the lived world itself. Covid was the straw that broke all our backs, because it showed us that amidst that chaos of images, there is one that can not be captured. That is death. This is the fact: all of of us will get sick and die. Covid showed us this and it scared the shit out of us. Because we could no longer be distracted from the one reality that is beyond the reach of our technologies. Given that our computer technologies do not privilege embodiment but rather embodiment as captured and experience through image, one that removed from the evanescent now and placed into a kind of undead forever now, the question we were starkly left with is how does matter matter?
The problem of the apparatus is that human cognition is not just an abstraction, it is not an array of menu choices, it is a lived, embodied process that has evolved over millions of years. The question is whether technology is impoverishing human consciousness or enriching it. Yes, there is more information, but perhaps because of our estrangement from our lived worlds, worlds not translated through the menu of an apparatus, that only leaves us in chaos?
SLIDE 11: It does leave us in chaos. Let’s go back to 1951. The Macy Conferences where information theory was born. What is information? On one side we had Claude Shannon, brilliant mathematician and computer scientist. He wanted to find a definition for information that would be quantifiable and something machines could understand.
The best way to illustrate Shannon’s definition of information is that the flip of a double headed quarter contains no information because there is only one possibility. Physicist Donald Mackay disagreed. He argued that meaning acts as an organizing function for a perceiving subject, it had an impact, it was not merely related to the number of bytes.
SLIDE 12 But, Shannon’s formulation, the efficiency of the code, the losslessness of transmission won out over MacKay’s meaning-making. As N. Kathryn Hayles points out, Shannon’s definition of information is essentially a disembodied form of information. So, what we are left with is a situation where human cognition is embodied, but our Umwelt as constructed by our technologies is not. And, We have no idea what this means.
But, consider for a moment what it might mean in terms of how we relate to our planet and our fellow humans and other sentient beings. I don’t have to put up graphs about the rates of opiate overdose, depression, suicide, mass shootings, species annihilation, ecosystem destruction to make a case for why we need to figure out new ways of being in the world and with each other and fast. The problem is that human beings have trouble with chaos, it makes us really nervous. So, as the totality of information increases and the probability of any one answer decreases, well, we got trouble. Plato kicked the lyric poets out of the polis for a reason. Fascism, religious fundamentalism, and, weapons of mass destruction or just down-home schoolyard gun massacres well these are all understandable responses.
Although this information explosion and concomitant disorientation, which honestly, is this close to insanity can be and will be and is being placated for a time through the dispersal of this tension through the pure act of repetition and compulsion (our swipes, our constant checking of emails and circulation of memes and data), our fear of annihilation by distributing ourselves in multiple identities across virtual spaces, a withdrawal from attempts at complex meaning-making to Tweet size bites, 10 second TikTok videos and Pixar like bright and shiny emotions. Even so, the center will not hold forever. Something is going to break.
SLIDE 13 There is no going back, return to the days of unextended embodiment. The horrible wonderful fact is that our ability to extend ourselves in space-time through machines has brought us to a critical moment where the death drive and Eros meet. Metamorphosis is to acquire a different border between self/other, a different form, different senses, different ways of relating to , privileging and processing information and our relationship to other living beings and to our home, this planet.As marabou suggests, Metamorphosis is what occurs when fleeing is necessary but impossible. And, it is what we need now.
SLIDE 14 Apologies to Elon Musk, your flight to Mars, yes, I understand it on a human level of terror and imminent annihilation, but, sweetheart, it ain’t going to work. And, by the way Twitter or X or whatever, probably not helping things… The bottom line is we must find a way to relate to each other and to the planet differently, because not everyone is gonna be on that spaceship.
SLIDE 15 HARAWAY As such, it is critical that artists stake a claim and use new technologies to interrogate the relationship between brutal historical, political and material realities and the pernicious “correction” of these facts, not only through omission e.g. Facebook Horizons, but also, and perhaps more importantly through “neutral” software based regulation of identity, behavior, interaction and access to and production of information in digital spaces. As Haraway suggests: the relation between organism and machine has been a border war. The stakes in the border war have been the territories of production, reproduction, and imagination. And, I would suggest, survival.
Where will this border war take place except in our digital spaces? One could attempt to hack dominant technological systems. It has been done. But, it’s probably unlikely that you will bring Twitter or X or whatever down for long, the question then becomes one of finding ways to be, interact and live differently inside those dominant technologies, that is to say, we must begin to play with and in them. We must not merely be users, but players.
SLIDE
So, that brings us to theater. In 1924, Italian Futurist F.T. Marinetti comes up with the idea for “total theater.” According to scholar Pierpaolo Antonello, “Marinetti saw in the theatre one domain where art could intervene to radically alter the cognitive and perceptive experience of the spectator.” Marinetti’s productions were not script-based, but multi-sensorial and performative. Among the senses, Marinetti privileged the sense of touch.
Marinetti’s Manifesto of Tactilism published in that year, is certainly worth a read in its own right, it is hyperbolic and poetic, but especially fascinating for me is this last paragraph which posits the possibility that there could be other senses, other kinds of cognitive inputs, beyond language and our five senses. I think what Marinetti is actually getting at is the possibility that a work of art might be able to elicit these new ways of experiencing the world. In other words, an artwork might enable us to overcome the conventional limits of how we perceive the world, not merely conceptually, but also materially as a lived experience.
SLIDE 17 Marinetti anticipated the possibility of total theater a century before that technology became widely available (thanks Apple!). So why not just immersive theater as Marinetti imagined it? Why XR. The power of XR with regards to total theater has to do with two factors. First is the ability of XR especially VR to generate a remarkable sense of presence—That sensation of being here and now in time and space that seems unmediated. There are many theories of presence in XR but the one that I like from Riva and colleagues draws upon the work of neuroscientist Antonio Damasio. Riva suggests that presence is a primordial form of consciousness, one that helps animals distinguish self from other, and internal from external stimuli. Interactivity especially action upon the world is critical to facilitating this. Through interactivity, an organism creates a motor-map of the universe. This allows it to plan future actions in the light of past knowledge.
SLIDE 18 This slide outlines Damasio’s description of the creation of the self, based on this, Riva describes three forms of presence. Proto presence is an organism’s ability to couple perceptions and movements and thus differentiate itself from the external world. Core presence is an organism’s ability to focus sensorial experience on present tasks and relevant objects in the environment. Extended presence is an organism’s ability to recognize the significance of previous experiences as they relate to the self. So in this last personal, historical, aesthetic and cultural memory play an important role.
So, if presence and self creation are linked, it’s clear that the power of XR for playing with new versions of being and being in the world is that the self of the audience occupies two positions in space and time. This duality may be formally acknowledged through the use of an avatar, or may be inferred by the visitor through the architecting of two or more space-times within the XR experience.
SLIDE 19 In XR, meaning-making in XR theater occurs in the oscillation between two registers within the audience/participant. This relates to the dual nature of self as embodied self and as avatar/other that XR encourages.On the one hand there is the sensual self the enacts an environment from the five senses, on the other hand there is a symbolic, representational self. Clearly the former is concerned with affect generated by action, the latter is concerned with how the self is represented which can be recognized on a cultural, historical and aesthetic level. In this way, XR theater offers the possibility of a new kind of storytelling, an active, enacted kind of storytelling—This language relies far less on symbols and signs, and more on code that elicits both the muscle memory of conventionally embodied sensations and the emergence of new ones. In such a language, affect and narrative results procedurally, through form and process even more than through content. The new form of being elicited in XR theater oscillates between self and other, material and virtual body, life and art, audience and performer. XR Theater is a laboratory for this metamorphosis.
SLIDE 20 In this slide I want to point out some of the dramatic tensions which operate generally in XR theater. These do not necessarily take the place of conventional elements of dramatic tension such as opposing characters, and dramatic action, but they are important to consider. I don’t have time to go through all these and I’m sure there are others, but I’d like to highlight a couple of these.
The first is thinking about the relationship between embodied material reality and the transcendent body interrelate. Two other that I would like to highlight is the relationship between the phenomenologically situated body and the character or symbolic body most commonly this might be experienced as an avatar. Now, not only do people project bodily action onto an avatar, such that the action of the avatar body overrides the orientation of the physical body, but how that body is represented becomes important. To represent an avatar body is anime form and using volumetric video are not the same. There are situations when the affordances of one are more useful. I’d also like to point out that aesthetics become a hallmark of authenticity in these worlds where everything is unreal. This is why art matters. Now, perhaps, more than it ever has. Technical images can not be judged as true or false but only as art or kitsch. Unfortunately, but not surprisingly, the tech industry privileges hyperrealism and Pixar seductiveness above all other aesthetics and emotional registers. AND< we love it! Of course we do, who could blame us? Like Elon Musk, we are fucking terrified. Kitsch is mother’s milk. In its embrace we can distance ourselves from chaos through irony or come together so we can all cry Milan Kundera’s second tear together. The memory of the “real”, preserved as codes borrowed from shared cultural experience and the memory of the body, preserved as Kitsch aesthetic and genre and gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms might be revealed and uncovered with the aid of human-computer interfaces remains to be seen.
SLIDE 21
Because, the point of XR theater is not to reconstitute traditional theater in the digital space, but to utilize the the affordances of technology to allow the art form to evolve in a way that speaks to and through machine extended bodies.. XR theater may not look like conventional theater. In fact, not only does it take its inspiration from multiple arts involved with spatial construction and movement, it may, take on the form of any of these.
SLIDE 22 Understand that in XR theater, the environment, the objects in that environment, and the virtual or other humans in the play all perform. This is not new. We saw this in Man Ray’s film, that nails can “dance.” Is this liveness? No, it is a simulacrum of liveness which, as we know from the concept of aesthetic sympathy, the original meaning of empathy, nevertheless constitutes a way for us, the audience, to experience our own liveness, our own bodies dancing like nails. The word empathy which psychologist Edward Titchener coined in the early 1900’s, did not mean what it means today, rather it was seen as a form of aesthetic sympathy, one which allowed audiences to appreciate the the abstract movements of modern dance and painting which were just emerging. Here is a beautiful evocation of that empathy.
SLIDE 23 LIVENESS
Since, in XR both environment and objects and virtual humans are responsive and respond, what constitutes liveness in XR theater? I do not have time to go into the debate represented in these quotes, but my own take is that our experience of “liveness” manifests on many levels, some more familiar and proximate than others. It is certainly not limited to conventional live performance. As long as the audience themselves is conscious, that is what is required. A live audience member brings with them a sense of self and a sense of being changed or wanting to be changed or touched in some way by the experience. So without getting into a debate on the nature of consciousness, I’d say that liveness brought to the work of theater by the audience relates to her awareness, her sense of self/other and to desire. Her consciousness is the spark that energizes the whole machine bringing it to life. In this way, the work of art itself is a kind of cyborg entity with manifestation of that liveness operating on multiple registers.
SLIDE 23 XR theater should explore new forms of “materiality” which I list here. To illustrate some of these concepts, I am going to dive into our newest project FLY ANGEL SOUL supported by the Jerome Foundation. Fly Angel Soul is a short experimental narrative film shot within virtual reality. It tells the story of Sebastian, a young gay physician estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart “(let us) rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” Fly Angel Soul is shot in real-time, from the unique points of view of three networked virtual cameras adopting the “roles” of the eponymous characters. The “human” p.o.v. will be that of a live cinematographer moving through the virtual set. Thus, in Fly Angel Soul, “liveness” resides in the “embodied” cameras even more so than in the actors in the story. Finding commonality with video games and live performance, Fly Angel Soul explores the potential for virtual production techniques to expand 2-D cinematic language. The human camera viewpoint which predominates in this first scene, what we affectionately refer to as the soul cam, is virtual as are the angel cam and the fly cam. Created by an amazing developer and Unity engineer named Elliott Mitchell. But, unlike the angel and the fly cam, an actual human, dancer, choreographer and creative technologist Clemence DeBaig created the path that the soul camera follows through the virtual mis-en-scene in Cinemachine.
SLIDE 24 Our goal here was to use our experience and skills honed in creating interactive stories and installations to create a film which put the film audience in a position of choice. Whether it is conscious or not, each viewer will have a preference for one or other of these perspectives. We have already found this to be the case among the film’s own team members. This points out not only the agency of the viewer, but also their responsibility. Since the world can be viewed in multiple ways, something both art and technology show us, then, how we choose to view it and how we choose to tell a story, what information and perspectives we privilege depend on what we value. There is no singular truth. The old gravitational centers of God and tradition are gone or fading fast, this is beauty and terror wrought by our technologies. But, amidst all this, there is one truth that few of us can see differently, one which Covid made us acutely remember, we are all of us mortal. We will all get sick or injured and die. This is what this story is about, the imperative to find new ways of relating to ourselves and each other. I have an MD after my name and I still practice medicine actually at Rikers Island in New York City, so my relationship to mortality and embodiment is perhaps less abstract than most people in tech. It was of critical interest to us that we find a way to use technology to make the viewer feel as though they were really intimately present in the scenes.
SLIDE 25 There were cinematic inspirations for this especially the long tracking shots in Max Ophuls films and Birdman. And, the use of gesture and architecture in Maya Deren’s films.
PLAY SIDE 26 I’m going to play a little bit from Birdman. Notice the architecture, notice the multiple rhythms of movement including sound, camera movement and actor movement, how this unites everything into an incredibly immersive experience.
SLIDE 27 First I want to let you know that I have never actually made a film before, so although I know we completely upended the usual workflows for filmmaking, it didn’t actually seem weird to us. Once I had a script, we began to work on what I call a responsive mis-en-scene: this was a responsive environment including architecture, spatial sound, and movement based triggers. To facilitate our collective imaginings, my artistic partner Cyril Tsiboulski, created a quick mock up in VR Chat from the initial architectural sketches that artist/architect Paolo Barlascini had come up with. We actually met there with our actors and rehearsed. Here, some of us are in VR Chat.
PLAY video
SLIDE 28
Aesthetically, we were interested in finding a sweet spot between the real and the transcendent, between intimacy and overt theatricality. For instance, the doctor’s clinic is an impossible structure. On the outside, it references Malian mud architecture, on the inside it references the radial architecture of the original plan for St. Paul’s Basilica in Rome. This is obviously possible only in VR. We could have created everything virtually, but we really wanted the hand of the artist to be present in the form of mark making, a trace of the artist’s performance. So Paolo painted and drew and then scanned these and working with developer Emmy Yupa, these paintings and drawings became the skins for the walls of the interior of the clinic. In addition, we worked with the sound designers so that the soundtrack could be responsive to the movement of the human camera and robust enough to sound great each time. Because, this film can be reshot and reshot. Lastly, we worked with Clem on movements for the actors. Mostly the second scene is literally waiting. The dialogue is created in units with breaks and movement sequences that can be triggered when the soul cam moves away. So again, we are creating a tension between intimacy and theatricality.
SLIDE 29
This intimacy, the liveness of the film comes not from the actors performances, these are essentially artifacts of a performance. They were filmed at the Microsoft mixed reality studio in San Francisco (unfortunately, now closed) and are essentially game objects. We used volumetric as opposed to Mocap or animation because we really think that volumetric not only captures the nuances of human movement, but it registers as a trace of the living, the real, it feels like an authentic memory of something that happened. But, the liveness of the film comes from the living camera and the misenscene that responds in real time to her presence.
SLIDE 30 This May we shot the soul cam path at the NYU Tandon Mixed Reality Lab. The first day was taken up by hardware malfunctions. The second day was revelatory. We literally had no idea whether this crazy project we were working on would work. But, after allowing for a discovery phase, I asked Clem to pretend that she was Sebastian’s estranged mother who finds herself in a dream. How would she move through the landscape, how would she move in relation to him. It was a revelation. I went back and forth between watching Clem moving in real time in headset to watching her move virtually through the misenscene. It was magic.
SLIDE 31 I’m going to play a little of this film for you, it is still a draft.
PLAY FILM
I hope it shows you that XR theater can take many forms. Our goal after this is to find funding for what we call real-time filmmaking. This would combine live theatrical performance and live-streaming of FLY ANGEL SOUL made there on the spot. If you miss it, the film never existed. Filmmaking itself becomes performance.
Using cutting-edge virtual production methods, “FLY ANGEL SOUL” invites viewers on a mother’s surreal journey through an alternate past, where she confronts her son’s AIDS-related death. Guided by his diary and memories, she discovers the transformative power of love, faith, and art to transcend tragedy.
“Soul” cam still
“LET US PRAY TO GOD THAT WE BE EMPTY OF GOD…” Drawn in by a Meister Eckhart quotation from her son’s diary, the mother envisions an alternate past where her son is alive and she is present when he receives his AIDS diagnosis. The film delves into the tension between the constraints of the physical world and the human yearning for something greater.
“FLY ANGEL SOUL” employs innovative storytelling techniques using three virtual cameras, each representing a distinct perspective: the Angel, the fly, and the soul. Unlike traditional cinema, the film doesn’t focus on “getting the shot.” Instead, it embraces the unexpected aesthetics of computer-encoded viewpoints and the intricate emotions of the human heart. The perspective from the “soul cam” is crafted through the creative path of Clémence Debaig, a choreographer, dancer, and creative technologist who embodies the role of Sebastian’s mother.
Through this first-person shooter perspective, viewers are intimately connected to the characters, immersing themselves in their emotions and experiences. As the mother navigates this alternate reality, the audience becomes an integral part of her journey, sharing in her heartache, hope, and exploring the boundaries between reality and the extraordinary.
Angel Cam view
“FLY ANGEL SOUL” explores the possibilities for virtual production methods to reshape film language. Our challenge lies in empowering viewer agency and narrative-shaping responsibility and invoking a heightened sense of presence within the constraints of 2D film. To tackle this, our film production has taken on the guise of a functional laboratory, where diverse talents including visual artists, architects, developers, cinematographers, choreographers, writers, directors, sound designers, and composers converge.Through collaborative experimentation, we strive to channel our expertise in interactive storytelling and installation art, envisioning how cinema can redefine perceptual limits and challenge established concepts of reality.
Fly Cam view (Work in Progress)
Shout out to the Incredible team! And my longtime creative partner Cyril Tsiboulski.
Michael Debartolo Sebastian Ingrid Jean-Baptiste Bathilde Tommie J. Moore The Doctor
Clemence Debaig Choreographer & Creative Technologist Christoph Mateka Composer Leo Kuraitė Supervising Sound Designer Paolo Barlascini Lead Visual Artist Produced by Illya Szilak & Cyril Tsiboulski Executive Producers Allen Yee Associate Producer BTL Volumetric Production Jason Waskey Creative Director & Technical Director Kathy Saelee Producer Tiago Washburn Capture Tech Tri Le Capture Coordinator Rhonda Callahan Technical Artist Brian Cantwell Processing Supervisor Sean Bittinger Technical Artist (Data) Eric Limcaoco Stage Sound Engineer Heather Manchester Hair & Make-Up On Set Medics On-Set Covid Compliance Officer Slipery Fish Catering and Events Catering 2D Production TBD Cory Allen Peter Bell Behind the scenes reel production Immersive Development Elliott Mitchell Lead Unity Engineer Emmy Yupa Lead Graphics Engineer Post-Production Sound Services (TBD) Post-Production Design Services (TBD) Legal Services John Pelosi, Esq. PELOSI WOLF EFFRON & SPATES / Fancy Rainbow Counsel Filmed on location at NYU Tandon @ the Yard. Special Thanks Todd Bryant Christopher Strawley Derek Chan
Cover for the original peer reviewed conference essays
I published this as part of a peer reviewed conference on Forms of Apocalypse at University Paris 8 in 2017. It holds up and speaks to why considerations of embodiment are critical in XR. Also, warning: Heidegger ahead..
The word “apocalypse” comes from the Greek apokaluptein meaning to uncover or reveal. In keeping with the word’s origins, I approach the concept of apocalypse not as the destruction of the material world, per se, but, epistemologically, as the end of the world as we have known it. If not the primary agent of this epistemic cataclysm, computer technology, which increasingly mediates human perception of reality is increasingly reflective of it. Unsurprisingly, these technologies have also dramatically changed how we communicate about the world. In his essay, “Database as a Symbolic Form.” new media theorist Lev Manovich describes this as a move from “narrative” to “database” storytelling.
As a cultural form, the database represents the world as a list of items and it refuses to order this list. In contrast, a narrative creates a cause-and-effect trajectory of seemingly unordered items (events). Therefore, database and narrative are natural enemies. Competing for the same territory of human culture, each claims an exclusive right to make meaning out of the world. (Manovich, 1999 “Database as a Symbolic Form”) Manovich concludes his essay by praising the Soviet film director, Dziga Vertov whose Man With a Movie Camera, he argues, successfully creates a new kind of narrative by ordering a database of images around the “kino-eye” of the filmmaker. This process of discovery is film’s main narrative and it is told through a catalog of discoveries being made. Thus, in the hands of Vertov, a database, this normally static and “objective” form, becomes dynamic and subjective. More importantly, Vertov is able to achieve something which new media designers still have to learn — how to merge database and narrative into a new form (Manovich, 1999, “Database as a Symbolic Form”).
Whereas, even today, both the novel and cinema remain mostly on the side of conventional narrative, many new media forms resist the linear arrangement of information. Of these, virtual reality (VR) most directly and completely mediates visual sense perception. In the VR headset, the world disappears, replaced by an alternative reality generated by a computer. Here, the user is no longer a passive consumer of someone else’s database. By exploring and interacting with the environment, she orients herself according to her own interests. One might even go so far as to say that she becomes a kind of film director framing close-ups and long-shots. Philosopher Vilém Flusser has characterized the time we live in as an epistemic crisis brought about by the recognition that, “scientific research is not the gesture of a transcendent intellect.” What Flusser calls “the gesture of searching,” for meaning or knowledge, has shifted from “a digging down for reasons” to an examination of the aesthetic and relational qualities of things and an exploration of how humans respond and attend to their environments as subjects and as objects (Flusser, 2014, “The Gesture of Searching”). Perhaps unsurprisingly, Flusser’s description of this gesture serves as an apt description of being in virtual reality, a reality which has become, in essence, a database […] the researcher is embedded in an environment that interests (matters to) him, both at close range and at a distance. There are aspects of the environment that interest him intensely and others that hardly touch him. The more an aspect of the environment interests the researcher, the more ‘real’ it is for him (Flusser, 2014, “The Gesture of Searching”).
In this essay, I will build upon Manovich’s observation that, “cinema language, which was originally an interface to narrative taking place in 3- D space, is now becoming an interface to all types of computer data and media” (Manovich, 2001, 326). I will argue that whatever the language of VR turns out to be, it will rely far less on symbols and signs (content) and more on a procedural code (form) that elicits both the memory of the body’s movements and affects as well as a shared cultural memory, not of the lived world, but of cinema, itself.
What determines the authenticity or veracity of this kind of storytelling? Here, it is useful to consider one of the earliest, most widely distributed experiences in VR. The Tuscan Villa (https://youtu.be/wuFTd5TVhHw), was provided free with the developer kit for the Oculus Rift DK2 head mounted display. It offers users the chance to meander through a country villa and surrounding landscape. A generic looking computer-generated farm house, its stones visually coded as “old,” suffices to invoke a nostalgic memory—one that will be even more poignant and believable to a viewer who has never actually stayed in a Tuscan villa. In other words, the Tuscan villa is meant to exist outside of history. In this work, the visual and auditory codes are perceived as “authentic” if they successfully activate a process of false memory and effect a change in the viewer’s affective state. Significantly, the experience does not provide a traditional narrative. Rather, it offers the vague sense of remembering a dream or retrieving something that has been lost which never actually existed.
Kitsch is nothing if not a suspended memory whose elusiveness is made ever more keen by its extreme iconicity […] kitsch is not an active commodity naively infused with a wish image, but rather a failed commodity […] a virtual image, existing in the impossibility of fully being. Here, for a second or even a few minutes, there reigns an illusion of completeness, a universe devoid of past and future […]. (Olalquiaga, 1998, 28).
In The Language of New Media, Manovich points out that as cinema becomes more and more reliant on software manipulation in postproduction, it functions less as “a record of perception,” and more like a computer screen— “as a record of memory” (Manovich, 2001, 325). In the case of the Tuscan Villa, cinema no longer serves an indexical function vis a vis reality, but rather, it registers the trace of a shared cinematic memory. The kitsch experience of the villa and garden offers a momentary illusion or simulacrum of “realness”. For this, it sacrifices the passage of time and complexity of meaning.
Flusser suggests that gesture, by which he means the “reality” of affective states made manifest through aesthetics, should be judged in terms of art or kitsch rather than whether the gesture is true or false. “The scale of values we use to evaluate (gesture) may not oscillate between truth and error or between truth and lies but must move between truth (authenticity) and kitsch (Flusser, 2014, The Gesture of Searching).
It is not artifice in itself that renders the Tuscan villa demo artless. To understand this, it is instructive to compare the Oculus demo to Maria Menken’s decidedly un-kitsch 1957 film Glimpse of the Garden (https://youtu.be/_fGg7D1naIs?t=2m10s). Even though both The Tuscan Villa demo and Menken’s film privilege surface effects over depth of meaning, the result is decidedly different. In her film, movement of the camera, a unifying looped soundtrack of birdsong, and color filters take precedence over content, which consists mostly of shots of cultivated nature. Despite the amateurish quality of the images, the result is strangely magical. Menken’s editing, which juxtaposes different scales, points of view, and speeds of movement, resists the Tuscan villa illusion of completeness. Thus, it is due to the authenticity of the filmmaking gesture itself, not its content, that Menken’s film can be called art whereas the Oculus demo remains kitsch. If there is any redeeming gesture in the Villa, it is in its ability to invoke an uncanny sense of bodily presence, a being there in a digital world that is generated by the user’s wandering or, in Flusser’s terminology, “searching” through the environment. According to neuroscientists, presence is a primordial mechanism by which organisms establish self and other, create a motor map of the universe and use memory to plan actions and interactions. In so far as the Tuscan Villa succeeds in creating this sense of presence, it sets up a queer dynamic in the user—an oscillation between interior and exterior, self and other, the “real” and the virtual. In this way it, perhaps inadvertently, succeeds in making the familiar strange. In other words, the odd and unresolvable experience of being in two places at once creates a kind of digital estrangement conceptually related to Bertolt Brecht’s Verfremdungseffekt, distancing effect, and Russian Formalist critic Viktor Shklovsky’s “making strange” (Russian: приём остранения priyom ostraneniya) whereby certain devices are used to disrupt the audience’s easy identification with an artwork.
To see how this might be productively used in VR, I turn to filmmaker Karen Cytter’s work for a contemporary example of estrangement. Cytter employs familiar, even kitsch codes of cinema, especially melodrama. But, rather than offer an illusion of completeness, she uses what Thomas LaMarre refers to as “internal montage” (LaMarre, 2009,125). LaMarre suggests that computer graphic imagery and digital effects have created a new emphasis on the animation technique of compositing —the creation of an image composed of multiple layers. If an artist suppresses the space between layers to create an illusion of wholeness, the result is either a unification in depth, a hyper- Cartesianism with a well defined vanishing point (e.g. Pixar type animations), or a flattening of all layers into a single “superflat” plane in which multiple frames of reference and sight lines coexist. Cytter’s films demonstrate a remarkable compositing of space, dramatic events, and sound. It is effectively a jumbled cinematic database in which voice, visual form, emotion, and plot nearly float free of each other on the level of meaning. Surface effects are at least as important as content here. Through a process of making the familiar strange, she astutely reveals the underlying game of seduction that takes place in cinema. In Cytter’s work we are left with empty codes, empty signs which nevertheless incite emotional responses. Devoid of linear narrative, in Rose Garden (https://vimeo.com/87553434), the mournful sound of the flute is not only a wry reference to Sergio Leone’s Once Upon A Time in the West, it codes the psychological tenor of the piece. Although there are no clear “characters,” as with avatars in video games, the human body remains the basis for emotional engagement. For instance, at the end of the film, when a boy is shot in the back, though we know nothing about him as a character, the violent collapse of a child’s body is still shocking and emotionally wrenching.
Rose Garden’s flattening out of time and space, and its move from depth to surface, which Flusser relates to the gesture of searching, resonates with Hiroki Azuma’s description of database narrativity. In his book, Otaku: Japan’s Database Animals, he notes a move from “grand narratives” to “grand non-narratives” or database (Azuma, 2009, 55). This is associated with the proliferation of small narratives the purpose of which is not to refer back to some greater or universal theme, but to invoke strong feeling or fulfill consumer preferences. Whereas, in the modernist era, personal fictions could be related in depth to a grand story about the nature or meaning of human life, now, there is a lateral move in which multiple small stories are spawned from the same database as in Cytter’s film. Likewise, in virtual reality, narrative unfolds as movements through space. However, distance is no longer measured in terms of meters or any other universal objective standard, but, as both Flusser and Azuma suggest, in terms of the user’s desires and interests interacting with the environment.
In Mark Amerika’s cell phone film Immobilité, one of the characters remarks that the world is disappearing before her eyes. Indeed, the opening scenes, a shaky, looped walk down a tree-canopied path, reveal just how the world will end— not in some fiery apocalypse, but rather by our recognition that the world was only ever virtual. In Immobilité, what remains of embodied experience, what Amerika calls “muscle memory,” finds its expression not in meaning, but in form. In effect, Immobilité — a film that remixes some of Flusser’s thought into the subtitle track — consists not so much as a series of images, but as a series of film gestures.
In Immobilité, the quality of the image is sacrificed to movement of the camera just as content is sacrificed to surface effects. Depth, in the form of philosophic musings, is separated from the images and brought to the surface as subtitle. What remains on the screen are vibrations, rhythms, and frequencies that resonate with or cancel each other out. In fact, in many parts of the film, the camera is decidedly unsteady or even appears to vibrate thereby creating a surface that destroys a Cartesian perspective and the static perceiving subject. “And then I would begin to lose myself, to play uncontrollably, becoming something like music,“ the unreliable narrator texts us from the beyond.
Heidegger’s concept of “profound boredom” operates here. This is state in which all beings “recede” into indifference and the one who is bored comes face to face with the experience of time itself (Heidegger, 2001, 80). Heidegger describes profound boredom as the ground for all the multifarious variations of being. Profound boredom leads us to suspect that reality is neither static (immobile) nor completely knowable. Concomitant with this loss of a fixed self, language fails to signify. Rather, it functions procedurally to carve up pure duration into pieces: life and death, existence and nonexistence, chaos and order. This resonates with Henri Bergson’s suggestion in Matter and Memory that homogeneous and universal perceptions of space and time are a trick of the mind, a refraction of pure duration into space. For Bergson, space and time do not exist anterior to perception and action. We do not act through space and time, rather we create these symbolic diagrams because we need to “divide the continuous, fix the becoming, and provide our activity with points to which it can be applied (Bergson, 1990. 212). Bergson suggests that our knowledge of the world is delimited by the concerns of the body and its possible actions and, perhaps, even more importantly, by our memories. If that is true, we are like Plato’s cave dwellers: always living in a virtual reality of our own making.
The key dynamic in Amerika’s film, one that will likely dominate in this time of apocalyptic narrative, is a tension between overt aesthetics and conventional notions of truth. Although Immobilité’s low production values and distinct lack of artful montage lend the film a “real-life” documentary feel, Amerika’s overt use of glitch, repetition, and jerky camera movement insists upon a kind of self-conscious performativity or aesthetics. When the narrator asks, “Was I authentic?” — the question is equally what does “authentic” mean if everything is virtual and, ultimately, data, and, furthermore, who is “I”? If the world is disappearing, so is the singular author, that figure of the auteur which haunts French New Wave cinema, to which Amerika pays homage in his use of subtitles. In Immobilité’, Amerika, like Flusser, suggests that the question of “authenticity” lies not in a “true” identity but in the aesthetic value of the gesture itself.
Antonioni’s masterwork, Zabriskie Point, also operates by disavowing conventional notions of authenticity. In Zabriskie Point all is surface: from the setting, an almost abstract stretch of ancient desert, to the way images are composed in multiple layers without depth. There is also a strange flatness to the main characters. They seem to have no ambitions and no particular future. With exquisite conciseness, Antonioni destroys the epic march of history with the first line spoken by the male protagonist: “I’m willing to die (for the revolution), but not out of boredom.” For the characters in Zabriskie Point, time operates as it does in profound boredom as a pure duration. Even on a personal level, events have no apparent consequence. Though one of the young lovers dies violently, there seems to be little emotional repercussion for the lover that remains. This flatness of character is matched by the physical beauty of the actors. It does not matter that beneath they are voids, the viewer can still be seduced into watching. In the end, it is up to the viewer to decide if Zabriskie Point’s aesthetics, which privilege form over content, offer any meaning.
The last scene of the film, which is overtly apocalyptic, masterfully captures this dynamic. Here, the female lover imagines blowing up the ultra-modern house of her boss, a slick real estate developer who wants to produce a housing development in the desert. Antonioni’s slow motion explosion which shows mostly intact objects floating across a horizonless, flat blue surface of sky disrupts conventional Cartesian perspective as well as scale. Moreover, using slow motion and repeating the moment of the explosion from multiple points of view, Antonioni disrupts the conventional flow of time and a singular perceiving subject. However, as with the rest of the movie, nothing actually transpires. The film concludes with the woman, a virtual terrorist, driving away from the intact house without a clear destination. The strange physics of Antonioni’s explosion brings to mind the procedural physics operative in the software used to create VR experiences. These are the rules by which objects behave in a virtual 3D landscape, rules of interaction put into play by an algorithm in a game engine. As Baudrillard notes in his odd little book Seduction, games are subversive precisely because they allow for a reversibility and fluidity of meaning that operates outside of law (Baudrillard, 1991,131-157). Because procedural physics operates outside of natural law, it offers the potential for radically new forms of narrative. As the quote below from Donald MacKay, one of the key figures at the Macy Conferences on Cybernetics, points out: meaning, itself, acts as an “organizing function” for a perceiving subject. Significantly, MacKay defines information according to its functional impact, without respect to whether it is “true” or “false,” an idea which resonates with Flusser’s insistence that aesthetics determines the truth of a gesture.
INFORMATION can now be defined as that which does logical work on the organism’s orientation (whether correctly or not, and whether by adding to, replacing, or confirming the functional linkages of the orienting system). Thus, we leave open the question whether the information is true, false, fresh, corrective or confirmatory, and so on… The MEANING of an indicative item of information to the organism may now be defined as its selective function on the range of the organism’s possible states of orientation (Wilden, 1982, 236).
Discussing filmmaker Peter Greenaway’s move from film to installation, Lev Manovich suggests that spatialization allows Greenaway to construct pure database narrative, something which film does not allow. No longer having to conform to the linear medium of film, the elements of a database are spatialized within a museum or even the whole city. This move can be read as the desire to create a database at its most pure form: the set of elements not ordered in any way. If the elements exist in one dimension (time of a film, list on a page), they will be inevitably ordered. So the only way to create a pure database is to spatialize it, distributing the elements in space. (Manovich, “Database as a Symbolic Form”).
Following Bergson’s emphasis on the body as the starting point for perception, I would counter that this does not constitute pure database. At least in the “real world,” the narrative structure of installation arises out of the interaction of bodies with the architectural environment. As Lakoff and Johnson, and others have shown, human language also revolves around the body. However, in the virtual world, this is not necessarily so. Here, even gravity need not operate according to natural law and, furthermore, reading text can often induce nausea. Yet, even in virtual reality, meaning-making requires some form of structure.
One way that virtual reality narratives might recoup meaning without resorting to conventional cause and effect is by using procedural physics to structure narrative and to create asymmetries in the environment (database) which allow for meaning-making according to MacKay’s definition. Artist Rachel Rossin’s virtual reality works illustrate this possibility of a new kind of narrative, one that speaks to and through virtualized, cyborg bodies. In an early work n=7/The Wake of Heat in Collapse, Rossin uses the procedural physics of gravity itself to shape narrative. Here, users progress through three contemporary Dantesque landscapes. Gravity in the landscape decreases algorithmically as the user descends. Invoking the virtual body of the user, Rossin notes:
In video games you have a certain type of hermeneutics…what you understand as the sort of “body language” or instinct that come with the world. Gravity – especially – falling off of things usually means death. We get this sad little feeling when we watch ourselves die in a video game and there is a pause that actually feels like critical distance and so I used that as a loose Virgil… the thing that pulls you through the layers of the separate worlds (Rossin, personal email correspondence December, 2015).
In a more recent work, I Came and Went as a Ghost Hand, Rossin scanned intimate photos of her domestic surroundings then manipulated these and used them to create a virtual reality landscape. As a result, the viewer sees familiar things differently, not just because of the data loss, which reduces some objects to their raw qualities – color, shape, volume – and which causes erasures, but because of the unsettling experience itself: moving through objects, speeding up, slowing down, torque. In lieu of a narrator, Rossin offers a reassuringly banal, reoccurring icon of a white gloved hand, a kitsch gesture that acts as a point of continuity for viewers as they traverse the disorienting, horizonless landscape.
In his essay, “Envisioning the Virtual,” Brian Massumi suggests that the virtual should be considered as the formative or potential dimension of reality, not as something which is in opposition to actuality. Analyzing an optical illusion where “Pac-man” circles cause a virtual triangle to appear, Massumi proposes that it is the tension between different forms of perception which produces “a field of intensity” within which the “pressure for resolution” acts as “a formative force.” The tensions are between modes of existence proposing themselves to the experience[…] The modes do not add up to a form. They are tensely, incommensurably different. Their incommensurability exerts a differential pressure. Something has to give (Massumi, 2014, 56). Massumi is describing a situation in which different ways of looking at the world seek to coexist in time. Cinema attempted to resolve this with montage, VR offers a different solution: the spatialization of time itself. How humans will organize that time-space in a meaningful way with respect to virtual bodies is evolving. Moreover, the political and social implications of this are as yet unknown. However, as I have shown, the memory of the real, preserved as a visual code borrowed from shared cinematic experience and the memory of the body, preserved as codified gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms might be revealed and uncovered with the aid of human-computer interfaces remains to be seen.
Bibliographie AZUMA, Hiroki, Otaku: Japan’s Database Animals, translated from the Japanese by Jonathan E. Abel and Shion Kono, Minneapolis (MN), University of Minnesota Press, 2009. BAUDRILLARD, Jean, Seduction, translated from the French by Brian Singer, New York, Palgrave Macmillan,1991. BERGSON, Henri, Matter and Memory, translated from the French by N.M. Paul and W.S. Palmer, Zone Books, Cambridge (Ma), MIT Press, 1990. FLUSSER, Vilém, Gestures, Kindle edition, translated from the German by Nancy Ann Roth, Minneapolis (MN), University of Minnesota Press, 2014. HEIDEGGER, Martin, The Fundamental Concepts of Metaphysics: World, Finitude, Solitude, translated from the German by William MacNeill and Nicholas Walker, Bloomington (IN), Indiana University Press, 2001. LAKOFF, George and Mark Johnson, Metaphors We Live By, Chicago (IL), University of Chicago Press, 2008. LAMARRE, Thomas, The Anime Machine: A Media Theory of Animation, Minneapolis (MN), University of Minnesota Press, 2009. MANOVICH, Lev, “Database as a Symbolic Form,” http://www.mfj- online.org/journalpages/Mfj34/Manovich_database_frameset.html, accessed online 22/01/2017., The Language of New Media, Cambridge (Ma), MIT Press, 2001. MASSUMI, Brian, “Envisioning the Virtual,” Oxford Handbook of Virtuality, Oxford, Oxford University Press, 2014. OLALQUIAGA, Celeste, The Artificial Kingdom: On the Kitsch Experience, Minneapolis (MN), University of Minnesota Press, 2002. WILDEN, Anthony, System and Structure: Essays in Communication and Exchange, New York, Tavistock Publications, 1980.
The “windows” we used for viewing in our In My Own Skin virtual installation, morph into the decorative grille/grids of Malian architecture (where the film takes place)
FAS tells the story of Sebastian, a young gay physician, estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart that we might “rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” FAS will be created using Unity’s Cinemachine software with three networked cameras adopting the p.o.v of the eponymous characters. What each camera “hears and sees and how each moves depends in part upon the actions of the other cameras in real time. The cameras function as the “players,” both in the video game sense and in a theatrical sense. The cameras’ real-time “performance” is the material for the 2D film. Thus, in FAS, “liveness” resides in the “embodied” cameras as much as in the actors whose performance is pre-recorded with volumetric video. Thus, the film, itself, is a poetic documentation of both human and computer machinations. Montage, as such, will not come through a post-production editing process, but occurs and becomes manifest as a result of the procedural logic of the game engine + the incommensurable logic of the human operator/performer. In keeping with Eckhart’s intent, the final film will display all the p.o.v’s on one screen, a tripartate montage of images and sounds, not created in post, but recorded “live” in real time.
Even though we work with cutting edge technology, I start projects with paper and scissors and glue. Today, Cyril, who is now used to this came over to discuss what I’d come up with. (queerskins a novel queerskins.com began with a 100 page pile of collages that I showed Cyril at lunch one day. They were made on children’s colored construction paper as I’d work late at night after my two young kids were asleep and that was all I had available at the time I started.)
It’s wild to see how your mind builds on things–the delving into architecture and windows for viewing in our In My Own Skin project last year is taken up ten levels. In this project, architectural space and light actual become as integral to the film as the actors.
Ophuls is a major influence for the set in scene 2 (use of multiple patterns/textures/l) which find there real equivalent in contemporary Malian apartment rentals!
We will build a set based on radial forms –see the original plan for St. Peter’s Basilica in Rome above.
Other artists’ inspire–screens by Jorge Pardo and paintings/installation by Lucio Fontana. I
In the final scene, I note that the Angel which manifests to the human as light is powerful force inspiring awe and terror.. Cyril and I spent part of our time today discussing “how does an angel see?” So, I look for this brain worm, later and find it on my shelf in Rilke’s Duino Elegies
“Who, if I cried out, would hear me among the angelic orders? And, even if one of them pressed me suddenly to his heart: I’d be consume in his stronger existence. For beauty is nothing but the beginning of terror, which we can just barely endure, and we stand in awe of it as it coolly disdains to destroy us. Every angel is terrifying. ” So, I guess this has been living in my mind all these years, waiting to emerge!
Fly Angel Soul is a short experimental narrative film shot within virtual reality which explores the potential for virtual production techniques to expand two dimensional cinematic language. FAS tells the story of Sebastian, a young gay physician, estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart that we might “rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” FAS will be created using Unity’s Cinemachine software with three networked cameras adopting the p.o.v of the eponymous characters. What each camera “hears and sees and how each moves depends in part upon the actions of the other cameras in real time. These positions are not characters, per se. They function as aspects of Sebastian’s interior milieu. Although all three p.o.v. will be equally represented in the final film, using a split screen, FAS unapologetically privileges the ineffable workings of the human heart as the driving and unprogrammable logic of the film. The human camera is the only one operated by an actual living being. The angel and fly exist as state machine, pre-programmed virtual entities.
Looking to “embodied” films like Max Ophul’s Le Plaisir for inspiration
We are not advocating an impossible return to a pre-technical state of “nature”, rather, we are asking what suffering means in our technologically embedded existence.If as Jean-Luc Godard famously stated, “The tracking shots are a matter of morality,” the use of virtual cameras in agile film production brings up pressing ethical questions which have yet to be confronted. In FAS, a “simple” and universal story of human suffering–a diagnosis of terminal illness–AIDS at the beginning of the epidemic, invites viewers to contemplate how suffering is mediated using digital technologies. Our intent is to construct the film in a way that reasserts an embodied, participatory perspective, one that acknowledges the primacy of a “human” perspective while, at the same time, offering the audience alternative, perhaps transcendent computer-mediated ways of seeing, hearing and moving through the same story.
In FAS, all three cameras function as the “players,” both in the video game sense and in a theatrical sense. The cameras’ real-time “performance” is the material for the 2D film. Thus, in FAS, “liveness” resides in the “embodied” cameras as much as in the actors whose performance is pre-recorded with volumetric video. Thus, the film, itself, is a poetic documentation of both human and computer machinations. Montage, as such, will not come through a post-production editing process, but occurs and becomes manifest as a result of the procedural logic of the game engine + the incommensurable logic of the human operator/performer. In keeping with Eckhart’s intent, the final film will display all the p.o.v’s on one screen, a tripartite ever changing montage of images and sounds, not created in post, but recorded “live” in real time.
Welcome home. Queerskins Home is LIVE. World #83614 in VRChat. Search #queer. Part art exhibition space, part 3D explorable story world. Due to constraints of VRChat, a free massive player social app, available for Oculus Quest, all tethered VR headsets and online PC, we do not have video (as does not support for Quest) nevertheless, I see the future of cinema here. It is like nothing I have ever experienced.
Of all the things Cyril Tsiboulski and I have made over the last ten years, this feels perfect–raw and unfinished and a gut punch. I know “Art” is pretty and/or cerebral and/or conceptual, but that is not what we made. So proud of it. Amazing lighting help from Unity engineering superstar Elliott Mitchell and architectural advice from Beom Jun Kim, and, of course, Cyril Tsiboulski who is my wonder twin, and created this in 4 days. Working together and on this project for close to ten years is what makes it so deep and so powerful. It took that kind of devotion…Thank you all.
Audio is a big part of this so please adjust by going to your settings on the menu and adjusting master, world sounds and people (other avatars around you) sound. Starting September 2nd, in conjunction with the premiere of Queerskins: ARK at The Venice International Film Festival, I will be leading daily tours with special guests Michael DeBartolo (who plays Sebastian) in Queerskins: ARK and Hadley Boyd (who plays Mary-Helen, his mother). She will be reading live from Sebastian’s diary. So follow us at Queerskins and @queerskins on Insta/Twitter to find out the day’s schedule.
Illya Szilak 