AI NFT’s and the WORK OF ART IN THE AGE OF MECHANICAL PRODUCTION

Jason Allen, Théâtre D’opéra Spatial.

Celeste Olalquiaga, The Artificial Kingdom: on the Kitsch Experience

In 2011, I went to North Korea. for an art project. (Atomic Vacation, a project that it will probably never see the light of day, unfortunately.) The DPRK was the kitschiest place I have ever been. It’s also the most quietly terrifying. In North Korea, aesthetic codes take the place of historical realities. A doric column in a provincial hotel lobby is an enduring and universal symbol of authority and prestige. The rococo gilt cherub chandelier in the same hotel lobby is less a functional light fixture than a vehicle for “taste” and wealth. Displaced from lived durational time and their countries of origin, these objects function as souvenirs of reality. They are kitsch.

In this essay, I will argue that the turn toward kitsch in NFTs and generative AI is related to four convergent trends:


The creation of digital tools and networks of distribution that allow and encourage infinite replication and iteration, resulting in a massive acceleration in the rate of unstructured information created, copied, and consumed.
The effects of information theory as defined by Claude Shannon in which “information” is unrelated to the meaning, structure or content of individual messages.
The privileging of affect/effect over complex meaning-making.
The privileging of machine-based procedural logic and physics over those related to the time and space of living bodies.

Moreover, I will suggest that all these contribute to a novel relationship to what Benjamin calls the “aura”— the transcendent potential of an artwork. I will show that the aura, which began to dim with the industrial revolution and mass production of objects, returns, in NFTs and AI art, in a machine-mediated fashion. For NFTs, the aura returns in a purely formal way via the blockchain. With AI artworks, the aura is approximated not by a singular work, but by the supernatural volume of unique objects which an AI produces in response to a single prompt.

In his essay, “The Work of Art in the Age of Mechanical Reproduction,” Walter Benjamin links the “aura” of an artwork to its original sacred use as a ritual object and to its “authenticity.”

“The authenticity of a thing is the essence of all that is transmissible from its beginning, ranging from its substantive duration to its testimony to the history which it has experienced. Since the historical testimony rests on the authenticity, the former, too, is jeopardized by reproduction when substantive duration ceases to matter. And what is really jeopardized when the historical testimony is affected is the authority of the object.”

See https://www.marxists.org/reference/subject/philosophy/works/ge/benjamin.htm

Within Benjamin’s framework, NFTs, which usually reference an infinitely reproducible digital file of visual image or sound, might appear to be devoid of aura. But, this is not so. Because an NFT is “non-fungible,” it is also unique. No complex, historically nuanced, provenance is required. Instead, “blockchain, a shared, immutable ledger that facilitates the process of recording transactions and tracking assets in a business network” https://www.ibm.com/topics/what-is-blockchain (accessed 8/30/22) guarantees an NFT’s “authenticity.”  Since there is nothing that prevents an NFT’s content from being duplicated to create another unique and authentic NFT, the content authenticity relates to ownership, the transfer of which is fossilized, kept as a permanent memory via the blockchain. Thus, although timeless, NFTs are born and “have a life” marked purely by ownership. Duration returns to the work of art via the operation of blockchain and capitalist exchange. The aura, emptied of human-centric meaning, returns as pure computer code.

Philosopher Villem Flusser defines code as “a sign system arranged in a regular pattern.” (Flusser pg 83 Toward a Philosophy of Photography). He suggests that every medium has its own system of signs (code) where a sign is “a phenomenon that signifies another” (pg 84) NFTs as a medium employ the blockchain as a system of signs, a code, which signifies a reality that is entirely self-referential and meaningful only in terms of capitalist exchange.

AI as a medium employs a code that signifies via repetition and iteration. Here, the aura of an artwork, the ineffable process of human creation is approximated through scale. Transcendence manifests, not through content, but as a side effect of the supernatural quantity of unique artworks spit out by the AI. In the face of this, curating becomes an act by which the artist attempts to recoup creative power. However, since the productive power can not be reclaimed from the machine entirely, AI artists augment this by playing along with the rules of the medium— privileging quantity, e.g. in the form of likes on Instagram, and sales, over quality.

In System and Structure: Essays In Communication and Exchange, Anthony Wilden likens the process of going from analog to digital to the solidification of difference into distinction. AI art is difference without distinction. It approximates depth of meaning by manifesting potentially infinite variety, that is to say entropy as it was defined by Claude Shannon. “Shannon’s theory defines information as a probability function with no dimension, no materiality, and no necessary connection with meaning. It is a pattern not a presence.” Hayles, How We Became Posthuman. This is contra the definition suggested during the Macy Conferences by physicist Donald MacKay that the quantity of information contained in a message ought to depend on the functional impact on a perceiving subject: “information is a distinction that makes a difference.” (Information, Mechanism, and Meaning) MacKay argued that meaning acts as an organization function for a perceiving subject. Thus, Shannon’s definition while mathematical and machine-readable was applicable only to telecommunications not to living organisms.

Does this machine-mediated return of the aura necessarily mean that NFTs and AI art are doomed to be kitsch? Not necessarily, however, in general one might argue that this is a predictable result of the way digital apparatuses function. Flusser defines an apparatus as “a plaything or game that simulates thought [trans. An overarching term for a non-human agency, e.g. the camera, the computer and the ‘apparatus’ of the State or of the market] ; organization or system that enables something to function.”

In his essay, “Towards a Philosophy of Photography,” Flusser describes the way that “technical” images, those produced by an apparatus, operate. No longer used for orientating human minds in a world of infinite data, that, in toto, would be and is beyond the limits of human understanding, these technical images distill the complexity of reality, becoming substitutes for reality itself.

“The technical images currently all around us are in the process of magically restructuring our ‘reality’ and turning it into a ‘global image scenario’. Essentially this is a question of ‘amnesia’. Human beings forget they created the images in order to orientate themselves in the world. Since they are no longer able to decode them, their lives become a function of their own images: Imagination has turned into hallucination.” Flusser, pg. 10.

In such a hallucinatory world, disorientation is expected. Indeed, I suggest that the repetition compulsion which is fundamental logic in the process of art-making with AI, one which mirrors our constant swiping, tweeting, checking of emails and messages, points to a desperate attempt to dissipate the tension of this disorientation. In a world where the image has become the world, the predictable rules of the material world: the law of gravity and Newton’s Second Law of Motion are replaced by aesthetic rules. To function with the authority of “laws”, these codes must be familiar as gravity, legible to all, and create a consistent emotional engagement. This may be the real answer NFTs and AI lean toward kitsch—kitsch distracts from the void, the constant whisperings of our imminent erasure as humans by machines (AI, nuclear weapons, social media), climate change, and disease, which results in a general feeling of dis-ease Thus, it is not surprising that the aesthetics of the most salable AI and NFT images (bored apes) are kitsch.

What is the simulation of thought, apparatus, involved in making NFT’s? What is the simulation of thought, apparatus, involved in making an AI artwork? To answer these questions, we might, like Flusser, examine the act of making in terms of gesture.

Flusser describes gesture as “a movement of the body or of a tool connected to the body for which there is no satisfactory causal explanation.” In other words, a gesture is the expression of a particular form of consciousness.

“If I accept that affect is a state of mind transformed into gesticulation, my primary interest is no longer in the state of mind but in the effect of the gesture. As they appear in symptoms and as I experience them through introspection, states of mind throw up ethical and epistemological problems. Affect, conversely, presents formal, aesthetic problems. Affect releases states of mind from their original contexts and allows them to become formal (aesthetic)—to take the form of gestures. They become “artificial.”

What then is the gesture of NFT’s? I will argue that it takes the form of a pointing to something in the distance, something that is so far away and growing more distant in time and space that it is barely seen or remembered. What NFT’s point toward is the loss of a shared human reality or rather our shared illusion of human community, a “brotherhood of man,” which would render blockchain encoding of authenticity unnecessary.

The gesture of AI relates to a translation of human creativity, freedom of thought, into mechanical creativity and freedom of choice. In other words, it distills human freedom into the consumerist logic of capitalism. As suggested earlier, another way to think of the gesture of AI is repetition compulsion, which Freud defined as “an urge inherent in organic life to restore an earlier state of things.” Interestingly, Olalquigia suggests that loss is an integral aspect of kitsch: “the pieces of the aura were put up for sale and personal possession became the medium for a fetishized experience of loss.”

One tactic that artists using AI have employed to resist the artist as consumer logic is to add the mystery of materiality back to the process of art making. In the sculptures of Sharan Elran and collaborators (http://www.sharanism.com/#/fine-motor-skills/) clay unmakes the ease of obtaining preprogrammed surface effects. Earth itself acts as a carrier of the traces of time passing where moisture, temperature, and other factors create the possibility of “mistake” and suddenly the machine stops. (See E. M. Forster “The Machine Stops”)

To further evaluate the reasons AI and NFT’s lean toward kitsch, we must make a distinction between kitsch and art.

Flusser suggests that all gestures are aesthetic in nature, and can not be judged as “true” or “false,” but only as art or kitsch. For Flusser, “the more information a gesture contains, the less it is like kitsch, and furthermore, that the quantity of information conveyed by the gesture is related to the gesture’s code.”

If we take Shannon’s winning definition as information being related to entropy, kitsch operates as kitsch because of its predictability. In a time of disorientation and chaos, kitsch is the toss of a two headed coin. Indeed, compared to the rich history of provenance, how much information does the gesture of blockchain contain? The answer is that, a single block in the bitcoin blockchain is about 1MB. At the level of human meaning, very little, at the level of Shannon’s purely mathematical quantification, the procedural logic of NFT’s is intrinsically kitsch.

The kitschiness of AI art may relate to the fact that most AI learn from data sets. They do not learn as humans do by moving through and interacting with the physical world. Human minds are messy because bodies are messy. Although it is beyond the scope of this essay to discuss embodied cognition and AI, I suggest that AI art reflects a loss of embodied existence and a physical world as opposed to a world constructed of text and images.

Importantly, kitsch does not have as much to do with content but, in how that content is consumed. Clement Greenberg in his famous 1939 essay “Kitsch and the Avant-Garde” suggests that “kitsch pretends to demand nothing of its customers except their money-not even their time.” Moreover, he suggests that the basis for kitsch is the same as for the avant-garde: “The precondition for kitsch, a condition without which kitsch would be impossible, is the availability close at hand of a fully matured cultural tradition, whose discoveries, acquisitions, and perfected self-consciousness kitsch can take advantage of for its own ends.”

The difference between the avant-garde and kitsch with respect to aesthetics lies in a privileging of “form” over content. Boris Groys quoting Greenberg, is instructive. “Greenberg defines the avant-garde as mimesis of mimesis: the avant-garde reproduces the artistic form of traditional painting instead of reproducing its content.” from https://www.radicalphilosophy.com/article/romantic-bureaucracy-2″

With this reasoning, one could make the argument that the mechanical, purely formal return of the aura that occurs with NFTs is an instance of the avant-garde. It is. One might insist that the hollowness and superficiality of most NFT content is a tacit acknowledgment of this, but, I don’t think it is. The fact is that kitsch has become rife in other digital media as well, especially AI art.

I suspect that the proliferation of kitsch has to do with a digitally-morphed perception of time and space and also to a change in social relations wherein the reproduced object becomes the medium via which, increasingly, especially during covid times, social interaction occurs. In digital media, the decay of an image that occurs with the transfer of data does not adhere to the image like a patina of time. For the digital image, this degradation exists only as mistake, as loss of data, or obsolesce. It is not productive. Similarly, in digital existence, site is replaced by a mobile and infinitely reproducible screen. A classic example of this is the Tuscan villa demo which came with the original developer kit for Oculus Rift, now close to a decade ago. This early “consumer” VR experience offers users the chance to meander through a country villa and surrounding landscape. A generic-looking computer-generated farmhouse, its stones visually coded as “old,” suffices to invoke a nostalgic memory—one that will be even more poignant and believable to a viewer who has never actually stayed in a Tuscan villa. In other words, the Tuscan villa is meant to exist outside of history. In this work, the visual and auditory codes are perceived as “authentic” if they successfully activate a process of false memory and effect a change in the viewer’s affective state. In the case of the Tuscan Villa, the image of the villa no longer serves an indexical function vis a vis reality, but rather, it registers the trace of a shared cinematic memory. The kitsch experience of the villa and garden offers a momentary illusion or simulacrum of “realness”. For this, it sacrifices the passage of time and complexity of meaning.

The opposite of kitsch is ambiguity, contradiction, dis-ease, in a word, the real. What is the real? It is an aesthetic experience as much as any artwork, however, in the real, we have a body that exists in time and space. This body is the foundation for the development of human consciousness over millions of years of evolution. Both body and consciousness can be extended through technology, but, for most of us, reality becomes non-fungible (the NF in NFT) when we face death.

Dis-ease is meaningful, not just because of Covid. Because we feel it in ourselves, it takes us beyond our skin. Or, to quote the Russian Futurist poet, Vladimir Mayakovsky, who worked in both kitsch and art, “ I feel my ‘I’ is much too small for me. Stubbornly a body pushes out of me.”

Clement Greenberg talks about this action of going beyond self in both time and space as the difference between art and kitsch.

“But the ultimate values which the cultivated spectator derives from Picasso are derived at a second remove, as the result of reflection upon the immediate impression left by the plastic values. It is only then that the recognizable, the miraculous and the sympathetic enter. They are not immediately or externally present in Picasso’s painting, but must be projected into it by the spectator sensitive enough to react sufficiently to plastic qualities. They belong to the “reflected” effect. In Repin, on the other hand, the “reflected” effect has already been included in the picture, ready for the spectator’s unreflective enjoyment. Where Picasso paints cause, Repin paints effect.”

Here, the two ideas to understand are that art happens in a potential space. The work itself creates a Brechtian distancing effect which allows for a circulation of ideas and feeling between artwork and viewer. Art acts as a catalyst for something that is more than itself. The artwork is fundamentally this emergent entity, not the object itself. With kitsch, the effect/affect is held close to the work of art, it is contained in the work of art itself. The work of art is a dead end.

Benjamin writes about this change in the spatial relationship between art to the viewer. For him, the social basis for the “decay of the aura… rests on two circumstances, both of which are related to the increasing significance of the masses in contemporary life. Namely, the desire of contemporary masses to bring things ‘closer’ spatially and humanly, which is just as ardent as their bent toward overcoming the uniqueness of every reality by accepting its reproduction.” It is understandable, then, why NFT’s are kept in a wallet. What could be closer to the heart than that?

In the case of most NFTs and AI art the effect/affect contained is, in a word, literally one word, most often a word like “awesome,” or, even better, wordless—a heart emoji or thumbs up like. Disconnected to time or space, there is no way to navigate or orient. With NFTs and AI art, we are stuck in “wow” contemplation of the void. This is why kitsch is so easily and rapidly consumed. It is self-contained like a potato chip and fills you up for a moment, then leaves you feeling kind of yucky and empty. Kitsch is food for disembodied beings. Kitsch creates affect that is only gestured towards because it speaks to us in a state of disembodiment, disconnected both from lived time as well as myth. Kitsch will never invoke a state of awe, only ever a state of “cool”. The social milieu of such states is not that of shared ritual but of what Milan Kundera referred to as “the second tear.”

“Kitsch causes two tears to flow in quick succession. The first tear says: How nice to see children running on the grass! The second tear says: How nice to be moved, together with all mankind, by children running on the grass! It is the second tear that makes kitsch kitsch. The brotherhood of man on earth will be possible on a basis of kitsch.”

Kundera’s second tear is important because it links kitsch to social unity. Politically, kitsch and fascism are bedfellows. History offers countless examples of the bad taste favored by dictators. Thus, the kitsch of NFTs and AI is in keeping with the zeitgeist of fundamentalism and “traditional values” that we see playing out worldwide, but, it is also, related to the proliferation of images itself and the change in perception of value and “truth” that the mass reproduction of images has wrought over time.

Interestingly, Benjamin understood that the loss of the aura means that the function of art changes.

“But the instant the criterion of authenticity ceases to be applicable to artistic production, the total function of art is reversed. Instead of being based on ritual, it begins to be based on another practice—politics.”

Indeed, the rise of NFT’s depends on the emergence of decentralized cryptocurrencies which have the potential to radically change how many things are bought, sold, and circulated including that work of art we call “reality.”

Similarly, the use of AI to simulate the real through the creation of “deep fakes” suggests not only that it will soon be difficult to physically distinguish a real video or photo from a fake one, but that at a certain point, this will not matter since these images will have already done what they were made to do—produce affect.

Here, again, Benjamin is prescient, for he understands that the loss of the aura will change how we perceive and validate reality.

“to destroy its (an object’s) aura, is the mark of a perception whose ‘sense of the universal equality of things’ has increased to such a degree that it extracts it even from a unique object by means of reproduction. Thus is manifested in the field of perception what in the theoretical sphere is noticeable in the increasing importance of statistics. The adjustment of reality to the masses and of the masses to reality is a process of unlimited scope, as much for thinking as for perception.”

Interestingly, in a recent analysis, new media theorist Lev Manovich noted that for Discord Midjourney AI, “all the top terms have to do with image aesthetics and looks: “detailed,” “style,” “realistic,” “lighting,” “render,’ “4k,” “8k,” “octane” (name of 3D CG rendering engine), “cinematic.” And the users’ desire to generate very “photo-realistic” images is also clear. “ In other words, the “real” of the world is reduced to an aesthetic called “real” which inevitably will feedback, especially in virtual spaces, to manifest as reality, itself.

One can find strains of this aestheticization in the work of philosopher Alexandre Kojève. Kojève links the ascendency of a global capitalist order with the “end of history” which, “philosophically… would end in absolute knowledge displacing ideology. Artistically, the reconciled consciousness would express itself through abstract art – while pictorial and representational art captured cultural specifics, these specifics would have been effaced, leaving abstract aesthetic forms as the embodiment of universal and homogeneous consciousness.” For Kojeve, “Post-historical man will no longer be ‘man’ as we understand him, but will be ‘reanimalized’, such that the end of history marks the ‘definitive annihilation of Man properly so-called’. With all desires fulfilled, humans will become mere animals again, essentially organic entities, tending themselves like home gardens.

Georges Bataille who attended Kojeve’s lectures on Hegel found this negation of “man so-called” abhorrent and suggested that resistance might be found in living a life as an “open wound.”
 
“Personally, I can only decide in one way, being myself precisely this ‘unemployed negativity’… I imagine that my life – or, better yet, its aborting, the open wound that is my life – constitutes all by itself the refutation of Hegel’s closed system.”

As a tourist in the DPRK, an ever-present possibility of being thrown into a Pyongyang jail for some misstep complicated the canned and planned tourist experience, which was obviously fake. This contrast heightened the effect of the artifice, creating a dynamic space where I had to negotiate between the seduction of spectacle and the terror of embodied reality.

I think it is in this space that NFT art and AI art have the potential to become important new forms. A future essay will look at specific examples of AI and NFT art that address this critical juncture in human history. It is clear that a critical perspective on the procedural logic and ethical implications of these art forms is required. As art and life become more intertwined, how we choose to use technologies and what values we translate into digital realities will affect not only human well-being but all life on this planet. As Lev Manovich states in reference to AI art in his “Letter to a Young Artist,” “the only relevant thing is our limitations.”

FLY ANGEL SOUL Short Film — Volumetric Video and Virtual Cameras—The Future of Film?

Angel Cam Scene 3 FLY ANGEL SOUL

“Inspired by the words of Meister Eckhart, “let us pray to God that we be empty of God, and that we rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,’ ‘FLY ANGEL SOUL’ transports viewers into a world where spirituality, love, and technology converge.

The story unfolds as a devoutly Catholic mother, drawn by this enigmatic quotation in her estranged son’s diary, envisions an alternate past. In this vision, she is present when her son, Sebastian, receives his AIDS diagnosis at the height of the epidemic. Employing cutting-edge technologies, the film invites us to explore the tension between the constraints of the physical world and the human pursuit of something greater.

‘FLY ANGEL SOUL’ introduces us to a groundbreaking narrative approach through three distinct eponymous virtual cameras: fly, angel and soul, each offering a unique perspective. This isn’t traditional cinema; it’s a journey into the uncharted territory of computer-encoded viewpoints and the unscripted depths of the human heart. The ‘soul cam,’ masterfully embodied by Clémence Debaig, a choreographer, dancer, and creative technologist, who “plays” the role of Sebastian’s mother, acts as a cinematic avatar, allowing viewers to actively participate in the narrative., as she navigates a world where past and present intertwine. 

With FLY ANGEL SOUL, we utilize virtual production methods to radically alter the process of filmmaking. Here, liveness rests as much in how Debaig responded to the mis-en-scene (spatial sound, captured performance, architecture, lighting, and objects) as it changed in real-time as in the actors’ performance (captured prior to filming with volumetric video). This sense of liveness is augmented by Jarrah Gurrie’s dynamic editing. 

Soul Cam scene 2

Soul Cam perspective Scene 1

Incorporating an immersive first-person perspective and weaving in audio monologues drawn from Sebastian’s fragmented diary entries and those who shared his life, viewers are not just spectators but integral participants in the narrative. Together, yet maintaining our individual viewpoints, we delve into the themes of human suffering, love, and faith. Meticulously designed scenes guide us through a dynamic journey, shifting from expansive computer-generated landscapes to intricately hand-painted cathedral-like interiors. These transitions parallel Sebastian’s impending diagnosis and the consequent erosion of his freedom, mirroring our own progression from discovery to intimate engagement. 

Fly cam perspective Scene 1

In a pivotal moment, The Doctor challenges Sebastian’s self-condemning logic, setting the stage for a powerful transformation. As our story culminates, an initially imposing interior yields to the embrace of nature. The sun’s harsh glare softens into a starlit night sky, symbolizing both Sebastian’s realization that he, too, is worthy of love and the realization of his mother’s provocative assertion that her son has “returned to God.” 

Angel cam perspective Scene 1

Great Time at SIGGRAPH!

Here is my talk on XR and Theater. If you are interested in discussing further, please get in touch! I love thinking! SLIDES

SLIDE 1: I’m Illya Szilak. I’m a writer, artist, director and also, a medical doctor.

SLIDE 2 : I’m going to start with a little movie from a hundred years ago. This is really the beginning of film as an art form. In this film, which is only about 2 min long, you will see what looks like a cosmos, generated by Man Ray be sprinkling salt and pepper on film and it will end with a body, of sorts. Man Ray was a surrealist. To some extent, World War 1, change everything. Suddenly nothing made sense and human progress, those enlightenment ideals of reason, were shown to be if not utterly bankrupt, not easily achieved. So, Man Ray made this. As you watch this, I want you to think about who or what is the actor? How does “liveness”, the quality of being alive and having agency arise in this work? And, is it reason that allows you to make sense of what you see? What organizes and connects information in the scenes of the film? Is the film as a whole a kind of performer? These questions of performativity and liveness are ones to which we will return.

PLAY SLIDE 2

SLIDE 3

To begin this talk, I am going to start with some very basic assumptions that I hope you will agree with. First, is that storytelling is perhaps the quintessentially human way of organizing information. Second, technology allows us to access information that is otherwise inaccessible to us through our senses. Third, the repercussions of this are not entirely benign. • In a world where information is being created so quickly, as you see on this slide, most of it unstructured that is to say it’s not part of a predefined data set or model, the critical question is how do we choose what information to pay attention to and value? How much does technology itself determine what kinds of information we pay attention to and value?

In this talk, I’m going to delve into some philosophical and neuroscientific perspectives on this. I’ll argue that although all technologies from language on up have influenced how humans see and organize their world, the impact of computer technologies is unprecedented. I will argue that in this Anthropocene age, we must find a way to relate to each other and to the planet differently if we are to survive and thrive as technologically extended humans. Theater is a way of playing out, rehearsing ways of being in the world and also seeing ourselves through the technologies that we use. XR Theater is especially powerful as it allows us to inhabit two positions, self and other at once. It allows us to play with new forms of liveness, and in a lived, as opposed to purely theoretical way,  explore what we machine extended beings should value. Do we value the body? Do we value gravity and work (in all its meanings). Which ways of perceiving time and space do we privilege? How do we represent ourselves? How do we relate to, represent, and acknowledge (or not) gender, race, age, species, ability in machine extended environments.

To start, I’m going to take you on a short foray into how philosophers of the machine age have thought about how we organize information. 

SLIDE 4: BERGSON
We won’t start too far back. Let’s start when media technology really started to take off telephone, photography, phonograph, film. The year is 1894. This is when French philosopher Henri Bergson published an amazing book which all of you should read. In Matter and Memory, he suggests that the universe is made up of an infinite number of images: sound images, thought images, visual images, etc. and the one’s that we pay attention to are ones related to two things: our bodies and memory. 

SLIDE 5: UXEKILL
Jump ahead thirty years and we have Jacob von Uxekill publishes a little book in which he creates a thought experiment imagining how the world to different species including the deaf and blind tick which perceives the world in terms of and responds to levels of butyric acid radiating off of a mammal’s skin. These worlds he referred to as “Umvelt” a lived-world of experience.

SLIDE 6: MCLUHAN
Now jump to 1967 where Marshall McLuhan coins the phrase “the medium is the message,” arguing that content per se could not be separated from the medium. Media acts as a kind of sense organ for humans, how we now construct our “Umwelt” is not simply through our native senses but through all our media extended senses. One might even argue that with most of us living through our devices much of the time, the human “Umwelt” has undergone an apocalyptic change.

SLIDE 6: VARELA
A century after Bergson, Francisco Varela and his colleagues published this book which created a revolution in thinking about how humans process information. They argued that the idea that human reason3Zxq ZXCGVHJIL;’’DF could some how be separated from the body, an idea which really took hold during the Enlightenment and became valorized with the advent of computing machines, was just plain wrong. We don’t just passively receive information from out there and translate these into abstract symbols which we then translate into actions. We enact our world.

SLIDE 9

Moving on to 2000, philosopher Villem Flusser argues for something even more provocative, technology , he suggests, doesn’t just alter the message, as McLuhan suggests, it alters the way human consciousness works, the way we organize information.  The ‘apparatus’ — the camera is a tool has its own ‘program,” one which supercedes human control. Photographers essentially are playing a kind of game according to the camera’s preprogrammed rules. It is a menu of choices that arise from the apparatus itself and not from within the human.

SLIDE 10:  The issue is that the images no longer help us orient ourselves into the world because their basis is an exploration of the program and potential of the apparatus, not of the lived world itself. Covid was the straw that broke all our backs, because it showed us that amidst that chaos of images, there is one that can not be captured. That is death. This is the fact: all of of us will get sick and die. Covid showed us this and it scared the shit out of us. Because we could no longer be distracted from the one reality that is beyond the reach of our technologies. Given that our computer technologies do not privilege embodiment but rather embodiment as captured and experience through image, one that removed from the evanescent now and placed into a kind of undead forever now, the question we were starkly left with is how does matter matter?

The problem of the apparatus is that human cognition is not just an abstraction, it is not an array of menu choices, it is a lived, embodied process that has evolved over millions of years. The question is whether technology is impoverishing human consciousness or enriching it. Yes, there is more information, but perhaps because of our estrangement from our lived worlds, worlds not translated through the menu of an apparatus, that only leaves us in chaos?

SLIDE 11:
It does leave us in chaos. Let’s go back to 1951. The Macy Conferences where information theory was born. What is information? On one side we had Claude Shannon, brilliant mathematician and computer scientist. He wanted to find a definition for information that would be quantifiable and something machines could understand.

The best way to illustrate Shannon’s definition of information is that the flip of a double headed quarter contains no information because there is only one possibility. Physicist Donald Mackay disagreed. He argued that meaning acts as an organizing function for a perceiving subject, it had an impact, it was not merely related to the number of bytes.

SLIDE 12
But, Shannon’s formulation, the efficiency of the code, the losslessness of transmission won out over MacKay’s meaning-making. As N. Kathryn Hayles points out, Shannon’s definition of information is essentially a disembodied form of information. So, what we are left with is a situation where human cognition is embodied, but our Umwelt as constructed by our technologies is not. And, We have no idea what this means.

But, consider for a moment what it might mean in terms of how we relate to our planet and our fellow humans and other sentient beings. I don’t have to put up graphs about the rates of opiate overdose, depression, suicide, mass shootings, species annihilation, ecosystem destruction to make a case for why we need to figure out new ways of being in the world and with each other and fast. The problem is that human beings have trouble with chaos, it makes us really nervous. So, as the totality of information increases and the probability of any one answer decreases, well, we got trouble. Plato kicked the lyric poets out of the polis for a reason. Fascism, religious fundamentalism, and, weapons of mass destruction or just down-home schoolyard gun massacres well these are all understandable responses.

Although this information explosion and concomitant disorientation, which honestly, is this close to insanity can be and will be and is being placated for a time through the dispersal of this tension through the pure act of repetition and compulsion (our swipes, our constant checking of emails and circulation of memes and data), our fear of annihilation by distributing ourselves in multiple identities across virtual spaces, a withdrawal from attempts at complex meaning-making to Tweet size bites, 10 second TikTok videos and Pixar like bright and shiny emotions. Even so, the center will not hold forever. Something is going to break.

SLIDE 13
There is no going back, return to the days of unextended embodiment. The horrible wonderful fact is that our ability to extend ourselves in space-time through machines has brought us to a critical moment where the death drive and Eros meet. Metamorphosis is to acquire a different border between self/other, a different form, different senses, different ways of relating to , privileging and processing information and our relationship to other living beings and to our home, this planet.As marabou suggests, Metamorphosis is what occurs when fleeing is necessary but impossible. And, it is what we need now.

SLIDE 14
Apologies to Elon Musk, your flight to Mars, yes, I understand it on a human level of terror and imminent annihilation, but, sweetheart, it ain’t going to work. And, by the way Twitter or X or whatever, probably not helping things… The bottom line is we must find a way to relate to each other and to the planet differently, because not everyone is gonna be on that spaceship.

SLIDE 15 HARAWAY
As such, it is critical that artists stake a claim and use new technologies to interrogate the relationship between brutal historical, political and material realities and the pernicious “correction” of these facts, not only through omission e.g. Facebook Horizons, but also, and perhaps more importantly through “neutral” software based regulation of identity, behavior, interaction and access to and production of information in digital spaces. As Haraway suggests: the relation between organism and machine has been a border war. The stakes in the border war have been the territories of production, reproduction, and imagination. And, I would suggest, survival.

Where will this border war take place except in our digital spaces? One could attempt to hack dominant technological systems. It has been done. But, it’s probably unlikely that you will bring Twitter or X or whatever down for long, the question then becomes one of finding ways to be, interact and live differently inside those dominant technologies, that is to say, we must begin to play with and in them. We must not merely be users, but players.

SLIDE

So, that brings us to theater. In 1924, Italian Futurist F.T. Marinetti comes up with the idea for “total theater.” According to scholar Pierpaolo Antonello, “Marinetti saw in the theatre one domain where art could intervene to radically alter the cognitive and perceptive experience of the spectator.” Marinetti’s productions were not script-based, but multi-sensorial and performative. Among the senses, Marinetti privileged the sense of touch. 

Marinetti’s Manifesto of Tactilism published in that year, is certainly worth a read in its own right, it is hyperbolic and poetic, but especially fascinating for me is this last paragraph which posits the possibility that there could be other senses, other kinds of cognitive inputs, beyond language and our five senses. I think what Marinetti is actually getting at is the possibility that a work of art might be able to elicit these new ways of experiencing the world. In other words, an artwork might enable us to overcome the conventional limits of how we perceive the world, not merely conceptually, but also materially as a lived experience.  

SLIDE 17
Marinetti anticipated the possibility of total theater a century before that technology became widely available (thanks Apple!). So why not just immersive theater as Marinetti imagined it? Why XR. The power of XR with regards to total theater has to do with two factors. First is the ability of XR especially VR to generate a remarkable sense of presence—That sensation of being here and now in time and space that seems unmediated. There are many theories of presence in XR but the one that I like from Riva and colleagues draws upon the work of
neuroscientist Antonio Damasio. Riva suggests that presence is a primordial form of consciousness, one that helps animals distinguish self from other, and internal from external stimuli. Interactivity especially action upon the world is critical to facilitating this. Through interactivity, an organism creates a motor-map of the universe. This allows it to plan future actions in the light of past knowledge.

SLIDE 18
This slide outlines Damasio’s description of the creation of the self, based on this, Riva describes three forms of presence. Proto presence is an organism’s ability to couple perceptions and movements and thus differentiate itself from the external world. Core presence is an organism’s ability to focus sensorial experience on present tasks and relevant objects in the environment. Extended presence is an organism’s ability to recognize the significance of previous experiences as they relate to the self. So in this last personal, historical, aesthetic and cultural memory play an important role.

So, if presence and self creation are linked, it’s clear that the power of XR for playing with new versions of being and being in the world is that the self of the audience occupies two positions in space and time. This duality may be formally acknowledged through the use of an avatar, or may be inferred by the visitor through the architecting of two or more space-times within the XR experience.

SLIDE 19
In XR, meaning-making in XR theater occurs in the oscillation between two registers within the audience/participant. This relates to the dual nature of self as embodied self and as avatar/other that XR encourages.On the one hand there is the sensual self the enacts an environment from the five senses, on the other hand there is a symbolic, representational self. Clearly the former is concerned with affect generated by action, the latter is concerned with how the self is represented which can be recognized on a cultural, historical and aesthetic level.
In this way, XR theater offers the possibility of a new kind of storytelling, an active, enacted kind of storytelling—This language relies far less on symbols and signs, and more on code that elicits both the muscle memory of conventionally embodied sensations and the emergence of new ones. In such a language, affect and narrative results procedurally, through form and process even more than through content. The new form of being elicited in XR theater oscillates between self and other, material and virtual body, life and art, audience and performer. XR Theater is a laboratory for this metamorphosis.

SLIDE 20
In this slide I want to point out some of the dramatic tensions which operate generally in XR theater. These do not necessarily take the place of conventional elements of dramatic tension such as opposing characters, and dramatic action, but they are important to consider. I don’t have time to go through all these and I’m sure there are others, but I’d like to highlight a couple of these.

The first is thinking about the relationship between embodied material reality and the transcendent body interrelate. Two other that I would like to highlight is the relationship between the phenomenologically situated body and the character or symbolic body most commonly this might be experienced as an avatar. Now, not only do people project bodily action onto an avatar, such that the action of the avatar body overrides the orientation of the physical body, but how that body is represented becomes important. To represent an avatar body is anime form and using volumetric video are not the same. There are situations when the affordances of one are more useful. I’d also like to point out that aesthetics become a hallmark of authenticity in these worlds where everything is unreal. This is why art matters. Now, perhaps, more than it ever has. Technical images can not be judged as true or false but only as art or kitsch. Unfortunately, but not surprisingly, the tech industry privileges hyperrealism and Pixar seductiveness above all other aesthetics and emotional registers. AND< we love it! Of course we do, who could blame us? Like Elon Musk, we are fucking terrified. Kitsch is mother’s milk. In its embrace we can distance ourselves from chaos through irony or come together so we can all cry Milan Kundera’s second tear together. The memory of the “real”, preserved as codes borrowed from shared cultural experience and the memory of the body, preserved as Kitsch aesthetic and genre and gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms 
might be revealed and uncovered with the aid of human-computer interfaces 
remains to be seen. 

SLIDE 21

Because, the point of XR theater is not to reconstitute traditional theater in the digital space, but to utilize the the affordances of technology to allow the art form to evolve in a way that speaks to and through machine extended bodies.. XR theater may not look like conventional theater. In fact, not only does it take its inspiration from multiple arts involved with spatial construction and movement, it may, take on the form of any of these.

SLIDE 22
Understand that in XR theater, the environment, the objects in that environment, and the virtual or other humans in the play all perform. This is not new. We saw this in Man Ray’s film, that nails can “dance.” Is this liveness? No, it is a simulacrum of liveness which, as we know from the concept of aesthetic sympathy, the original meaning of empathy, nevertheless constitutes a way for us, the audience, to experience our own liveness, our own bodies dancing like nails. The word empathy which psychologist Edward Titchener coined in the early 1900’s, did not mean what it means today, rather it was seen as a form of aesthetic sympathy, one which allowed audiences to appreciate the the abstract movements of modern dance and painting which were just emerging. Here is a beautiful evocation of that empathy.

SLIDE 23 LIVENESS

Since, in XR both environment and objects and virtual humans are responsive and respond, what constitutes liveness in XR theater? I do not have time to go into the debate represented in these quotes, but my own take is that our experience of “liveness” manifests on many levels, some more familiar and proximate than others. It is certainly not limited to conventional live performance. As long as the audience themselves is conscious, that is what is required. A live audience member brings with them a sense of self and a sense of being changed or wanting to be changed or touched in some way by the experience. So without getting into a debate on the nature of consciousness, I’d say that liveness brought to the work of theater by the audience relates to her awareness, her sense of self/other and to desire. Her consciousness is the spark that energizes the whole machine bringing it to life. In this way, the work of art itself is a kind of cyborg entity with manifestation of that liveness operating on multiple registers.

SLIDE 23
XR theater should explore new forms of “materiality” which I list here. To illustrate some of these concepts, I am going to dive into our newest project FLY ANGEL SOUL supported by the Jerome Foundation. Fly Angel Soul is a short experimental narrative film shot within virtual reality. It tells the story of Sebastian, a young gay physician estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart “(let us) rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” Fly Angel Soul is shot in real-time, from the unique points of view of three networked virtual cameras adopting the “roles” of the eponymous characters. The “human” p.o.v. will be that of a live cinematographer moving through the virtual set. Thus, in Fly Angel Soul, “liveness” resides in the “embodied” cameras even more so than in the actors in the story. Finding commonality with video games and live performance, Fly Angel Soul explores the potential for virtual production techniques to expand 2-D cinematic language. The human camera viewpoint which predominates in this first scene, what we affectionately refer to as the soul cam, is virtual as are the angel cam and the fly cam. Created by an amazing developer and Unity engineer named Elliott Mitchell. But, unlike the angel and the fly cam, an actual human, dancer, choreographer and creative technologist Clemence DeBaig created the path that the soul camera follows through the virtual mis-en-scene in Cinemachine.

SLIDE 24
Our goal here was to use our experience and skills honed in creating interactive stories and installations to create a film which put the film audience in a position of choice. Whether it is conscious or not, each viewer will have a preference for one or other of these perspectives. We have already found this to be the case among the film’s own team members. This points out not only the agency of the viewer, but also their responsibility. Since the world can be viewed in multiple ways, something both art and technology show us, then, how we choose to view it and how we choose to tell a story, what information and perspectives we privilege depend on what we value. There is no singular truth. The old gravitational centers of God and tradition are gone or fading fast, this is beauty and terror wrought by our technologies. But, amidst all this, there is one truth that few of us can see differently, one which Covid made us acutely remember, we are all of us mortal. We will all get sick or injured and die. This is what this story is about, the imperative to find new ways of relating to ourselves and each other. I have an MD after my name and I still practice medicine actually at Rikers Island in New York City, so my relationship to mortality and embodiment is perhaps less abstract than most people in tech. It was of critical interest to us that we find a way to use technology to make the viewer feel as though they were really intimately present in the scenes.

SLIDE 25
There were cinematic inspirations for this especially the long tracking shots in Max Ophuls films and Birdman. And, the use of gesture and architecture in Maya Deren’s films.

PLAY SIDE 26
I’m going to play a little bit from Birdman. Notice the architecture, notice the multiple rhythms of movement including sound, camera movement and actor movement, how this unites everything into an incredibly immersive experience.

SLIDE 27
First I want to let you know that I have never actually made a film before, so although I know we completely upended the usual workflows for filmmaking, it didn’t actually seem weird to us. Once I had a script, we began to work on what I call a responsive mis-en-scene: this was a responsive environment including architecture, spatial sound, and movement based triggers. To facilitate our collective imaginings, my artistic partner Cyril Tsiboulski, created a quick mock up in VR Chat from the initial architectural sketches that artist/architect Paolo Barlascini had come up with. We actually met there with our actors and rehearsed. Here, some of us are in VR Chat.

PLAY video

SLIDE 28

Aesthetically, we were interested in finding a sweet spot between the real and the transcendent, between intimacy and overt theatricality. For instance, the doctor’s clinic is an impossible structure. On the outside, it references Malian mud architecture, on the inside it references the radial architecture of the original plan for St. Paul’s Basilica in Rome. This is obviously possible only in VR. We could have created everything virtually, but we really wanted the hand of the artist to be present in the form of mark making, a trace of the artist’s performance. So Paolo painted and drew and then scanned these and working with developer Emmy Yupa, these paintings and drawings became the skins for the walls of the interior of the clinic. In addition, we worked with the sound designers so that the soundtrack could be responsive to the movement of the human camera and robust enough to sound great each time. Because, this film can be reshot and reshot. Lastly, we worked with Clem on movements for the actors. Mostly the second scene is literally waiting. The dialogue is created in units with breaks and movement sequences that can be triggered when the soul cam moves away. So again, we are creating a tension between intimacy and theatricality.

SLIDE 29

This intimacy, the liveness of the film comes not from the actors performances, these are essentially artifacts of a performance. They were filmed at the Microsoft mixed reality studio in San Francisco (unfortunately, now closed) and are essentially game objects. We used volumetric as opposed to Mocap or animation because we really think that volumetric not only captures the nuances of human movement, but it registers as a trace of the living, the real, it feels like an authentic memory of something that happened. But, the liveness of the film comes from the living camera and the misenscene that responds in real time to her presence.

SLIDE 30
This May we shot the soul cam path at the NYU Tandon Mixed Reality Lab. The first day was taken up by hardware malfunctions. The second day was revelatory. We literally had no idea whether this crazy project we were working on would work. But, after allowing for a discovery phase, I asked Clem to pretend that she was Sebastian’s estranged mother who finds herself in a dream. How would she move through the landscape, how would she move in relation to him. It was a revelation. I went back and forth between watching Clem moving in real time in headset to watching her move virtually through the misenscene. It was magic.

SLIDE 31
I’m going to play a little of this film for you, it is still a draft.

PLAY FILM

I hope it shows you that XR theater can take many forms. Our goal after this is to find funding for what we call real-time filmmaking. This would combine live theatrical performance and live-streaming of FLY ANGEL SOUL made there on the spot. If you miss it, the film never existed. Filmmaking itself becomes performance.

In Post-Production on FLY ANGEL SOUL

Using cutting-edge virtual production methods, “FLY ANGEL SOUL” invites viewers on a mother’s surreal journey through an alternate past, where she confronts her son’s AIDS-related death. Guided by his diary and memories, she discovers the transformative power of love, faith, and art to transcend tragedy.

“Soul” cam still

“LET US PRAY TO GOD THAT WE BE EMPTY OF GOD…”
Drawn in by a Meister Eckhart quotation from her son’s diary, the mother envisions an alternate past where her son is alive and she is present when he receives his AIDS diagnosis. The film delves into the tension between the constraints of the physical world and the human yearning for something greater.


“FLY ANGEL SOUL” employs innovative storytelling techniques using three virtual cameras, each representing a distinct perspective: the Angel, the fly, and the soul. Unlike traditional cinema, the film doesn’t focus on “getting the shot.” Instead, it embraces the unexpected aesthetics of computer-encoded viewpoints and the intricate emotions of the human heart. The perspective from the “soul cam” is crafted through the creative path of Clémence Debaig, a choreographer, dancer, and creative technologist who embodies the role of Sebastian’s mother.


Through this first-person shooter perspective, viewers are intimately connected to the characters, immersing themselves in their emotions and experiences. As the mother navigates this alternate reality, the audience becomes an integral part of her journey, sharing in her heartache, hope, and exploring the boundaries between reality and the extraordinary.


Angel Cam view

“FLY ANGEL SOUL” explores the possibilities for virtual production methods to reshape film language. Our challenge lies in empowering viewer agency and narrative-shaping responsibility and invoking a heightened sense of presence within the constraints of 2D film. To tackle this, our film production has taken on the guise of a functional laboratory, where diverse talents including visual artists, architects, developers, cinematographers, choreographers, writers, directors, sound designers, and composers converge.Through collaborative experimentation, we strive to channel our expertise in interactive storytelling and installation art, envisioning how cinema can redefine perceptual limits and challenge established concepts of reality.

Fly Cam view (Work in Progress)

Shout out to the Incredible team! And my longtime creative partner Cyril Tsiboulski.


Michael Debartolo Sebastian
Ingrid Jean-Baptiste Bathilde
Tommie J. Moore The Doctor


Clemence Debaig Choreographer & Creative Technologist
Christoph Mateka Composer
Leo Kuraitė Supervising Sound Designer
Paolo Barlascini Lead Visual Artist
Produced by
Illya Szilak & Cyril Tsiboulski Executive Producers
Allen Yee Associate Producer
BTL
Volumetric Production
Jason Waskey Creative Director & Technical Director
Kathy Saelee Producer
Tiago Washburn Capture Tech
Tri Le Capture Coordinator
Rhonda Callahan Technical Artist
Brian Cantwell Processing Supervisor
Sean Bittinger Technical Artist (Data)
Eric Limcaoco Stage Sound Engineer
Heather Manchester Hair & Make-Up
On Set Medics On-Set Covid Compliance Officer
Slipery Fish Catering and Events Catering
2D Production TBD
Cory Allen
Peter Bell Behind the scenes reel production
Immersive Development
Elliott Mitchell Lead Unity Engineer
Emmy Yupa Lead Graphics Engineer
Post-Production Sound Services (TBD)
Post-Production Design Services (TBD)
Legal Services
John Pelosi, Esq. PELOSI WOLF EFFRON & SPATES / Fancy Rainbow Counsel
Filmed on location at NYU Tandon @ the Yard.
Special Thanks Todd Bryant
Christopher Strawley
Derek Chan

This is Not a Person Part 2 at SXSW today Sunday March 12 at 4PM Fairmont Hotel Manchester B

I’m speaking on what it means to be a person in this machine extended time with incredibly smart and insightful fellow artist/thinkers: Scarlett Kim (Oregon Shakespeare Festival), Kiira Benzing (Double Eye), Jason Waskey. I’ll be discussing our short film FLY ANGEL SOUL (planning to shoot it with the wonderful Clemence Debaig in May).

It will be a free-wheeling and surprising discussion. No boring slide show, just big and outrageous thoughts from our years of combined experience creating works that explore “liveness” and performativity in novel ways.

For those of you who like slides,,, here are some of the things I’ll be getting into… Happy to discuss these ideas with anyone interested.

VR and New Forms of Matter (ing)

POP-UP at OREGON SHAKESPEARE FESTIVAL FEATURING RAJA FEATHER KELLY’S ORDINARY GESTURE VR EXPERIENCE

Last year we had the honor and wonder of collaborating with Raja Feather Kelly (choreographer for the 2022 Tony Award winning musical The Strange Loop ) on his first VR work. If you are in Ashland for OSF, June 29 – July 14, 2022, stop by the Black Swan Theater and check it out.

https://www.osfashland.org/en/productions/activities-and-events/quills-fest/ordinary-gesture.aspx

Revisiting my Essay Storytelling at the End of the World: Cinema and Narrativity in VR or Why Matters of Embodiment Matter in Digital Media

Cover for the original peer reviewed conference essays

I published this as part of a peer reviewed conference on Forms of Apocalypse at University Paris 8 in 2017. It holds up and speaks to why considerations of embodiment are critical in XR. Also, warning: Heidegger ahead..


The word “apocalypse” comes from the Greek apokaluptein meaning to uncover or reveal. In keeping with the word’s origins, I approach the concept of apocalypse not as the destruction of the material world, per se, but, epistemologically, as the end of the world as we have known it. If not the primary agent of this epistemic cataclysm, computer technology, which increasingly mediates human perception of reality is increasingly reflective of it. Unsurprisingly, these technologies have also dramatically changed how we communicate about the world. In his essay, “Database as a Symbolic Form.” new media theorist Lev Manovich describes this as a move from “narrative” to “database” storytelling.

As a cultural form, the database represents the world as a list of items and it refuses to order this list. In contrast, a narrative creates a cause-and-effect trajectory of seemingly unordered items (events). Therefore, database and narrative are natural enemies. Competing for the same territory of human culture, each claims an exclusive right to make meaning out of the world. (Manovich, 1999 “Database as a Symbolic Form”)
Manovich concludes his essay by praising the Soviet film director, Dziga Vertov whose Man With a Movie Camera, he argues, successfully creates a new kind of narrative by ordering a database of images around the “kino-eye” of the filmmaker. This process of discovery is film’s main narrative and it is told through a catalog of discoveries being made. Thus, in the hands of Vertov, a database, this normally static and “objective” form, becomes dynamic and subjective. More importantly, Vertov is able to achieve something which new media designers still have to learn — how to merge database and narrative into a new form (Manovich, 1999, “Database as a Symbolic Form”).


Whereas, even today, both the novel and cinema remain mostly on the side of conventional narrative, many new media forms resist the linear arrangement of information. Of these, virtual reality (VR) most directly and completely mediates visual sense perception. In the VR headset, the world disappears, replaced by an alternative reality generated by a computer. Here, the user is no longer a passive consumer of someone else’s database. By exploring and interacting with the environment, she orients herself according to her own interests. One might even go so far as to say that she becomes a kind of film director framing close-ups and long-shots.
Philosopher Vilém Flusser has characterized the time we live in as an epistemic crisis brought about by the recognition that, “scientific research is not the gesture of a transcendent intellect.” What Flusser calls “the gesture of searching,” for meaning or knowledge, has shifted from “a digging down for reasons” to an examination of the aesthetic and relational qualities of things and an exploration of how humans respond and attend to their environments as subjects and as objects (Flusser, 2014, “The Gesture of Searching”). Perhaps unsurprisingly, Flusser’s description of this gesture serves as an apt description of being in virtual reality, a reality which has become, in essence, a database […] the researcher is embedded in an environment that interests (matters to) him, both at close range and at a distance. There are aspects of the environment that interest him intensely and others that hardly touch him. The more an aspect of the environment interests the researcher, the more ‘real’ it is for him (Flusser, 2014, “The Gesture of Searching”).


In this essay, I will build upon Manovich’s observation that, “cinema language, which was originally an interface to narrative taking place in 3- D space, is now becoming an interface to all types of computer data and media” (Manovich, 2001, 326). I will argue that whatever the language of VR turns out to be, it will rely far less on symbols and signs (content) and more on a procedural code (form) that elicits both the memory of the body’s movements and affects as well as a shared cultural memory, not of the lived world, but of cinema, itself.


What determines the authenticity or veracity of this kind of storytelling? Here, it is useful to consider one of the earliest, most widely distributed experiences in VR. The Tuscan Villa (https://youtu.be/wuFTd5TVhHw), was provided free with the developer kit for the Oculus Rift DK2 head mounted display. It offers users the chance to meander through a country villa and surrounding landscape. A generic looking computer-generated farm house, its stones visually coded as “old,” suffices to invoke a nostalgic memory—one that will be even more poignant and believable to a viewer who has never actually stayed in a Tuscan villa. In other words, the Tuscan villa is meant to exist outside of history. In this work, the visual and auditory codes are perceived as “authentic” if they successfully activate a process of false memory and effect a change in the viewer’s affective state. Significantly, the experience does not provide a traditional narrative. Rather, it offers the vague sense of remembering a dream or retrieving something that has been lost which never actually existed.

Kitsch is nothing if not a suspended memory whose elusiveness is made ever more keen by its extreme iconicity […] kitsch is not an active commodity naively infused with a wish image, but rather a failed commodity […] a virtual image, existing in the impossibility of fully being. Here, for a second or even a few minutes, there reigns an illusion of completeness, a universe devoid of past and future […]. (Olalquiaga, 1998, 28).

In The Language of New Media, Manovich points out that as cinema becomes more and more reliant on software manipulation in postproduction, it functions less as “a record of perception,” and more like a computer screen— “as a record of memory” (Manovich, 2001, 325). In the case of the Tuscan Villa, cinema no longer serves an indexical function vis a vis reality, but rather, it registers the trace of a shared cinematic memory. The kitsch experience of the villa and garden offers a momentary illusion or simulacrum of “realness”. For this, it sacrifices the passage of time and complexity of meaning.

Flusser suggests that gesture, by which he means the “reality” of affective states made manifest through aesthetics, should be judged in terms of art or kitsch rather than whether the gesture is true or false. “The scale of values we use to evaluate (gesture) may not oscillate between truth and error or between truth and lies but must move between truth (authenticity) and kitsch (Flusser, 2014, The Gesture of Searching).

It is not artifice in itself that renders the Tuscan villa demo artless. To understand this, it is instructive to compare the Oculus demo to Maria Menken’s decidedly un-kitsch 1957 film Glimpse of the Garden (https://youtu.be/_fGg7D1naIs?t=2m10s). Even though both The Tuscan Villa demo and Menken’s film privilege surface effects over depth of meaning, the result is decidedly different. In her film, movement of the camera, a unifying looped soundtrack of birdsong, and color filters take precedence over content, which consists mostly of shots of cultivated nature. Despite the amateurish quality of the images, the result is strangely magical. Menken’s editing, which juxtaposes different scales, points of view, and speeds of movement, resists the Tuscan villa illusion of completeness. Thus, it is due to the authenticity of the filmmaking gesture itself, not its content, that Menken’s film can be called art whereas the Oculus demo remains kitsch.
If there is any redeeming gesture in the Villa, it is in its ability to invoke an uncanny sense of bodily presence, a being there in a digital world that is generated by the user’s wandering or, in Flusser’s terminology, “searching” through the environment. According to neuroscientists, presence is a primordial mechanism by which organisms establish self and other, create a motor map of the universe and use memory to plan actions and interactions. In so far as the Tuscan Villa succeeds in creating this sense of presence, it sets up a queer dynamic in the user—an oscillation between interior and exterior, self and other, the “real” and the virtual. In this way it, perhaps inadvertently, succeeds in making the familiar strange. In other words, the odd and unresolvable experience of being in two places at once creates a kind of digital estrangement conceptually related to Bertolt Brecht’s Verfremdungseffekt, distancing effect, and Russian Formalist critic Viktor Shklovsky’s “making strange” (Russian: приём остранения priyom ostraneniya) whereby certain devices are used to disrupt the audience’s easy identification with an artwork.


To see how this might be productively used in VR, I turn to filmmaker Karen Cytter’s work for a contemporary example of estrangement. Cytter employs familiar, even kitsch codes of cinema, especially melodrama. But, rather than offer an illusion of completeness, she uses what Thomas LaMarre refers to as “internal montage” (LaMarre, 2009,125). LaMarre suggests that computer graphic imagery and digital effects have created a new emphasis on the animation technique of compositing —the creation of an image composed of multiple layers. If an artist suppresses the space between layers to create an illusion of wholeness, the result is either a unification in depth, a hyper- Cartesianism with a well defined vanishing point (e.g. Pixar type animations), or a flattening of all layers into a single “superflat” plane in which multiple frames of reference and sight lines coexist. Cytter’s films demonstrate a remarkable compositing of space, dramatic events, and sound. It is effectively a jumbled cinematic database in which voice, visual form, emotion, and plot nearly float free of each other on the level of meaning. Surface effects are at least as important as content here. Through a process of making the familiar strange, she astutely reveals the underlying game of seduction that takes place in cinema. In Cytter’s work we are left with empty codes, empty signs which nevertheless incite emotional responses. Devoid of linear narrative, in Rose Garden (https://vimeo.com/87553434), the mournful sound of the flute is not only a wry reference to Sergio Leone’s Once Upon A Time in the West, it codes the psychological tenor of the piece. Although there are no clear “characters,” as with avatars in video games, the human body remains the basis for emotional engagement. For instance, at the end of the film, when a boy is shot in the back, though we know nothing about him as a character, the violent collapse of a child’s body is still shocking and emotionally wrenching.


Rose Garden’s flattening out of time and space, and its move from depth to surface, which Flusser relates to the gesture of searching, resonates with Hiroki Azuma’s description of database narrativity. In his book, Otaku: Japan’s Database Animals, he notes a move from “grand narratives” to “grand non-narratives” or database (Azuma, 2009, 55). This is associated with the proliferation of small narratives the purpose of which is not to refer back to some greater or universal theme, but to invoke strong feeling or fulfill consumer preferences. Whereas, in the modernist era, personal fictions could be related in depth to a grand story about the nature or meaning of human life, now, there is a lateral move in which multiple small stories are spawned from the same database as in Cytter’s film. Likewise, in virtual reality, narrative unfolds as movements through space. However, distance is no longer measured in terms of meters or any other universal objective standard, but, as both Flusser and Azuma suggest, in terms of the user’s desires and interests interacting with the environment.

In Mark Amerika’s cell phone film Immobilité, one of the characters remarks that the world is disappearing before her eyes. Indeed, the opening scenes, a shaky, looped walk down a tree-canopied path, reveal just how the world will end— not in some fiery apocalypse, but rather by our recognition that the world was only ever virtual. In Immobilité, what remains of embodied experience, what Amerika calls “muscle memory,” finds its expression not in meaning, but in form. In effect, Immobilité — a film that remixes some of Flusser’s thought into the subtitle track — consists not so much as a series of images, but as a series of film gestures.


In Immobilité, the quality of the image is sacrificed to movement of the camera just as content is sacrificed to surface effects. Depth, in the form of philosophic musings, is separated from the images and brought to the surface as subtitle. What remains on the screen are vibrations, rhythms, and frequencies that resonate with or cancel each other out. In fact, in many parts of the film, the camera is decidedly unsteady or even appears to vibrate thereby creating a surface that destroys a Cartesian perspective and the static perceiving subject. “And then I would begin to lose myself, to play uncontrollably, becoming something like music,“ the unreliable narrator texts us from the beyond.

Heidegger’s concept of “profound boredom” operates here. This is state in which all beings “recede” into indifference and the one who is bored comes face to face with the experience of time itself (Heidegger, 2001, 80). Heidegger describes profound boredom as the ground for all the multifarious variations of being. Profound boredom leads us to suspect that reality is neither static (immobile) nor completely knowable. Concomitant with this loss of a fixed self, language fails to signify. Rather, it functions procedurally to carve up pure duration into pieces: life and death, existence and nonexistence, chaos and order. This resonates with Henri Bergson’s suggestion in Matter and Memory that homogeneous and universal perceptions of space and time are a trick of the mind, a refraction of pure duration into space. For Bergson, space and time do not exist anterior to perception and action. We do not act through space and time, rather we create these symbolic diagrams because we need to “divide the continuous, fix the becoming, and provide our activity with points to which it can be applied (Bergson, 1990. 212). Bergson suggests that our knowledge of the world is delimited by the concerns of the body and its possible actions and, perhaps, even more importantly, by our memories. If that is true, we are like Plato’s cave dwellers: always living in a virtual reality of our own making.


The key dynamic in Amerika’s film, one that will likely dominate in this time of apocalyptic narrative, is a tension between overt aesthetics and conventional notions of truth. Although Immobilité’s low production values and distinct lack of artful montage lend the film a “real-life” documentary feel, Amerika’s overt use of glitch, repetition, and jerky camera movement insists upon a kind of self-conscious performativity or aesthetics. When the narrator asks, “Was I authentic?” — the question is equally what does “authentic” mean if everything is virtual and, ultimately, data, and, furthermore, who is “I”? If the world is disappearing, so is the singular author, that figure of the auteur which haunts French New Wave cinema, to which Amerika pays homage in his use of subtitles. In Immobilité’, Amerika, like Flusser, suggests that the question of “authenticity” lies not in a “true” identity but in the aesthetic value of the gesture itself.


Antonioni’s masterwork, Zabriskie Point, also operates by disavowing conventional notions of authenticity. In Zabriskie Point all is surface: from the setting, an almost abstract stretch of ancient desert, to the way images are composed in multiple layers without depth. There is also a strange flatness to the main characters. They seem to have no ambitions and no particular future. With exquisite conciseness, Antonioni destroys the epic march of history with the first line spoken by the male protagonist: “I’m willing to die (for the revolution), but not out of boredom.” For the characters in Zabriskie Point, time operates as it does in profound boredom as a pure duration.
Even on a personal level, events have no apparent consequence. Though one of the young lovers dies violently, there seems to be little emotional repercussion for the lover that remains. This flatness of character is matched by the physical beauty of the actors. It does not matter that beneath they are voids, the viewer can still be seduced into watching. In the end, it is up to the viewer to decide if Zabriskie Point’s aesthetics, which privilege form over content, offer any meaning.


The last scene of the film, which is overtly apocalyptic, masterfully captures this dynamic. Here, the female lover imagines blowing up the ultra-modern house of her boss, a slick real estate developer who wants to produce a housing development in the desert. Antonioni’s slow motion explosion which shows mostly intact objects floating across a horizonless, flat blue surface of sky disrupts conventional Cartesian perspective as well as scale. Moreover, using slow motion and repeating the moment of the explosion from multiple points of view, Antonioni disrupts the conventional flow of time and a singular perceiving subject. However, as with the rest of the movie, nothing actually transpires. The film concludes with the woman, a virtual terrorist, driving away from the intact house without a clear destination. The strange physics of Antonioni’s explosion brings to mind the procedural physics operative in the software used to create VR experiences. These are the rules by which objects behave in a virtual 3D landscape, rules of interaction put into play by an algorithm in a game engine. As Baudrillard notes in his odd little book Seduction, games are
subversive precisely because they allow for a reversibility and fluidity of meaning that operates outside of law (Baudrillard, 1991,131-157). Because procedural physics operates outside of natural law, it offers the potential for radically new forms of narrative.
As the quote below from Donald MacKay, one of the key figures at the Macy Conferences on Cybernetics, points out: meaning, itself, acts as an “organizing function” for a perceiving subject. Significantly, MacKay defines information according to its functional impact, without respect to whether it is “true” or “false,” an idea which resonates with Flusser’s insistence that aesthetics determines the truth of a gesture.


INFORMATION can now be defined as that which does logical work on the organism’s orientation (whether correctly or not, and whether by adding to, replacing, or confirming the functional linkages of the orienting system). Thus, we leave open the question whether the information is true, false, fresh, corrective or confirmatory, and so on… The MEANING of an indicative item of information to the organism may now be defined as its selective function on the range of the organism’s possible states of orientation (Wilden, 1982, 236).


Discussing filmmaker Peter Greenaway’s move from film to installation, Lev Manovich suggests that spatialization allows Greenaway to construct pure database narrative, something which film does not allow. No longer having to conform to the linear medium of film, the elements of a database are spatialized within a museum or even the whole city. This move can be read as the desire to create a database at its most pure form: the set of elements not ordered in any way. If the elements exist in one dimension (time of a film, list on a page), they will be inevitably ordered. So the only way to create a pure database is to spatialize it, distributing the elements in space. (Manovich, “Database as a Symbolic Form”).

Following Bergson’s emphasis on the body as the starting point for perception, I would counter that this does not constitute pure database. At least in the “real world,” the narrative structure of installation arises out of the interaction of bodies with the architectural environment. As Lakoff and Johnson, and others have shown, human language also revolves around the body. However, in the virtual world, this is not necessarily so. Here, even gravity need not operate according to natural law and, furthermore, reading text can often induce nausea. Yet, even in virtual reality, meaning-making requires some form of structure.


One way that virtual reality narratives might recoup meaning without resorting to conventional cause and effect is by using procedural physics to structure narrative and to create asymmetries in the environment (database) which allow for meaning-making according to MacKay’s definition.
Artist Rachel Rossin’s virtual reality works illustrate this possibility of a new kind of narrative, one that speaks to and through virtualized, cyborg bodies. In an early work n=7/The Wake of Heat in Collapse, Rossin uses the procedural physics of gravity itself to shape narrative. Here, users progress through three contemporary Dantesque landscapes. Gravity in the landscape decreases algorithmically as the user descends. Invoking the virtual body of the user, Rossin notes:

In video games you have a certain type of hermeneutics…what you understand as the sort of “body language” or instinct that come with the world. Gravity – especially – falling off of things usually means death. We get this sad little feeling when we watch ourselves die in a video game and there is a pause that actually feels like critical distance and so I used that as a loose Virgil… the thing that pulls you through the layers of the separate worlds (Rossin, personal email correspondence December, 2015).

In a more recent work, I Came and Went as a Ghost Hand, Rossin scanned intimate photos of her domestic surroundings then manipulated these and used them to create a virtual reality landscape. As a result, the viewer sees familiar things differently, not just because of the data loss, which reduces some objects to their raw qualities – color, shape, volume – and which causes erasures, but because of the unsettling experience itself: moving through objects, speeding up, slowing down, torque. In lieu of a narrator, Rossin offers a reassuringly banal, reoccurring icon of a white gloved hand, a kitsch gesture that acts as a point of continuity for viewers as they traverse the disorienting, horizonless landscape.


In his essay, “Envisioning the Virtual,” Brian Massumi suggests that the virtual should be considered as the formative or potential dimension of reality, not as something which is in opposition to actuality. Analyzing an optical illusion where “Pac-man” circles cause a virtual triangle to appear, Massumi proposes that it is the tension between different forms of perception which produces “a field of intensity” within which the “pressure for resolution” acts as “a formative force.”
The tensions are between modes of existence proposing themselves to the experience[…] The modes do not add up to a form. They are tensely, incommensurably different. Their incommensurability exerts a differential pressure. Something has to give (Massumi, 2014, 56).
Massumi is describing a situation in which different ways of looking at the world seek to coexist in time. Cinema attempted to resolve this with montage, VR offers a different solution: the spatialization of time itself. How humans will organize that time-space in a meaningful way with respect to virtual bodies is evolving. Moreover, the political and social implications of this are as yet unknown. However, as I have shown, the memory of the real, preserved as a visual code borrowed from shared cinematic experience and the memory of the body, preserved as codified gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms might be revealed and uncovered with the aid of human-computer interfaces remains to be seen.

Bibliographie
AZUMA, Hiroki, Otaku: Japan’s Database Animals, translated from the Japanese by Jonathan E. Abel and Shion Kono, Minneapolis (MN), University of Minnesota Press, 2009.
BAUDRILLARD, Jean, Seduction, translated from the French by Brian Singer, New York, Palgrave Macmillan,1991.
BERGSON, Henri, Matter and Memory, translated from the French by N.M. Paul and W.S. Palmer, Zone Books, Cambridge (Ma), MIT Press, 1990.
FLUSSER, Vilém, Gestures, Kindle edition, translated from the German by Nancy Ann Roth, Minneapolis (MN), University of Minnesota Press, 2014.
HEIDEGGER, Martin, The Fundamental Concepts of Metaphysics: World, Finitude, Solitude, translated from the German by William MacNeill and Nicholas Walker, Bloomington (IN), Indiana University Press, 2001.
LAKOFF, George and Mark Johnson, Metaphors We Live By, Chicago (IL), University of Chicago Press, 2008.
LAMARRE, Thomas, The Anime Machine: A Media Theory of Animation, Minneapolis (MN), University of Minnesota Press, 2009.
MANOVICH, Lev, “Database as a Symbolic Form,” http://www.mfj- online.org/journalpages/Mfj34/Manovich_database_frameset.html, accessed online 22/01/2017., The Language of New Media, Cambridge (Ma), MIT Press, 2001. MASSUMI, Brian, “Envisioning the Virtual,” Oxford Handbook of
Virtuality, Oxford, Oxford University Press, 2014.
OLALQUIAGA, Celeste, The Artificial Kingdom: On the Kitsch Experience,
Minneapolis (MN), University of Minnesota Press, 2002.
WILDEN, Anthony, System and Structure: Essays in Communication and
Exchange, New York, Tavistock Publications, 1980.

ARCHITECTURE IN FLY ANGEL SOUL

In a clinic in Mali, in the late 1980’s, Sebastian, a young gay American physician discovers that to live with his new diagnosis of AIDS, he first needs to change the story he tells about himself.

The Great Mosque of Djenne in Mali

With FLY ANGEL SOUL, we utilize virtual production methods to radically alter the process of filmmaking. Here, liveness rests less in the performance of the actors (captured prior to filming with volumetric video) and more in the way the cinematographer responds to the mis-en-scene (spatial sound, captured performance, architecture, lighting, and objects) as it changes in real time. In so far as the mis-en-scene responds to the presence and actions of the cinematographer and vice versa. Filmmaking is a documentation of a call and response between the story”machine” and the cinematographer. As such, the set is critical to the filmmaking–not only to provide context for the story, but also to provide an environment for the cinematographer to inhabit and interact with. In such an environment, rules of the game stand in for “natural” laws. We are so pleased to have found Paolo Barlascini, an intuitive and intrepid artist who is acting as production designer/environmental architect.

Michel Foucault suggests that the contemporary organization of space resembles how data is organized on a computer in that it takes the form of “relations among sites” rather than a fixed local. For him, “the heterotopia is capable of juxtaposing in a single real place, several spaces, several sites that are in themselves incompatible.”

Paolo Barlascini, Artist and Production Designer working on the architecture for FLY ANGEL SOUL. VR is the ultimate heterotopia.

For FLY ANGEL SOUL we create a heterotopic building and environment. On the outside, the medical clinic references the soft edges and earth colors of Malian mud buildings. However, the interior itself is rigid. Whereas, the exterior displays the passage of time as erosion, inside time is regulated, represented by a rhythm of repeated arches.

Interestingly, Foucault suggests:

“the last trait of heterotopias is that they have a function in relation to all the space that remains….their role is to create a space that is other, another real space, as perfect, as meticulous, as well arranged as ours is messy, ill constructed, and jumbled. This latter type would be the heterotopia, not of illusion, but of compensation.

Indeed, in FLY ANGEL SOUL, the massive scale of the waiting room suggests a kind of over-compensation that tends towards the mythic (or the pathological… )

Architectural Plan of Waiting Room for FLY ANGEL SOUL

Here, the regimentation of the interior reflects both society’s attempt to “straighten” him as well as his own attempt to regulate his frenetic emotional state (exposed through rapid fire speech and large erratic gestures in scene 1. In this space, memory also complicates and unmakes the order of the architectural space. The uniformity of the space is disrupted by videos of other spaces from Sebastian’s past life in LA. These videos offer the tantalizing possibility of escape from the space Sebastian and the cinematographer finds herself in. It is as if the film audience could actually step into another film entirely.

Paolo’s design for the Doctor’s office in FLY ANGEL SOUL

In the second scene, the doctor’s office is designed as a long hall lined with windows, at the end of which is a massive wooden desk. Here, the the ceiling slopes down and the width of the room narrows to something less regal and more domestic as the doctor as king becomes doctor as father figure. The windows look onto a garden, an almost impossible oasis in the desert, which suggests that underneath all this sand, there is water, life. The shadow play of flora and fauna (birds and the eponymous flies) on the floor of the clinic brings the outside in. These arabesques queer and enliven the space, complicating the linear swaths of light and shadow that divide the room. In the same way, in his conversation with the doctor, Sebastian begins to loosen his rigid conception of good and bad which he has used to damn himself with all of his life.

In the last scene, both shadow and light disrupt the linearity and regularity of the waiting room space. The architecture itself has not changed, but now the interior resembles the exterior more. Similarly, the exit which mirrors the entrance, appears worn away by the passage of time, sand has built up in the corners, a little plant grows. Although Sebastian will die, his perspective has changed. With the rigidity of God, the father subsumed by the home/womb of the mother, there is the possibility that for the time that remains, Sebastian can find a new way to live.

Here, again, is Foucault:

The space in which we live, which draws us out of ourselves, in which
the erosion of our lives, our time and our history occurs, the space that claws and
gnaws at us, is also, in itself, a heterogeneous space.

WORK BEGINS ON FLY ANGEL SOUL –Our experimental short shot entirely in VR

The “windows” we used for viewing in our In My Own Skin virtual installation, morph into the decorative grille/grids of Malian architecture (where the film takes place)

FAS tells the story of Sebastian, a young gay physician, estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart that we might “rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” FAS will be created using Unity’s Cinemachine software with three networked cameras adopting the p.o.v of the eponymous characters. What each camera “hears and sees and how each moves depends in part upon the actions of the other cameras in real time. The cameras function as the “players,” both in the video game sense and in a theatrical sense. The cameras’ real-time “performance” is the material for the 2D film. Thus, in FAS, “liveness” resides in the “embodied” cameras as much as in the actors whose performance is pre-recorded with volumetric video. Thus, the film, itself, is a poetic documentation of both human and computer machinations. Montage, as such, will not come through a post-production editing process, but occurs and becomes manifest as a result of the procedural logic of the game engine + the incommensurable logic of the human operator/performer. In keeping with Eckhart’s intent, the final film will display all the p.o.v’s on one screen,  a tripartate montage of images and sounds, not created in post, but recorded “live” in real time. 

Even though we work with cutting edge technology, I start projects with paper and scissors and glue. Today, Cyril, who is now used to this came over to discuss what I’d come up with. (queerskins a novel queerskins.com began with a 100 page pile of collages that I showed Cyril at lunch one day. They were made on children’s colored construction paper as I’d work late at night after my two young kids were asleep and that was all I had available at the time I started.)

It’s wild to see how your mind builds on things–the delving into architecture and windows for viewing in our In My Own Skin project last year is taken up ten levels. In this project, architectural space and light actual become as integral to the film as the actors.

Ophuls is a major influence for the set in scene 2 (use of multiple patterns/textures/l) which find there real equivalent in contemporary Malian apartment rentals!

We will build a set based on radial forms –see the original plan for St. Peter’s Basilica in Rome above.

Other artists’ inspire–screens by Jorge Pardo and paintings/installation by Lucio Fontana. I

In the final scene, I note that the Angel which manifests to the human as light is powerful force inspiring awe and terror.. Cyril and I spent part of our time today discussing “how does an angel see?” So, I look for this brain worm, later and find it on my shelf in Rilke’s Duino Elegies

“Who, if I cried out, would hear me among the angelic orders? And, even if one of them pressed me suddenly to his heart: I’d be consume in his stronger existence. For beauty is nothing but the beginning of terror, which we can just barely endure, and we stand in awe of it as it coolly disdains to destroy us. Every angel is terrifying. ” So, I guess this has been living in my mind all these years, waiting to emerge!

Jerome Foundation Awards Us a Grant to Shoot a 2D Experimental Film in VR!

Fly Angel Soul is a short experimental narrative film shot within virtual reality which explores the potential for virtual production techniques to expand two dimensional cinematic language. FAS tells the story of Sebastian, a young gay physician, estranged from his rural Catholic Missouri family, who, having moved to Mali to heal the sick, is diagnosed with AIDS. Inspired by a quote from Meister Eckhart that we might “rejoice in the everlasting truth in which the highest angel and the soul and the fly are equal,” FAS will be created using Unity’s Cinemachine software with three networked cameras adopting the p.o.v of the eponymous characters. What each camera “hears and sees and how each moves depends in part upon the actions of the other cameras in real time. These positions are not characters, per se. They function as aspects of Sebastian’s interior milieu. Although all three p.o.v. will be equally represented in the final film, using a split screen, FAS unapologetically privileges the ineffable workings of the human heart as the driving and unprogrammable logic of the film. The human camera is the only one operated by an actual living being. The angel and fly exist as state machine, pre-programmed virtual entities.

Looking to “embodied” films like Max Ophul’s Le Plaisir for inspiration

We are not advocating an impossible return to a pre-technical state of “nature”, rather, we are asking what suffering means in our technologically embedded existence.If as Jean-Luc Godard famously stated, “The tracking shots are a matter of morality,” the use of virtual cameras in agile film production brings up pressing ethical questions which have yet to be confronted. In FAS, a “simple” and universal story of human suffering–a diagnosis of terminal illness–AIDS at the beginning of the epidemic, invites viewers to contemplate how suffering is mediated using digital technologies. Our intent is to construct the film in a way that reasserts an embodied, participatory perspective, one that acknowledges the primacy of a “human” perspective while, at the same time, offering the audience alternative, perhaps transcendent computer-mediated ways of seeing, hearing and moving through the same story.


In FAS, all three cameras function as the “players,” both in the video game sense and in a theatrical sense. The cameras’ real-time “performance” is the material for the 2D film. Thus, in FAS, “liveness” resides in the “embodied” cameras as much as in the actors whose performance is pre-recorded with volumetric video. Thus, the film, itself, is a poetic documentation of both human and computer machinations. Montage, as such, will not come through a post-production editing process, but occurs and becomes manifest as a result of the procedural logic of the game engine + the incommensurable logic of the human operator/performer. In keeping with Eckhart’s intent, the final film will display all the p.o.v’s on one screen, a tripartite ever changing montage of images and sounds, not created in post, but recorded “live” in real time.