About Illya Szilak

Illya Szilak is a transmedia writer/artist, independent scholar, and curator. She and her long time collaborator Cyril Tsiboulski (Cloudred Studio) were recently awarded a grant from Tribeca Film Institute/MacArthur Foundation to create a VR experience inspired by their online narrative installation Queerskins. Reconstructing Mayakovsky www.reconstructingmayakovsky.com was included in the second Electronic Literature Collection and was a jury pick for The Japan Media Arts Festival 2010. The animation done in collaboration with Pelin Kirca has been shown in eight film festivals around the world. Her second multimedia novel Queerskins www.queerskins.com was recognized by the Webby's in the category of NetArt in 2013 and was exhibited at the 5th International Digital Storytelling Conference in Ankara and at the Bibliotheque National in Paris. It was recently featured as part of a group show Queertech.io at three LGBTQ festivals in Australia. She and VR artist Oscar Raby (VRTOV Studio) received a grant from the Sundance Institute/Arcus Foundation to make a VR experience inspired by Queerskins. She is an Oculus Launchpad Fellow. Her longtime collaborator is interactive designer Cyril Tsiboulski at Cloudred Studio (NYC). Their first VR experience Queerskins: a love story which combined VR, site specific installation and crowdsourced performance photography was awarded the Special Jury Prize for VR by the Columbia University Digital Storytelling Lab and a Peabody Futures of Media Award for transmedia. Their second VR experience Queerskins: ark is being co-produced by Intel Studios and is expected early 2020.

My Conversation with #Architect Beom Jun Kim about #VR, #Narrative #Design, and #Architecture and #Queerskins #Home #Art #Installation in #VRChat

Church in Queerskins Home: “All People with AIDS are Innocent” by Gran Fury, photo of stained glass window by Cyril Tsiboulski of cathedral in Missouri, “Annunciation” by Fra Angelico

My fascinating conversation with architect Beom Jun Kim (w.a.k) about VR and architecting space and narrative with reference to Queerskins Home, the free, open 24/7 interactive installation we made in VRChat. Talking sound as a spatial element, kitsch, symbolism, Richard Serra, queer phenomenology, time and duration, Brecht, North Korea and English follies Next level, folks. If you make in VR, you want to see this.

#Learning from Las Vegas? or when is a #closet just a closet? #Architecture, #VR, #Narrative and #Game –a talk with architect Beom Jun Kim (wa.k studio) tomorrow in VRChat

Tomorrow September 4 at 2 PM EST (8 PM CEST), architect Beom Jun Kim (wa.k studio), who teaches architectural design, augmented reality and virtual reality at the Spitzer School of Architecture (CUNY), Barnard College, and Yale University and who advised us on the creation of Queerskins Home will be in conversation with me about architecture, VR, and narrative/game design in the “truckstop” in Queerskins Home world in VRChat. This should be really interesting. I mean when is the last time you heard Venturi/Scott Brown/Izenour’s “Learning from Las Vegas” referenced in a talk on VR? Architecture is an under-utilized aspect of narrative in VR. It will also be live-streamed on Queerskins page. You can also watch video later on QS page.

Also, tomorrow September 4 at 1 PM EST (7 PM CEST) Queerskins: ARK choreographer Brandon Powers and I (writer/director) will be in the Venice VR Expanded Garden in VRChat( accredited visitors only ) for a “meet the directors” social hour. Hope to see you there!

Queer + Skins = Welcome + Home

Queerskins Home public world in VRChat

It is easy living in a place like NYC or LA to forget that there are a lot of places where coming out could mean losing friends, family, or worse. Tonight, I was reminded poignantly why I called our story “Queerskins” — not only do I want people to say “queer” and “skin” together, and maybe feel a little uncomfortable in their own skin, but, perhaps even more important, it is a neon sign flashing– “welcome queers.”“Queerskins Home” our art exhibition, interactive story world in VRChat is a 24 hour seven day a week safe haven for exploring difficult topics: sexuality, gender, family, forgiveness, shame and spirituality. Come, check it out! You will not be disappointed.We’ve decided to switch the artist lead tours through the space to once daily at 11AM through September 12th. These are intense for us. Twice a day is just too much to sustain. Hope to see you in VRChat tomorrow. Or, watch the livestream or catch the video recording later on https://www.facebook.com/queerskins

Special Events Planned for Queerskins: ARK #VR #Premiere at The Venice International Film Festival

We are the hardest working artists in show business!  In honor of the premiere of Queerskins:ARK VR, We created an amazing Queerskins Home world in VRChat — part art exhibition, part interactive story world with spatial audio objects and images accumulated over the last ten years of our queerskins world building. All are free and open to the public. Anyone with a PC or VR headset can make an account on VRChat and come into “Queerskins Home” world.



ARTIST LEAD installation walks daily 11 AM These will also be livestreamed or video posted later to FB on the Queerskins Page. https://www.facebook.com/search/top?q=queerskins

Please join writer/director Illya Szilak and special guest for an illuminating installation walk-through that spans the ten years of our Peabody Futures of Media award-winning trans-media story creation. Enter “Queerskins Home” through VRChat directly. at 6 PM EST/12AM CEST You can also watch a livestream of it on FB https://www.facebook.com/queerskins.
Special guests, actors Michael DeBartolo (Sebastian ) and Hadley Boyd (MaryHelen) from Queerskins: ARK will perform readings from the diary and make appearances in the installation In selected tours. Follow us @queerskins on Insta and Twitter for daily schedules.


Artist Talks Short talks will take place in the “truckstop” space in Queerskins Home world on VRChat with time for question and answer. These will also be live-streamed to Queerskins page on FB. https://www.facebook.com/queerskins.

Friday September 5 at 2 PM EST/8 PM CEST,Architecture, VR, Narrative, Game,” join writer/director Illya Szilak and architect Beom Jun Kim (wa.k studio) in the public “Queerskins Home” world on VRChat for a fascinating discussion about architecture and VR, also live-streamed to Queerskins page on FB.

https://www.facebook.com/zach.milonas/videos/3189137824474405 LIVE STREAM video capture

Saturday September 5 at 2PM EST/8 PM CEST
, “Creating Interactive Choreography for Volumetric Capture” join choreographer and lead artist for Queerskins: ARK, Brandon Powers, as he discusses the key considerations for movement design in volumetric experiences. Enter into Queerskins home world through the VeniceVR Exhibition Hall. 

Sunday September 6 at 2PM EST/8PM CEST, “Expanded Approaches to Empathy in VR: Lessons from Queerskins: ARK” join Illya Szilak/Writer and Director for Queerskins: ARK as she discusses theoretical and practical considerations for designing narratives for empathy in VR. Actor Michael DeBartolo (Sebastian) offers his insights as lead actor in Queerskins: ARK and as a gay man. Enter into Queerskins home world through the VeniceVR Exhibition Hall or through VRChat directly. 

Wednesday September 9th at 11AM EST/5PM CEST, “Expanded Approaches to Empathy in VR: Lessons from Queerskins: ARK” join Illya Szilak/Writer and Director for Queerskins: ARK as she discusses theoretical and practical considerations for designing narratives for empathy in VR. Actor Michael DeBartolo (Sebastian) offers his insights as lead actor in Queerskins: ARK and as a gay man. Meet at the “truck stop” in “Queerskins Home” public world in VRChat.

Thursday September 10th at 5PM EST/11 PM CEST, Creating Interactive Choreography for Volumetric Capture” the public is invited to join choreographer and lead artist for Queerskins: ARK , Brandon Powers, on Zoom as he discusses the key considerations for movement design in volumetric experiences.

I guarantee these talks will bring our outside-the-box perspective to these topics. Hope to see you there!


Queerskins HOME is now live on VRChat–Is this the #future of #film? #VR #XR #Art #installation #interactive #game #storytelling?

Welcome home. Queerskins Home is LIVE. World #83614 in VRChat. Search #queer. Part art exhibition space, part 3D explorable story world. Due to constraints of VRChat, a free massive player social app, available for Oculus Quest, all tethered VR headsets and online PC, we do not have video (as does not support for Quest) nevertheless, I see the future of cinema here. It is like nothing I have ever experienced.

Of all the things Cyril Tsiboulski and I have made over the last ten years, this feels perfect–raw and unfinished and a gut punch. I know “Art” is pretty and/or cerebral and/or conceptual, but that is not what we made. So proud of it. Amazing lighting help from Unity engineering superstar Elliott Mitchell and architectural advice from Beom Jun Kim, and, of course, Cyril Tsiboulski who is my wonder twin, and created this in 4 days. Working together and on this project for close to ten years is what makes it so deep and so powerful. It took that kind of devotion…Thank you all.

Audio is a big part of this so please adjust by going to your settings on the menu and adjusting master, world sounds and people (other avatars around you) sound. Starting September 2nd, in conjunction with the premiere of Queerskins: ARK at The Venice International Film Festival, I will be leading daily tours with special guests Michael DeBartolo (who plays Sebastian) in Queerskins: ARK and Hadley Boyd (who plays Mary-Helen, his mother). She will be reading live from Sebastian’s diary. So follow us at Queerskins and @queerskins on Insta/Twitter to find out the day’s schedule.

Storytelling at the End of the World: Cinema and Narrativity in Virtual Reality

zabriskie

 

I’m revisiting this essay because I am trying to get together a book proposal tentatively titled Designing Narratives for VR: philosophical, theoretical and practical considerations. It is inspired by my teaching classes in this and seeing enthusiastic student response, and from my wide-ranging reading on related topics from dance film to Heidegger’s standing reserve. Based on my experience, no one approaches VR this way. That is neither good nor bad, but, VR has rapidly been coopted by corporate/commercial concerns (and, god knows they aren’t interested in Marinetti’s Manifesto of Tactilism… ).  I think this book would help open up important conversations. I would love to know what your favorite books on this topic are–so I can actually confirm my suspicions.   Also, is this a book you would want to read? Recommendations for publishing houses also welcome. I love University of Minnesota offerings, but open to anything.  Thanks!

 

Really pleased that my paper will be published in Transatlantica Journal, a biannual publication of the French Association for American Studies. This is based on a talk I gave in Paris last March.  This is useable under Creative Commons,  attribution, please. #VR #cinema #apocalypse #epistemology #narrative

The word “apocalypse” comes from the Greek apokaluptein meaning to uncover or reveal. In keeping with the word’s origins, I approach the concept of apocalypse not as the destruction of the material world, per se, but, epistemologically, as the end of the world as we have known it. If not the primary agent of this epistemic cataclysm, computer technology, which increasingly mediates human perception of reality, certainly reflects these changes. As might be expected, these technologies have also dramatically changed how we communicate about the world. In his essay, “Database as a Symbolic Form, ” new media theorist Lev Manovich describes this as a move from “narrative” to “database” storytelling.

As a cultural form, database represents the world as a list of items and it refuses to order this list. In contrast, a narrative creates a cause-and-effect trajectory of seemingly unordered items (events). Therefore, database and narrative are natural enemies. Competing for the same territory of human culture, each claims an exclusive right to make meaning out of the world (Manovich, 1999 “Database as a Symbolic Form”).

Manovich concludes his essay by praising the Soviet film director, Dziga Vertov whose Man With a Movie Camera, he argues, successfully creates a new kind of narrative by ordering a database of images around the “kino-eye” of the filmmaker.

This process of discovery is film’s main narrative and it is told through a catalog of discoveries being made. Thus, in the hands of Vertov, a database, this normally static and “objective” form, becomes dynamic and subjective. More importantly, Vertov is able to achieve something which new media designers still have to learn — how to merge database and narrative merge into a new form (Manovich, 1999, “Database as a Symbolic Form”).

Whereas, even today, both the novel and cinema remain mostly on the side of conventional narrative, many new media forms resist the linear arrangement of information. Of these, virtual reality (VR) most directly and completely mediates visual sense perception. In the VR headset, the world disappears, replaced by an alternative reality generated by a computer. Here, the user is no longer a passive consumer of someone else’s database. By exploring and interacting with the environment, she orients herself according to her own interests. One might even go so far as to say that she becomes a kind of film director framing close-ups and long-shots.

Philosopher Vilém Flusser has characterized the time we live in as an epistemic crisis brought about by the recognition that, “scientific research is not the gesture of a transcendent intellect.”  What Flusser calls “the gesture of searching,” for meaning or knowledge, has shifted from “a digging down for reasons” to an examination of the aesthetic and relational qualities of things and an exploration of how humans respond and attend to their environments as subjects and as objects (Flusser, 2014, “The Gesture of Searching”). Perhaps unsurprisingly, Flusser’s description of this gesture serves as an apt description of being in virtual reality, a reality which has become, in essence, a database.

…the researcher is embedded in an environment that interests (matters to) him, both at close range and at a distance. There are aspects of the environment that interest him intensely and others that hardly touch him. The more an aspect of the environment interests the researcher, the more ‘real’ it is for him (Flusser, 2014, “The Gesture of Searching).

In this essay, I will build upon Manovich’s observation that, “cinema language, which was originally an interface to narrative taking place in 3-D space, is now becoming an interface to all types of computer data and media” (Manovich, 2001, 326). I will argue that whatever the language of VR turns out to be, it will rely far less on symbols and signs (content)  and more on a procedural code (form) that elicits both the memory of the body’s movements and affects as well as a shared cultural memory, not of the lived world, but of cinema, itself.

What determines the authenticity or veracity of this kind of storytelling? Here, it is useful to consider one of the earliest, most widely distributed experiences in VR. The Tuscan Villa  was provided free with the developer kit for the Oculus Rift DK2 head mounted display. It offers users the chance to meander through a country villa and surrounding landscape. A generic looking computer-generated farm house, its stones visually coded as “old,” suffices to invoke a nostalgic memory—one that will be even more poignant and believable to a viewer who has never actually stayed in a Tuscan villa. In other words, the Tuscan villa is meant to exist outside of history. In this work, the visual and auditory codes are perceived as “authentic” if they successfully activate a process of false memory and effect a change in the viewer’s affective state. Significantly, the experience does not provide a traditional narrative. Rather, it  offers the vague sense of remembering a dream or retrieving something that has been lost which never actually existed.

Kitsch is nothing if not a suspended memory whose elusiveness is made ever more keen by its extreme iconicity […] kitsch is not an active commodity naively infused with a wish image, but rather a failed commodity […] a virtual image, existing in the impossibility of fully being. Here, for a second or even a few minutes, there reigns an illusion of completeness, a universe devoid of past and future […]. (Olalquiaga, 1998, 28).

In The Language of New Media, Manovich points out that as cinema becomes more and more reliant on software manipulation in postproduction, it functions less as “a record of perception,” and more like a computer screen— “as a record of memory” (Manovich, 2001, 325). In the case of the Tuscan Villa, cinema no longer serves an indexical function vis a vis reality, but rather, it registers the trace of a shared cinematic memory. The kitsch experience of the villa and garden offers a momentary illusion or simulacrum of “realness”. For this, it sacrifices the passage of time and complexity of meaning.

Flusser suggests that gesture, by which he means the “reality” of affective states made manifest through aesthetics, should be judged in terms of art or kitsch rather than whether the gesture is true or false. “The scale of values we use to evaluate (gesture) may not oscillate between truth and error or between truth and lies but must move between truth (authenticity) and kitsch (Flusser, 2014, The Gesture of Searching).

It is not artifice in itself that renders the Tuscan villa demo artless. To understand this,  it is instructive to compare the Oculus demo to Maria Menken’s  decidedly un-kitsch 1957 film Glimpse of the Garden . Even though both The Tuscan Villa demo and Menken’s film privilege surface effects over depth of meaning, the result is decidedly different. In her film, movement of the camera, a unifying looped soundtrack of birdsong, and color filters take precedence over content, which consists mostly of shots of cultivated nature. Despite the amateurish quality of the images, the result is strangely magical. Menken’s editing, which juxtaposes different scales, points of view, and speeds of movement, resists the Tuscan villa illusion of completeness. Thus, it is due to the authenticity of the filmmaking gesture itself, not its content, that Menken’s film can be called art whereas the Oculus demo remains kitsch.

If there is any redeeming gesture in the Villa, it is in its ability to invoke an uncanny sense of bodily presence, a being there in a digital world that is generated by the user’s wandering or, in Flusser’s terminology, “searching” through the environment. According to neuroscientists,  presence is a primordial mechanism by which organisms establish self and other, creates a motor map of the universe and uses memory to plan actions and interactions. In so far as the Tuscan Villa succeeds in creating this sense of presence, it sets up a queer dynamic in the user—an oscillation between interior and exterior, self and other, the “real” and the virtual. In this way it, perhaps inadvertently, succeeds in making the familiar strange. In other words, the odd and unresolvable experience of being in two places at once creates a kind of digital estrangement conceptually related to Bertolt Brecht’s Verfremdungseffekt, distancing effect, and Russian Formalist critic Viktor Shklovsky’s “making strange” (Russian: приём остранения priyom ostraneniya) whereby certain devices are used to disrupt the audience’s easy identification with an artwork.

To see how this might be productively used in VR, I turn to filmmaker Karen Cytter’s work for a contemporary example of estrangement. Cytter employs familiar, even kitsch codes of cinema, especially melodrama. But, rather than offer an illusion of completeness, she uses what Thomas LaMarre refers to as “internal montage” (LaMarre, 2009,125). LaMarre suggests that computer graphic imagery and digital effects have created a new emphasis on the animation technique of compositing —the creation of an image composed of multiple layers. If an artist suppresses the space between layers to create an illusion of wholeness, the result is either a unification in depth, a hyper-Cartesianism with a well defined vanishing point (e.g. Pixar type animations), or a flattening of all layers into a single “superflat” plane in which multiple frames of reference and sight lines coexist. Cytter’s films demonstrate a remarkable compositing of space, dramatic events, and sound. It is effectively a jumbled cinematic database in which voice, visual form, emotion, and plot nearly float free of each other on the level of meaning.

Surface effects are at least as important as content here. Through a process of making the familiar strange, she astutely reveals the underlying game of seduction that takes place in cinema. In Cytter’s work we are left with empty codes, empty signs which nevertheless incite emotional responses. Devoid of linear narrative, in Rose Garden , the mournful sound of the flute is not only a wry reference to Sergio Leone’s Once Upon A Time in the West, it codes the psychological tenor of the piece. Although there are no clear “characters,” as with avatars in video games, the human body remains the basis for emotional engagement. For instance, at the end of the film, when a boy is shot in the back, though we know nothing about him as a character, the violent collapse of a child’s body is still shocking and emotionally wrenching.

Rose Garden’s flattening out of time and space, and a move from depth to surface, which Flusser relates to the gesture of searching, resonates with Hiroki Azuma’s description of database narrativity. In his book, Otaku: Japan’s Database Animals, he notes a move from “grand narratives” to “grand non-narratives” or database (Azuma, 2009, 55). This is associated with the proliferation of small narratives the purpose of which is not to refer back to some greater or universal theme, but to invoke strong feeling or fulfill consumer preferences. Whereas, in the modernist era, personal fictions could be related in depth to a grand story about the nature or meaning of human life, now, there is a lateral move in which multiple small stories are spawned from the same database as in Cytter’s film. Likewise, in virtual reality, narrative unfolds as movements through space. However, distance is no longer measured in terms of meters or any other universal objective standard, but, as both Flusser and Azuma suggest, in terms of the user’s desires and interests interacting with the environment.

In Mark Amerika’s cell phone film Immobilité, one of the characters remarks that the world is disappearing before her eyes. Indeed, the opening scenes, a shaky, looped walk down a tree-canopied path, reveal just how the world will end— not in some fiery apocalypse, but rather by our recognition that the world was only ever virtual. In Immobilité, what remains of embodied experience, what Amerika calls “muscle memory,” finds its expression not in meaning, but in form. In effect, Immobilité a film that remixes some of Flussers thought into the subtitle track consists not so much as a series of images, but as a series of film gestures.

In Immobilité, the quality of the image is sacrificed to movement of the camera just as content is sacrificed to surface effects. Depth, in the form of philosophic musings, is separated from the images and brought to the surface as subtitle. What remains on the screen are vibrations, rhythms, and frequencies that resonate with or cancel each other out. In fact, in many parts of the film, the camera is decidedly unsteady or even appears to vibrate thereby creating a surface that destroys a Cartesian perspective and the static perceiving subject. “And then I would begin to lose myself, to play uncontrollably, becoming something like music,“ the unreliable narrator texts us from the beyond.

Heidegger’s concept of “profound boredom” operates here. This is state in which all beings “recede” into indifference and the one who is bored comes face to face with the experience of time itself (Heidegger, 2001, 80). Heidegger describes profound boredom as the ground for all the multifarious variations of being. Profound boredom leads us to suspect that reality is neither static (immobile) nor completely knowable. Concomitant with this loss of a fixed self, language fails to signify. Rather, it functions procedurally to carve up pure duration into pieces: life and death, existence and nonexistence, chaos and order. This resonates with Henri Bergson’s suggestion in Matter and Memory that homogeneous and universal perceptions of space and time are a trick of the mind, a refraction of pure duration into space. For Bergson, space and time do not exist anterior to perception and action. We do not act through space and time, rather we create these symbolic diagrams because we need to “divide the continuous, fix the becoming, and provide our activity with points to which it can be applied (Bergson, 1990. 212). Bergson suggests that our knowledge of the world is delimited by the concerns of the body and its possible actions and, perhaps, even more importantly, by our memories. If that is true, we are like Plato’s cave dwellers: always living in a virtual reality of our own making.

The key dynamic in Amerika’s film, one that will likely dominate in this time of apocalyptic narrative, is a tension between overt aesthetics and conventional notions of truth. Although Immobilité’s low production values and distinct lack of artful montage lend the film a “real-life” documentary feel, Amerika’s overt use of glitch, repetition, and jerky camera movement insists upon a kind of self-conscious performativity or aesthetics. When the narrator asks, “Was I authentic?” the question is equally what does “authentic” mean if everything is virtual and, ultimately, data, and, furthermore, who is “I”? If the world is disappearing, so is the singular author, that figure of the auteur which haunts French New Wave cinema, to which Amerika  pays homage in his use of subtitles. In Immobilité’,  Amerika, like Flusser, suggests that the question of “authenticity” lies not in a “true” identity but in the aesthetic value of the gesture itself.

Antonioni’s masterwork, Zabriskie Point, also operates by disavowing conventional notions of authenticity. In Zabriskie Point all is surface: from the setting, an almost abstract stretch of ancient desert, to the way images are composed in multiple layers without depth. There is also a strange flatness to the main characters. They seem to have no ambitions and no particular future. With exquisite conciseness, Antonioni destroys the epic march of history with the first line spoken by the male protagonist: “I’m willing to die (for the revolution), but not out of boredom.” For the characters in Zabriskie Point,  time operates as it does in profound boredom as a pure duration.

Even on a personal level, events have no apparent consequence. Though one of the young lovers dies violently, there seems to be little emotional repercussion for the lover that remains. This flatness of character is matched by the physical beauty of the actors. It does not matter that beneath they are voids, the viewer can still be seduced into watching. In the end, it is up to the viewer to decide if Zabriskie Point’s aesthetics, which privilege form over content, offer any meaning.

The last scene of the film, which is overtly apocalyptic, masterfully captures this dynamic. Here, the female lover imagines blowing up the ultra-modern house of her boss, a slick real estate developer who wants to produce a housing development in the desert. Antonioni’s slow motion explosion which shows mostly intact objects floating across a horizonless, flat blue surface of sky disrupts conventional Cartesian perspective as well as scale. Moreover, using slow motion and repeating the moment of the explosion from multiple points of view, Antonioni disrupts the conventional flow of time and a singular perceiving subject. However, as with the rest of the movie, nothing actually transpires. The film concludes with the woman, a virtual terrorist, driving away from the intact house without a clear destination.

The strange physics of Antonioni’s explosion brings to mind the procedural physics operative in the software used to create VR experiences. These are the rules by which objects behave in a virtual 3D landscape, rules of interaction put into play by an algorithm in a game engine. As Baudrillard notes in his odd little book Seduction, games are subversive precisely because they allow for a reversibility and fluidity of meaning that operates outside of law (Baudrillard, 1991,131-157). Because procedural physics operates outside of natural law,  it offers the potential for radically new forms of narrative.

As the quote below from Donald MacKay, one of the key figures at the Macy Conferences on Cybernetics, points out: meaning, itself, acts as an “organizing function” for a perceiving subject. Significantly, MacKay defines information according to its functional impact, without respect to whether it is “true” or “false,”  an idea which resonates with Flusser’s insistence that aesthetics determines the truth of a gesture.

INFORMATION can now be defined as that which does logical work on the organism’s orientation (whether correctly or not, and whether by adding to, replacing, or confirming the functional linkages of the orienting system). Thus, we leave open the question whether the information is true, false, fresh, corrective or confirmatory, and so on…The MEANING of an indicative item of information to the organism may now be defined as its selective function on the range of the organism’s possible states of orientation (Wilden, 1982, 236).

Discussing filmmaker Peter Greenaway’s move from film to installation, Lev Manovich suggests that spatialization allows Greenaway to construct pure database narrative, something which film does not allow.

No longer having to conform to the linear medium of film, the elements of a database are spatialized within a museum or even the whole city. This move can be read as the desire to create a database at its most pure form: the set of elements not ordered in any way. If the elements exist in one dimension (time of a film, list on a page), they will be inevitably ordered. So the only way to create a pure database is to spatialize it, distributing the elements in space. (Manovich, “Database as a Symbolic Form”).

Following Bergson’s emphasis on the body as the starting point for perception, I would counter that this does not constitute pure database. At least in the “real world,” the narrative structure of installation arises out of the interaction of bodies with the architectural environment. As Lakoff and Johnson, and others have shown, human language, also revolves around the body. However, in the virtual world, this is not necessarily so. Here, even gravity need not operate according to natural law and, furthermore, reading text can often induce nausea. Yet, even in virtual reality, meaning-making requires some form of structure.

One way that virtual reality narratives might recoup meaning without resorting to conventional cause and effect is by using procedural physics to structure narrative and to create asymmetries in the environment (database) which allow for meaning-making according to MacKay’s definition.

Artist Rachel Rossin’s virtual reality works illustrate this possibility of a new kind of narrative, one that speaks to and through virtualized, cyborg bodies. In an early work n=7/The Wake in Heat of Collapse, Rossin uses the procedural physics of gravity itself to shape narrative. Here, users progress through three contemporary Dantesque landscapes. Gravity in the landscape decreases algorithmically as the user descends. Invoking the virtual body of the user, Rossin notes:

In video games you have a certain type of hermeneutics…what you understand as the sort of “body language” or instinct that come with the world. Gravity – especially – falling off of things usually means death. We get this sad little feeling when we watch ourselves die in a video game and there is a pause that actually feels like critical distance and so I used that as a loose Virgil… the thing that pulls you through the layers of the separate worlds (Rossin, personal email correspondence December, 2015).

In a more recent work, I Came and Went as a Ghost Hand, Rossin scanned intimate photos of her domestic surroundings then manipulated these and used them to create a virtual reality landscape. As a result, the viewer sees familiar things differently, not just because of the data loss, which reduces some objects to their raw qualities: color, shape, volume, and which causes erasures, but because of the unsettling experience itself: moving through objects, speeding up, slowing down, torque. In lieu of a narrator, Rossin offers a reassuringly banal, reoccurring icon of a white gloved hand, a kitsch gesture that acts as a point of continuity for viewers as they traverse the disorienting, horizonless landscape.

In his essay, “Envisioning the Virtual,”  Brian Massumi suggests that the virtual should be considered as the formative or potential dimension of reality, not as something which is in opposition to actuality. Analyzing an optical illusion where “Pac-man” circles cause a virtual triangle to appear, Massumi proposes that it is the tension between different forms of perception which produces “a field of intensity” within which the “pressure for resolution” acts as “a formative force.”

The tensions are between modes of existence proposing themselves to the experience[…] The modes do not add up to a form. They are tensely, incommensurably different. Their incommensurability exerts a differential pressure. Something has to give (Massumi, 2014, 56).

Massumi is describing a situation in which different ways of looking at the world seek to coexist in time. Cinema attempted to resolve this with montage, VR offers a different solution: the spatialization of time itself. How humans will organize that time-space in a meaningful way with respect to virtual bodies is evolving. Moreover, the political and social implications of this are as yet unknown. However, as I have shown, the memory of the real, preserved as a visual code borrowed from shared cinematic experience and the memory of the body, preserved as codified gesture will likely dominate the language of VR, at least for now. What gives way after that, what new perceptions and new narrative forms might be revealed and uncovered with the aid of human-computer interfaces remains to be seen. 

Bibliography

AZUMA, Hiroki, Otaku: Japan’s Database Animals, translated from the Japanese by Jonathan E. Abel and Shion Kono, Minneapolis (MN), University of Minnesota Press, 2009.

BAUDRILLARD, Jean, Seduction, translated from the French by Brian Singer, New York, Palgrave Macmilan,1991.

BERGSON, Henri, Matter and Memory, translated from the French by N.M. Paul and W.S. Palmer, Zone Books, Cambridge (MA), MIT Press, 1990.

FLUSSER, Vilém, Gestures, Kindle edition, translated from the German by Nancy Ann Roth, Minneapolis (MN), University of Minnesota Press, 2014.

HEIDEGGER, Martin, The Fundamental Concepts of Metaphysics: World, Finitude, Solitude, translated from the German by William McNeill and Nicholas Walker, Bloomington (IN), Indiana University Press, 2001.

LAKOFF, George and Mark JOHNSON, Metaphors We Live By, Chicago (IL), University of Chicago Press, 2008.

LAMARRE, Thomas, The Anime Machine: A Media Theory of Animation, Minneapolis (MN), University of Minnesota Press, 2009.

MANOVICH, Lev, “Database as a Symbolic Form,” http://www.mfj-online.org/journalPages/MFJ34/Manovich_Database_FrameSet.html, accessed online 22/01/2017.

———, The Language of New Media, Cambridge (MA), MIT Press, 2001.

MASSUMI, Brian, “Envisioning the Virtual,” Oxford Handbook of Virtuality, Oxford, Oxford University Press, 2014.

OLALQUIAGA, Celeste, The Artificial Kingdom: On the Kitsch Experience, Minneapolis (MN), University of Minnesota Press, 2002.

WILDEN, Anthony,  System and Structure: Essays in Communication and Exchange, New York, Tavistock Publications, 1980.

Queerskins: ARK #VR #Premiere at The Venice International Film Festival Will Feature an Exclusive Private World/ #Interactive #Art #Installation in #VRChat

VRchat gang

We have finally found our queer home–psychologically architected domestic space, unlike anything I have ever seen. Imagine a dream in which you are wearing your still beating heart on your sleeve…
Because of the content–some of it quite sexual, Queerskins is a private world, curated from our extensive archive of images and audio,  collected over the last ten years, in VRChat. Space will be limited. We will be offering exclusive “installation” walkthroughs lead by artists from Queerskins ARK VR , an Intel Studios original, (co-produced by Cloudred and Intel Studios)  and talks (not the usual if you know me) by me, Cyril Tsiboulski, and choreographer Brandon Powers. We will have a better sense of how we will invite interested people into our world during The Venice International Film Festival in the coming week or so. If you have a Quest, Vive, Oculus,  or PC (for online access to VRChat) and want to beta test the world, leave your interest in comments and how to best reach you.

 

If you are or know a curator or press interested in cutting-edge new media art and/or immersive storytelling, please let us know so we can offer them an invite.
Here are details on accessing the Venice International Film Festival VR expanded.
The works of Venice VR Expanded , the Virtual Reality section of the 77th Venice International Film Festival (2-12 September 2020) of the Venice Biennale, will be accessible in the VR Lounges of various cultural institutions around the world – as well as online (FOR FREE!) through Viveport for Vive Pro, Vive Cosmos, Oculus Rift, Quest with PC link, Valve Index, Windows Mixed Reality.
The cultural institutions involved are:
China Academy of Art – Sandbox Immersive Festival, Hangzhou
Comédie de Genève , Geneva
Design Center Flacon , Moscow
Espace CENTQUATRE-PARIS – Diversion cinema , Paris
ESPRONCEDA – Institute of Art & Culture , Barcelona
Foundation of Venice – M9 – Museum of the 20th century , Venice Mestre
Open Laboratory of Modena , former AEM Central, Modena
Open Laboratory of Piacenza , former Church of the Carmine, Piacenza
HTC Corporation – VIVE ORIGINALS , Taiwan
INVR. SPACE , in Partnership with VRBB and supported by Medienbord, Berlin
Nikolaj Kunsthal, Copenhagen
PHI Center , Montréal
Portland Art Museum & Northwest Film Center , Portland
Stichting Eye Filmmuseum , Amsterdam

 

Hope to see you there!

The Future of VR

As the big platforms continue to promote certain content (games) and favor certain kinds of stories, we see a real need to continue to make boundary-defying works that might be considered controversial.   VR is such a powerful storytelling medium, that beyond “diversity” as a code word,  it is critical that different kinds of stories are supported and promoted. Why is VR so powerful?

I am going to argue that there are two key aspects which make it an especially powerful medium for storytelling–both of which relate to the concept of presence as put forth by Riva, et al, based on Antonio Damasio’s neuroscientific theories.

presence slide

  1. It is a spatial medium so the language of VR will be spatial (a language drawn from dance, installation and sound art, theater and performnce art, architecture, 3D video games, also film, which offers the illusion of 3D space and movement through the camera’s eye.
  2. It invites procedural storytelling and alternative forms of logic and physics including our relationship to objects in space.

I’m going to give a quick art history lesson because it important that to understand that seeing and moving through space is something that we are taught to do in a certain fashion, so much so that it seems natural, but it isn’t.

Raphael linear perspective

The discovery of linear perspective during the Renaissance defines for us what “reality” is supposed to be. Although we take this perspective to be natural , it is, in fact,  a mathematical abstraction of how we humans see. Linear perspective posits an infinite and homogeneous space that makes objects relate to each other through their position alone. (Panofsky) If you think about it, this is really a negation of psychophysiological space. Humans can’t see or even conceive of infinite space, moreover,  how we perceive a space depends not only on the kinds of objects in that space, that is to say what we attend to and ascribe importance to, but also how we move through that space.

Greek vase

Interestingly,  pre-Renaissance, the Greeks understood that  retinal images are spherical not linear so you get something like this.  I mean look at this—this is a really complicated space. And, I don’t know who this person is, but based on the size (look they are so big they have to kind of bend their neck to not hit the ceiling) I am going to predict that this person is more important than this other person.

oculus tuscan villa

Now, skip ahead a couple centuries—and guess what —we have a kind amalgamation of these two perspectives.  Because whereas Renaissance perspective assumed that we see with a single and immobile eye, in VR today we have a move towards stereoscopic images and also the ability to look if not move in 360 degrees.

Unity screen linear

But, even today, the de facto reality that you find in say Unity game engine is organized according to a linear perspective. Nevertheless, VR makes apparent the role of the mobile body in determining visibility and invisibility—what is seen and what is not seen..

 

queer phenomenology slide

As Sarah Ahmed notes: how we move through space is incredibly important. To me, this speaks not only to how I think about my own work in VR but also the power of VR to change how we think of  and construct reality. Unfortunately, graphic “realism” has become a goal in much VR. Not only, is this bound to be unsatisfactory (it will never be as good as the gold standard, which is reality). Moreover, it overlooks the true power of VR as a tool that reveals how we construct reality, as well as the ways we might construct reality differently.

So moving through an environment,  what kinds of information do we generally privilege as humans? Riva’s theories point to this. But, the answers were actually suggested by French philosopher Henri Bergson at the turn of the last century. He suggests that what we perceive is a universe of images—sound-images, touch-images, smell images, etc.  and that our consciousness privileges those that relate to the body and to memory.

For me, those are two touchstones for creating interactive narratives in VR. I understand that bodies are messy and, for many people in tech, it is just this possible escape from that messiness that appeals, but, as a working physician, not only do I think that is preposterous on many levels, it is a dangerous magical thinking. More than anyone I can think of working in VR right now, our work is about exploring this tension between material embodied reality and the desire to transcend it.

 

So, part of the Queerskins: ARK project, is a collaboration with Loise Braganza, a Mumbai based textile artist and fashion designer, with whom I worked to create “queer skins” –20 unique garments for our Queerskins: ARK installation that was to happen in conjunction with a physical installation. The idea behind this was to create soft machines, which would create an opportunity for potential wearers to viscerally consider their relationship to gender, sexuality, taste and physical body size/form, and to create novel forms of intimacy and connection with the storyworld/art and with each other. Since Covid- we have pivoted to a photography series, some of which you will be able to see at our installation in VRChat for the Venice International Film Festival VR Expanded exhibition. Here are just a few from Loise Braganza’s photo shoot with Shruti Viswan.  This dialogue between our embodied existence and transcendence through technology and storytelling is what needs to happen now. We know this is pushing not only buttons, but also boundaries of art-making and storytelling with VR.  Some of these photos will appear in our virtual installation that we are creating for The Venice International Film Festival exhibition. This will be an explorable, expanding “home” for Queerskins. We also see it as a stage for creating live virtual performances.

 

Queerskins: ARK VR selected for Venice International Film Festival

Excited to announce that Queerskins: ARK, co-produced by Cloudred and Intel Studios, which I wrote and directed,  will be exhibited at the Venice International Film Festival VR Expanded September 2nd-12th. ARK was created with my long-time artistic partner Cyril Tsiboulski, who is creative head of Cloudred,  a Brooklyn based interactive design studio, in collaboration with choreographer Brandon Powers.  ARK is the second of four planned VR episodes. In this, Mary-Helen, a devout Catholic Missouri mother, reads her estranged gay son’s diary, and allows herself to imagine him alive and in love. It features a transcendent and viscerally sexy and intimate dance between Sebastian, her son, and his lover, Alex.

 

Due to the COVID-19 pandemic, many people will not be able to travel to Venice this year because of travel restrictions. Venice VR Expanded will be exhibited in an entirely virtual format, placing all selected immersive content on VRChat, a social media app accessible, through VR headsets and PC laptops, to VIFF accredited viewers. Working with Beom Jun Kim who leads W.AK Studio, the Brooklyn based architecture and design practice, the Queerskins team will create an interactive “world” within VRChat that functions as an interactive art installation. Here, visitors can interact with objects linked to audio monologues from the interactive online narrative Queerskins: a novel. Wandering through the maze-like domestic space, they piece together the story of Sebastian.

Home In My Own Skin: Missing Pieces, Blue Velvet, and Donuts

 

Art is a virus–I only now recognized how much Blue Velvet  has influenced me, although Mayakovsky’s ear falling off in Pelin Kirca’s wonderful animation in our first interactive narrative  Reconstructing Mayakovsky was inspired by a combination of the opening scene of Blue Velvet (I put it in a slide show I made to show Pelin when we started collaborating on the animation) and Mayakovsky’s own play, with the characters missing body parts, Vladimir Mayakovsky: a Tragedy.

“All my life I have been looking for a place. Home. Sometimes, I’d see it in passing, driving down a street or jogging on the beach in Malibu. Sometimes I’d dream about it.

Sometimes it was a mansion and sometimes just a cabin in the woods. Though, I don’t think I have ever found it, I have found places where, at least for a time, I was loved.” Sebastian A.

 

I wrote Queerskins: a novel  as a gay man living in Missouri. The purpose was not to fool anyone, it was to kind of succeed and kind of fail, and in doing so, explore and transcend my own boundaries. It was a way of being at home in my own skin in a way that I never felt in real life. Many of the diary entries deal with painful parts and relationships from my “real” life remade and approached through the eyes of another.

Having recently found VRChat, I feel I have found my place in social VR. It’s like a Japanese anime world come to life. Imagine Toy Story + Faulkneresque stream of consciousness + many 12 year olds. I love it. It’s absolutely mad. Here is when I decided that who I really wanted to be was a donut man. Stay tuned–we are planning something special for Queerskins: ARK…More soon..