Progress–I fly out to Pittsburgh tomorrow to shoot 360 panoramas and room scan two 1950s rooms installed at the Heinz Historical Center. I am working with a grad student from CMU and a film lighting professional to get the best possible photo panorama to use as our VR environment. Also, ebay shopping for real objects that I will scan with my Structure Sensor and import into the game environment. These will then be used as part of an installation. Right now bidding on phone, t.v. and radio. I already got a phonograph with kids 45 records including “Mr. Bunny and the Rainbow”. Needs a new needle but turntable works!

Please follow on FB:

This week:

Spatial script for Atomic Vacation–nearly complete. Below is an image with only a few elements.  Audio is a major part of scene 1. Raw audio is all complete. Most video is edited, most gif’s have been curated.

Attended workshop on Kazuo Ohno’s work (Butoh dance) at The Japan Society with actress Tomi Heady (who plays Rae). Amazing mix of mime, robot, and human. This is going to inform our filming of the memory rooms for scene 1. I signed up for workshops at DCTV with the goal of filming Tomi in front of green screen in December and importing her into CGI environments.

Heinz History Center gave it’s okay for shooting 360 panoramas and room scanning their 1950s installation. This will hopefully happen some time in October. 

Here is a link to the short piece I wrote for The Kitchen blog.

Aug 29, 2016

If we define “analog” as a continuous variable which has no “truth” function, no negative, and no zero, and, “digital” as information composed of discrete values or states, then, moving from analog to digital requires not merely difference, but distinction. One is not equal to zero, human is not equal to machine, and there is nothing in between. Moreover, in so far as language involves digitizing our analog experience, whether we scratch a word on a stone tablet or “process” it with software, we have always been digital.

In We Have Never Been Modern , Bruno Latour argues that modern civilization has secularized rituals of purification to create boundaries between “nature” and “culture,” “human” and “thing,” even as we construct hybrid systems that mix politics, art, technology, and biology. Similarly, language participates in and perpetuates divisions even as it performs “the work of translation.” Electronic literature is uniquely situated to explore and reveal this. Although, like conventional literature, it offer users the opportunity to develop a critical awareness through content, electronic literature also reveals through form and process—by making manifest what Donna Haraway calls, “the translation of the world into a problem of coding”.

“Communications technologies and biotechnologies are the crucial tools for recrafting our bodies…Furthermore, communications sciences and modern biologies are constructed by a common move—the translation of the world into a problem of coding…” — Donna Haraway, The Cyborg Manifesto

Etymologically, the word “code” relates both to law and, through the Latin “codex,” to the book. As Marshall McLuhan points out in Understanding Media: Extensions of Man , written phonetic language offered early civilizations the potential for abstraction, universality, and transferability. He argues that without this, the “objective” disciplines of science and history could not have emerged. If the purity of these practices is now in question, (making us again, in Latour’s terminology, “pre-modern,”) this has occurred simultaneously with a revolution in communication. Looking at the origins of the word “code,” it is not surprising that games, with their rules that operate outside the law, poetry, with its use of language that operates outside of conventional syntax, computer code that creates and manifests as a language, machine-body gestures, via which we and our computers now read and write, and “books” that go beyond the page are key devices in electronic literature.

The salient question vis-a-vis electronic literature is not analogue vs. digital, but how we, as Haraway’s “cyborgs” or Marshall McLuhan’s “extended” bodies, communicate. Like Alan Turing’s “imitation game,” the importance of electronic literature lies less in who or what is doing the thinking and writing, (human, machine or both), and more in its capacity to procedurally explore our evolving relationship to language and, in so doing, to challenge the very notion of what “human” means.

Really excited to start shooting “Rae” for Atomic Vacation.
Something I have been obsessed with came again to me in postproduction of her melodramatic selfie farewell video to my little robot AI in space. I realized that I want to see how much “erasure” is possible to still transmit emotional content. Think emoticons.. Really wish I knew more about Noh. This from Wikipedia–“Emotions are primarily conveyed by stylized conventional gestures while the iconic masks represent the roles such as ghosts, women, children, and old people.” I am interested in thinking about facial recognition –not in terms of surveillance but in terms of an economy of means
for visual language in VR. Cyborg bodies have processing times that will require that we develop a gestural language in the broadest sense. Here are some stills from the video which was shot as a selfie farewell to Shizuku by the actress Tomi Heady, herself. In the last photo, the kanji symbol for Shizuku becomes a substitute body for her. Did not even realize that would happen when I wrote it in the script….Even though the mask covers an actor’s facial expressions, the use of the mask in Noh is not an abandonment of facial expressions altogether. Rather, its intent is to stylize and codify the facial expressions through the use of the mask and to stimulate the imagination of the audience. By using masks, actors are able to convey emotions in a more controlled manner through movements and body language. Some masks utilize lighting effect to convey different emotions through slight tilting of the head. Facing slightly upward, or “brightening” the mask, will let the mask to capture more light, revealing more features that appear laughing or smiling. Facing downward, or “clouding” it, will cause the mask to appear sad or mad.
I think in the future, hard data will be the work of finite computing machines, is it too idealistic to think the work of cyborg humans will be poetry?

As Oculus Launchpad fellow, I could submit for “scholarship” funding. Cyril Tsiboulski, my longtime collaborator decided to learn Unity for this project. In a few short weeks, he got an environment up and running with head position tracking and interactivity and spatial sound.

I just got to see the proof of concept video. Those that know me know how long I have been working on AV. To see it starting to actually take shape is amazing. I posted this in our developer forum, but I am going to post it here, too, because it really is what I feel. Really. I work on huge complex projects and sooo sloooowly. A slow, slow burn. In service of Atomic Vacation, I have traveled to Japan, and North Korea, and the American South-west to visit sites of nuclear warhead storage and testing, I have read hundreds of books –from Japanese 1950’s scifi noir, to linguistics, to natural language processing, books on the tourism and nationalism and the relationship between 1950s domestic design and cold war propaganda, and John Hershey’s Hiroshima and on an on… I have thought, written, rewritten, planned, schemed, destroyed, recreated, Skyped at 1 am with the head of a Japanese robotics lab about AI and what it means to be human and what that has to do with poetry, I’ve had people tell me I would never be able to do this, I’ve had artists tell me it’s the most amazing project they’ve ever heard of, I’ve taken conversational Japanese, and classes on shooting 360 video, and did stealth filming in a motel in upstate New York, and got into that zone where I don’t need to eat or pee or sleep. And, I fell in love with a little robot girl, who maybe exists right now or maybe is about to be born, and who just may save all of humanity.
And, because of Oculus Launchpad, I get to work with my incredible collaborator, again, Cyril Tsiboulski who has been with me from the beginning back in 2008 when I put out my first massive narrative interactive project–90,000 words of text, a soundscape, a manifesto, a hand drawn animation, a fake investment video, a concrete poetry machine, proposals for performances that will never be performed, an self-destructing archive. The Kindle hadn’t even come out yet, and no one had any idea what I was doing. And, through it all,Cyril never raises an eyebrow, but takes in everything and then makes something beautiful that works and moves people. He doesn’t ask why I need to scan a circa 1980’s book “How to Talk to Your Cat and Make it Do What You Want,” so I can pair it with archival footage of world war one soldiers with neurotic conversion disorders from PTSD that walk like robots. He just gets it.
Tonight, I saw that again. What we made is so weird and provocative and seductive and disturbing… The phone rings in this slightly blurry, dreamy 1950’s living room, it jangles the nerves, so you pick it up. and there, on the line is my little robot girl talking to you from a near post-apocalyptic future, accusing all of humanity of being tourists on their own planet. Atomic Vacation.. It’s gonna be great.

I knew we should have marketed it as a game–but that was 2008. Now, the definition of game has expanded. Still, happy to be next to The Stanley Parable.

ENG 1131

Writing Through Media: Electronic Literature

Caleb Milligan

A variety of platforms and genres will be represented in our readings. On the page, we will encounter Salvador Plascencia’s digitally progressive print novel The People of Paper. Through electronic interface, we will experience hypertext fictions like Judd Morrissey’s The Jew’s Daughter, generative poetry like Nick Montfort’s “Taroko Gorge,” narratively experimental games like Davey Wreden’s The Stanley Parable, and Illya Szilak’s new media “novel” Reconstructing Mayakovsky.

“We would like to include your work, Reconstructing Mayakovsky (2008), in our upcoming exhibition “Shapeshifiting texts”, taking place during the International Conference on Digital Media and Textuality, from the 3^rd to the 5^th of November 2016, in Bremen, Germany. The Exhibition “Shapeshifting Texts” will be curated by Daniela Côrtes Maduro in collaboration with the ongoing project “Archiv der deutschsprachigen elektronischen Literatur” (ADEL), developed by Professor Peter Gendolla and Professor Jörgen Schäfer. It will feature electronic literature and experimental works collected by several archives within the Consortium on Electronic Literature (CELL). ”

Also, RM got a thoughtful analysis in The Futurist Yearbook–

Decoding the DNA of Poetry: Reconstructing Mayakovsky in the Digital Era.

After my BANFF digital narratives residency and attending Oculus Launchpad, I pitched the project to my long time collaborator Cyril Tsiboulski of Cloudred Studio. He offered to work on the project, learning Unity in the process. So excited to be working together again. We will applying for the Oculus Launchpad “Scholarship” fund to develop the project. Right now seems a combo of 360 video and CGI.  Storyboarded it last week. Here’s a sample.

At BANFF, I had the opportunity to collaborate with master animator Sasha Stanojevic. We made an animation with my text spoken by a little girl text to speech program and sounds of Kepler courtesy of NASA. It’s beautiful. So excited to get working on this.

Atomic Vacation is a narrative game in VR, a journey through database landscapes—the visual representation of stored memories. These memories belong to Shizuku, a robot girl from the near apocalyptic future who is trying to recover the deleted history of her  former life on Earth. By retrieving these memories for her, you enable Shizuku to reconstruct and recount her story to you. In journeying through,  the user will  have the opportunity to consider the question: as our experiences become more and more computer-mediated, what or who passes/counts as human? 

Each landscape is a thought experiment of sorts. Each organizes and reveals information differently. It is expected that you will feel uncomfortable with some of these puzzles for which there is not one correct solution.  

Please contact me for more information or for collaboration/funding/exhibition opportunities.

May 21st. Yes.