University of Southern California Dissertations and Theses

Shayd: the pursuit of magic, illusion, and interactive worlds

(USC Thesis Other)

Shayd: the pursuit of magic, illusion, and interactive worlds

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content SHAYD: THE PURSUIT OF MAGIC, IMMERSION, AND INTERACTIVE WORLDS by Juliana Christine Griffo A Thesis Presented to the FACULTY OF THE USC SCHOOL OF CINEMATIC ARTS UNIVERSITY OF SOURTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree MASTER OF FINE ARTS (INTERACTIVE MEDIA) May 2012 Copyright 2012 Juliana Christine Griffo ii Table of Contents Acknowledgements ............................................................................................... iv List of Figures....................................................................................................... viii Introduction........................................................................................................... 1 Epigraph................................................................................................................ 2 Abstract................................................................................................................. 3 Chapter 1: Advances in Human Technology 1.1 Technologies Change, The Goals Remain the Same ............................. 4 1.2 Platformers or Platforms?..................................................................... 6 1.3 Histories and Definitions, Part I ........................................................... 7 1.4 “Help Me Obi-Wan Kenobi, You’re My Only Hope!”.......................... 9 1.5 “With the Safeties Off, Even a Holographic Bullet Can Kill.” .............. 10 1.6 Desperately Seeking Holodeck............................................................. 13 1.7 The Allure of the Holodeck.................................................................. 14 Chapter 2: Alternate Dimensions 2.1 Histories and Definitions, Part II.......................................................... 17 2.2 Dragon Slaying From The Living Room .............................................. 18 2.3 We are Physical Beings! ...................................................................... 19 2.4 Inspirational, Alternate, Immersive, Interactive Worlds and Spaces...... 20 2.5 Theme Parks are Themed Worlds......................................................... 22 2.6 Interactive Theater ............................................................................... 24 2.7 Public Interactive Installations ............................................................. 25 2.8 Histories and Definitions, Part III......................................................... 26 2.9 Building a Holodeck From Scratch....................................................... 28 Chapter 3: Discover Shayd 3.1 Goal #1: Total Immersion ................................................................... 29 3.2 Goal #2: Alien World.......................................................................... 30 3.3 Goal #3: Seems Like Magic ................................................................ 31 3.4 Introducing Shayd................................................................................ 33 3.5 Syyrette & Maubwe Walk You Through the Uncanny Valley Unscathed ............................................................................................ 38 3.6 The Making of Syyrette: Digital Puppetry........................................... 39 3.7 Hitting the Road… or Not: Why Can’t Shayd Be Installed Everywhere? ........................................................................................ 40 3.8 Designer’s Dream, Designer’s Nightmare ............................................ 41 iii Chapter 4: It was Just One Thesis Project, Wasn’t It? 4.1 Summer School.................................................................................... 42 4.2 Geared Up for Winteract...................................................................... 45 4.3 Shayd Children’s Book ........................................................................ 46 4.4 Shayd Mobile....................................................................................... 48 Chapter 5: From One Little Installation to What Lies Beyond 5.1 Possibilities for Shayd to Improve in Near Future ................................ 48 5.2 A Lifetime to Shayd on the Holodeck?................................................. 49 5.3 The Shayd Children’s Book: Vague Adaptation, Expansion, Transmedia?......................................................................................... 50 5.4 Syyrette & Maubwe: Smart Enough to be C-3PO & R2-D2 in Shayd? ............................................................................................. 54 5.5 Shayd Team, Fall 2011-2012: “Original” ............................................ 55 5.6 Is This Interactive Media?.................................................................... 56 5.7 Why Do We Need This? ...................................................................... 57 Chapter 6: What’s Next? 6.1 A Year of Immersion in Immersive Technology is Not Enough............ 58 6.2 Shayd Needs to Be Tested on a Larger Scale for Better Design Feedback.................................................................................. 59 6.3 Shayd is Limited By Funds, Team Size, and Deadlines ........................ 60 6.4 So Where’s My Holodeck? Should There Be Such a Thing and What Is It Now?................................................................................... 60 6.5 Content Gives Meaning to Technology ................................................ 64 6.6 Of Course, All Of This Requires Further Research............................... 65 Glossary ................................................................................................................ 68 References............................................................................................................. 71 Appendices Appendix A: Shayd Tormee/Wah-zay Side (Virtual World)...................... 74 Appendix B: Shayd Zoop Side (Physical Installation) ............................... 75 Appendix C: Zoop .................................................................................... 76 Appendix D: Tormee ................................................................................ 77 Appendix E: Wah-zay............................................................................... 78 Appendix F: Nad’l .................................................................................... 79 Appendix G: Syyrette ............................................................................... 80 Appendix H: Maubwe............................................................................... 82 Appendix I: Shayd Children’s Book Excerpt, Pages 1-12.......................... 83 Appendix J: Shayd Children’s Book Excerpt, Last 3 Pages ....................... 85 Appendix K: Tormee Language Lyrics ..................................................... 86 Appendix L: Shayd Mobile....................................................................... 87 iv Acknowledgements I would like to extend deep gratitude to faculty, staff, and students in the Interactive Media Division, the School of Cinematic Arts, and the University of Southern California as a whole for their support and collaboration in making Shayd a reality. Thank you to my thesis committee, Mark Bolas, Scott Fisher, and Joseph Garlington. Mark, thank you for seeing a spark in my work and taking a chance on a storyteller with no engineering experience. Thank you for introducing me to the Wide 5, offering me the use of HMDs belonging to your Mixed Reality Lab, and having faith in me to figure this stuff out and even pass along the knowledge. Your genius and your quirks are inspiring and you have been my biggest supporter for three years. I couldn’t imagine pursuing this degree without your support. Scott, thank you for shaping this program and always readily agreeing to listen to any crazy ideas I have. Thank you for agreeing to be on my committee before even hearing my proposal. And most of all, thank you for all of your support not only during my thesis but for three years. Your feedback and prior knowledge of mixed reality (because, like Mark, you were part of the creation of it all!) greatly influenced my work and this paper. Joe, thank you for agreeing to be on my committee before having met me. When you agreed to be a part of my thesis year after a blind e-mail and a little trailer video, I jumped up and did a dance. Thank you for all of your amazing insight, feedback, time, v support, and even enduring my nervous first presentation of Shayd (I barely finished shaking your hand before I stuffed you into that HMD, entirely not what I had planned!) so graciously. Thanks also to Jeremy Gibson and Laird Malamed for all the great advice and feedback as professors of our Master’s Thesis class this year. Thanks to Adrienne Capirchio, who we are all lost without, for advising us so well for three years and helping us through the process of pursuing our degrees. Thanks to Andrew Sacher for agreeing to produce our thesis show this year and ensuring that it will be a phenomenal show. Thank you to the incredible powerhouse people who are part of the Shayd team: Chris Chamberlain, James Iliff, Chao Huang, Logan ver Hoef, Jeremy Tisser, Madison Orgill, Sean Najarro, and many more. Thanks especially to James who spent many, many days pouring over every detail, creating our website at www.DiscoverShayd.com, and eagerly taking on new projects constantly – I’m overjoyed that someone can “get” my style and create beautiful worlds like I see in my imagination. Thank you to Chao for sticking it out all year, in spite of our team’s mysterious inability to retain programmers, and making the world of Shayd happen no matter what problems we threw at you. Special thanks to Jeremy for putting together the many, many people it takes to record live music and writing an incredible amount of gorgeous, original music for Shayd – all while somehow maintaining an enthusiasm that’s contagious. Thank you to Logan for lovingly working on Syyrette and Maubwe and getting them just right, they are so amazingly like what I see in my head. Thank you to Chris for helping to produce this monster of a project, particularly while working two other jobs in your first year of this vi program. Thank you to Sean for riding in to right our animation crisis with his amazing motion capture skills. And thanks to Madi for agreeing to design and construct Shayd’s physical cave and props and saving the day by adding lots of set design and art experience to our team just in the knick of time. Special thanks to all of the musicians, singers, and sound technicians that performed or were part of the music for Shayd. Thanks especially to Jeff Tinsley for arranging all of Jeremy’s lovely music and working so hard to keep my crazy lyrics intact as much as possible. I could hardly contain my excitement watching the music be recorded and hearing such amazing music. Thank you to everyone involved. I’ve never seen music recorded live before and it was absolutely a fantastic experience. Thank you to the Mixed Reality Lab at the Institute for Creative Technologies for all the equipment loaned to Shayd. Thank you to the people there who supported the hardware, software, and everything in between. Thai Phan, for being the most patient technical adviser and incredible rain dancer and also for letting me record you repeatedly for documentation and how-to videos, Palmer Luckey, for making amazing HMDs and helping set up hardware, Evan Suma for writing all the scripts to make Palmer’s HMD talk to Unity and look just right, Evan Suma, Nathan Burba, and Robyn Tong Gray for being open with support for multiple Kinects, and of course Mark Bolas for allowing me to be a part of the MxR and putting us all in a position to work together. Thank you to Eric Furie for allowing Shayd temporary housing in the Motion Capture Lab on campus and Dan Weishaupt for all of the agile ladder-hopping and set up vii in there. Thank you to Dan Weishaupt and Laird Malamed for championing the effort to find Shayd a home in spring semester and helping to make Shayd a possibility. Thanks to my classmates of three years, who I also count as friends. Especially thanks to those who took extra time out to give individual advice for Shayd, Michael Annetta, Jay Bulvanoski, and Andy Uehara. Thanks to my family and friends for support, encouragement, and patience. Thanks to my husband, Ryan King, for listening when I needed to talk and for getting out of the way when I didn’t want to talk. Thanks to my mom and dad Joanne and Anthony Griffo for listening and supporting me. Thanks especially to my mom for editing this paper. Thanks to my aunt Annette Ferrari for agreeing to try to illustrate the Shayd children’s book in a short amount of time. Thanks to my whole family and my Canadian in-laws for just being supportive of heading to a three year graduate program on the other side of the continent. Thanks also to my friends, in particular Vanessa Lema, who rarely went a week without becoming a human sounding board and Ryan King, who graciously starred in some video documentation for me. viii List of Figures Figure 1: Still image from the Lumeire Brothers’ film, “Arrival of a Train at La Ciotat.” 4 Figure 2: Still image from the Georges Méliès’ film, “A Trip to the Moon.” 5 Figure 3: Still image from the Georges Méliès film, “A Trip to the Moon.” 6 Figure 4: Still image from the James Cameron film, “Avatar.” 6 Figure 5: Screenshot of Tetris played on original Game Boy. 7 Figure 6: Screenshot of Tetris in Game Over state on original Game Boy. 7 Figure 7: Image of a Chess board. 7 Figure 8: The Mixed Reality Lab’s Wide 5 HMD. 8 Figure 9: Still image from the George Lucas film, “Star Wars.” Obi-Wan Kenobi viewing the holo-recording of Princess Leia played back by R2-D2. 9 Figure 10: Still image from the George Lucas film, “Star Wars.” Luke Skywalker accidentally discovering a piece of the holo- recording of Princess Leia in R2-D2’s memory banks. 9 Figure 11: Still image from the George Lucas film, “Star Wars.” Princess Leia finishing her holo-recording and saving it in R2-D2’s memory banks. 9 Figure 12: Captain Janeway and the Doctor on the Holodeck in the series “Star Trek: Voyager.” 10 Figure 13: Still image of an empty, earlier model (prior to Star Trek: Voyager) of the Holodeck. 10 Figure 14: Jean-Luc Picard and Lily on the Holodeck from the film, “Star Trek: First Contact.” 11 ix Figure 15: Captain Picard shooting Borg on the Holodeck with a holographic machine gun. 12 Figure 16: Paul Milgram’s Reality-Virtuality (RV) Continuum. 27 Figure 17: Screenshot of Glow World, created by Juli Griffo in Spring 2011, which can be seen in 3D and played in conjunction with a Microsoft Kinect. 30 Figure 18: Stuffed rabbit that embodies Madelyyne’s spirit from home. 32 Figure 19: Inside the Cave in the alternate dimension of Shayd. 34 Figure 20: Syyrette on left; Maubwe on right. The main characters of the Shayd project. 35 Figure 21: The Clearing in the alternate dimension of Shayd. 36 Figure 22: The Hollow in the alternate dimension of Shayd. 36 Figure 23: Flying over the alternate dimension of Shayd near the Cave. 37 Figure 24: Maubwe (left) and Syyrette (right) spend time together perched on a large branch in the alternate dimension of Shayd. 66 1 Introduction I am in my third and final year working towards my M.F.A. in Interactive Media at USC. I came here with no engineering skills, no programming skills, no design experience, some cinema and photography experience, and four full-length feature screenplays, one television spec script, and just a ton of pages of short stories, novels, and poems. I was a writer at heart. In the three years that I’ve been here, I’ve made an amendment to that, though. I am really a storyteller. And the platform for the storytelling doesn’t matter, because I can tell stories anywhere and in any format. My ultimate goal became a way to unleash my imagination entirely around my audience (I say audience, but I mean reader, viewer, player, user, visitor, guest, child or “big kid” playing with a toy, gamer, schemer, dreamer, etc.). This project was about this pursuit, but the paper covers only the journey of my thesis year, and not what led up to it. It leaves out lots of hard work to get to this point, like building a robot in my first year here, playing with the Kinect, learning and loving Unity3D, my love and creation of multiplayer cooperative games (particularly with a survival motivation), or my obvious desire to create cute characters, worlds, and robots to compensate for my lack of drawing, painting, and modeling skills. This year I learned I am decent with clay modeling. I’ve learned a lot and been inspired by even more, but my life experiences have all led up to this thesis and have all pointed me in this direction. When I got to my thesis year, I knew what I wanted to pursue for an entire year. Now, on to the thesis journey… 2 Epigraph “The only way of discovering the limits of the possible is to venture a little way past them into the impossible.” -- Second of Clarke’s Three Laws Arthur C. Clarke 3 Abstract The path to reaching the ultimate in immersive, alternate space is long and broad. Mixed reality experiences already exist, but more engaging content is needed to help drive forward innovation and reach the ultimate goal of what currently seems an impossible dream – a physically immersive world that can be controlled by technology to interact with users in ways that appear to be magic and can include alien landscapes, alternate physics, highly sophisticated artificial intelligences, and consumables like scents or food. Until this ultimate technology is achieved, designers, engineers, artists, and storytellers must work together to create experiences which utilize illusion and elements of everything from theater sets to fantastic visual displays. Creating an illusion that feels highly immersive requires lots of design planning, iteration, and engineering; but it should make interesting content available to the public and help to encourage other designers and engineers to continue to push the current limits of technology in future to keep moving towards the epitome of immersive experiences. 4 Chapter 1: Advances in Human Technology 1.1 Technologies Change, The Goals Remain the Same In 1895, the Lumiere Brothers began showing their film creations to the public. The story goes that when they showed the Arrival of a Train at La Ciotat, people were so convinced of the reality of the train that they jumped up and fled the theater, believing that a train was really arriving mere feet in front of them. (L’Arrivee) While this story is generally agreed upon as false (L’Arrivee), one thing clearly did happen in 1895: technology advanced to enhance something that humans already had. Though humans didn’t previously have films, humans already had the capacity to Figure 1: Still Image from Lumiere Brothers' film, "Arrival of a Train at La Ciotat." 5 see the world and remember. Humans can see the world without a frame on a flat screen, in color, and not in 2D. Deep down at the core of these advances in technology, the concept is always the same: more immersive spaces that tell a story and create an illusion of reality. Georges Méliès, an illusionist and early filmmaker, was a dedicated storyteller who was responsible for much of the advancement in special effects starting in 1896. His artistic and beautiful sets made use of his previous experience as a magician. He even had his black and white films hand-painted to create color films, like A Trip to the Moon from 1902, which is also the film with one of his most famous images – the moon with a rocket ship that landed it’s the eye of the human-like face on the moon. (Georges) Figure 2: Still image from the Georges Méliès’ film, “A Trip to the Moon.” 6 Whether it’s the train arriving at the station (Arrival of a Train at La Ciotat), a dream-like voyage to the moon (A Trip to the Moon), or the tale of the first interactions between humans and blue giants on a Jupiter moon (Avatar), the technology is always just trying to keep up with the story to create media that is highly immersive, no matter what technical boundaries might be pushed in the meantime. 1.2 Platformers or Platforms? Media comes in many shapes and sizes. While interactive media may be a much wider field that often has to battle with naming itself, interactive media is in some ways an evolved state of film. Film is a partial illusion with passive storytelling, while very often interactive media is an illusion with agency. Whether it is a board game, a digital, screen-based game, interactive theater, or a ride at a Disney theme park, interactive media still seeks to create an immersive narrative space while addressing visual and spatial cues for an audience. This concept can apply to chess or even Tetris. Chess: A King, a Queen, pawns, knights… it’s clear they have a narrative space on the board. Tetris may seem like it isn’t narrative, but isn’t it essentially “the end of the world” if the spatially Figure 3: Still image from the Georges Méliès film, “A Trip to the Moon.” Figure 4: Still image from the James Cameron film, “Avatar.” 7 oriented blocks fill the whole column and the player the “god” of this world? This smells like narrative and it certainly engages the player in an immersive space. Interactive media comes in many forms, beyond digital games on a screen or board games with physical pieces, and draws on tools from a wide range of fields, from theater to architecture. These tools, no matter the advancements or “shiny” factor, are all used to manipulate media in an interactive way to tell a story and construct an alternate space. These tools, while each merits exploration and analysis, are not the object of this paper – it’s about the magic that these tools help to create. 1.3 Histories and Definitions, Part I Head Mounted Displays are, at their most basic function, an advanced form of a “traditional” film. There are two individual displays, generally in lenses that are not flat displays, which display two separate images in each eye, which is mounted on front of the user’s eyes, supported on a band or helmet on the user’s head. The user’s eyes see Figure 5: Screenshot of Tetris played on original Game Boy. Figure 6: Screenshot of Tetris in Game Over state on original Game Boy. Figure 7: Image of a Chess board. 8 two slightly different images of the same scene or object. From there, the human vision system takes over and interprets the two slightly different images as a three dimensional scene or object. (Head-Mounted) This is a stereoscopic image, which is also generally referred to as simply “3D.” Think back briefly to the Lumiere Brothers and their moving images: 64 years previously, in 1832, Charles Wheatstone invented his stereoscopic viewer (Bimber, 3) by creating a viewer that had two slightly different images that human brains interpret as three dimensional, much like View-Masters of today. View-Master started in 1911, but became mass manufactured and popularized in the 1950’s and in the 1960’s when they began marketing for children. (View-Master) Head Mounted Displays have been around for quite some time, though they have never been widely popular at the consumer level. Ivan Sutherland’s Head Mounted Display created in the 1960’s was the first in pursuit of what he called an “ultimate display” that “would, of course, be a room within which the computer can control the existence of matter.” (Bimber, 2) He used a method in his first HMD to augment a user’s reality, making it so that the user could see both computer generated images and the physical world that the user was in. Thus, HMDs set out on the path forward. (Bimber, 3-4) Figure 8: The Mixed Reality Lab's Wide 5 HMD. 9 1.4 “Help Me Obi-Wan Kenobi, You’re My Only Hope!” Who could watch Star Wars for the first time and not come away with a lasting image of Princess Leia being displayed by R2-D2 in a holographic message to Obi-Wan Kenobi? (Star Wars Lucas) R2-D2, an autonomous artificial intelligence, highly mobile in his little droid shell, can record and playback a holographic image – and a holographic image that excludes anything that isn’t the subject of the scene, no less! Figure 9: Still image from the George Lucas film, “Star Wars.” Obi-Wan Kenobi viewing the holo-recording of Princess Leia played back by R2-D2. Figure 10: Still image from the George Lucas film, “Star Wars.” Luke Skywalker accidentally discovering a piece of the holo- recording of Princess Leia in R2- D2’s memory banks. Figure 11: Still image from the George Lucas film, “Star Wars.” Princess Leia finishing her holo-recording and saving it in R2-D2’s memory banks. 10 Science fiction fans have been ready for total immersion for a long time. Fans want to forget reality and be completely encompassed by another world. This desire leads to more dreaming, and the more dreaming about alternate worlds, the more fuel for developing technology to make those dreams come true. Fans and science fiction writers alike are more than happy to continue to think about the future, the magical advances in technology in it, and how those amazing things can change the world. If not change the world, science fiction writers and storytellers of all genres need no assistance in dreaming up stories and worlds that are compelling in which audiences would love to become a part of. 1.5 “With the Safeties Off, Even a Holographic Bullet Can Kill.” Even adamant Star Wars fans who could debate tirelessly that Star Wars is better than Star Trek would have to admit: Star Trek’s Holodeck is a pretty cool idea. Seen Figure 12: Captain Janeway and The Doctor on the Holodeck in the series Star Trek: Voyager. Figure 13: Still image of an empty, earlier model (prior to Star Trek: Voyager) of the Holodeck. 11 often in the later generation series in Star Trek, the Holodeck is a special computer controlled environment that is a blank room at first, but can be turned into anything by the manipulation of photons. These photons are not only controlled to create an illusion; they can be made to feel like the object or creature they are mimicking. Very advanced AI can inhabit a photonic entity and behave according to its programming; the entity can be convinced of its life, for example, as a simple farmer of the past and ignore any technology or strange remarks from the human participants in the “holonovel.” (Star Trek) The Holodeck is memorably featured in the Star Trek movie First Contact. Captain Jean-Luc Picard of the Federation Starship Enterprise ends up interfering with the past while simultaneously saving it from the Borg during a battle to protect Earth. His ship is orbiting Earth of centuries in his past, which is crawling with Borg drones who are rapidly taking over, and he is cut off from his crew with a woman from a different century at his side. Lily was saved by the Jean-Luc’s medical staff but was then separated from them. Though she’s a major scientist in her time, the Enterprise (and the Borg) are confusing and terrifying. (Star Trek) Figure 14: Jean-Luc Picard and Lily on the Holodeck from the film, “Star Trek: First Contact.” 12 But even a person disoriented by being displaced in time is able to keep up with Jean-Luc with very little information about Holodeck, following his lead as chapters of a mobster story change the room around them. It looked and felt real to Lily, in spite of whatever magical force that might be creating the illusion, so it was hard to misunderstand that the space around her was changing and interacting with her. Very quickly, her fears swung back to the Borg pursuing them. Once engaged in the new scene on the Holodeck, it just made sense what was happening to her and around. (Star Trek) The Holodeck is a magical room that can manipulate photonic energy. A user can interact with artificial intelligences, touch and change the scenery or objects around them, and cannot get seriously harmed with the safety protocols turned on. Even more amazingly, with those safety protocols turned off, a holographic bullet can kill just the same as a real bullet. As Jean-Luc explains to Lily after shooting several Borg pursuers to death with a holographic machine gun, “With the safety protocols off, even a holographic bullet can kill.” (Star Trek) Figure 15: Captain Picard shooting Borg on the Holodeck with a holographic machine gun. 13 Ultimately, the Holodeck successfully blurs the lines of reality beyond any technology humans currently have today. Ivan Sutherland would likely jump at the chance to test such a thing out! 1.6 Desperately Seeking Holodeck Now that technology has advanced to give media-makers some finer control over the creation of immersive experiences, we should all be able to ward of the zombie apocalypse or visit the setting of a fantasy novel from our living rooms, right? Sadly, seekers of immersive story worlds are still not going to get to enjoy slaying dragons from a perch on the back of the couch while wielding a pillow for some time. Humans can still use their imaginations for these things, perhaps enhanced by some games or technology currently in existence, but it is still not at the level of sophistication of the Holodeck. Currently, the closest advances in technology are making big steps towards a Holodeck, but there are still many more big steps to get there. Technical problems abound: creating and manipulating matter, being able to manipulate the physics of a space, creating highly adaptable artificial intelligences that can learn or not learn whatever we want in real time. We are not able to do any of that just yet. What we can do is create autostereoscopic images (stereoscopic images that do not require special wearable viewing devices), use advances in 3D data-gathering cameras like Microsoft’s Kinect to interpret a human “controller,” or track larger physical spaces with many motion capture cameras tracking a user wearing a head mounted display. We can 14 currently make pretty great visual illusions, but we have little ability to create matter from nothing in an instant. We can physically manipulate space to coincide with sophisticated visual illusions if carefully planned out like a magician might secretly physically manipulate an illusion. Creating the feeling of a Holodeck before we actually build a working Holodeck takes a lot of design, planning, trickery, luck, hope, technology, frustration, tears, imagination, and at least a pinch of magic. Making what feels similar to a Holodeck requires more than stereoscopic viewing technology, more than motion capture, and more than motion tracking. It also could use a really good story world that helps blend away anything in between that might jar the user from the immersive illusion. 1.7 The Allure of the Holodeck In a conversation on March 12, 2012 between myself and Mr. Joseph Garlington, Garlington asked me what it was I liked about the Holodeck. I answered that it appears to be magic and you can easily forget that it is actually technology creating the world. Garlington was satisfied with this reasoning and continued to discuss with me his own theories of how people currently perceive the world and how designers have two different paths to take depending on the kind of experience they want to make. (Garlington Conversation) “Garlington Theory:” I think that we tend to reference stuff in two ways: I think sometimes we reference to our whole body and sometimes we reference just to our head. And when we’re … in a place like wearing an HMD or you’re in an attraction or something like that, it is about referencing the whole body. … The scale of things 15 I reference to my whole body so things feel big or little relative to my body. … When we look at models [physical models] we tend to … I don’t think reference our whole body. We just kind of more reference our head because scale changes in the model. So there’s a point and … when all of a sudden the reference point changes. And when that reference point changes, then you feel immersed differently. …When you look through a peep hole… then you’re doing the head reference thing. (Garlington Conversation) Garlington continued to describe what sounded to me like a bit of Professor Mark Bolas’ talk at the VES Immersive Experiences conference on Feburary 25, 2012. I brought up that peep holes, an example Garlington had made, particularly in reference to a virtual world represented on an iPhone, are frames and that Bolas had been talking about frames and how they are clear borders between virtual and physical, which he spoke of in the section of his talk about “mixed reality.” (Bolas Immersive) Garlington made a point that frames have clearly changed from even two hundred years ago and that film has certainly had a huge impact in that regard. Before film, pictures were often in large and ornate frames to show “picture’s in here, real reality’s out there” to those in view of it. (Garlington Conversation) Frames have blended over the years from illusions that might use a lack of a frame to create a trick to the newest technology, designed to have screens that come so close to the end of the object (whether a laptop, a phone, or a tablet) that the user might forget where it is, becoming ever more immersed in the tech toy’s world. Understanding some of how we currently see the world and some of the illusions we already encounter, sometimes even in our own homes, helps support design methods for creating large scale illusions like the Holodeck. This means a fully immersive physical space that encompasses someone completely, where imagination can be made to 16 move on its own, without thinking about the mechanical or technological methods of that movement. A story that can completely become alive all around someone to explore in, role play in, imagine in, work in, study in, theorize in, even sleep in – there is no other word reference for this kind of immersion except “life.” Holodecks could then be defined as “alternate life spaces” that can unleash all of your imagination and more. Technological limits aside, the only limit would be one’s own imagination. What parts of a Holodeck are possible right now? What we can make is a virtual reality experience contained in a physical world designed completely for physical immersion and with the nature of human physicality in mind – a change of smells and air, temperature, climate, weather. All of these things are in reach today. We use all of these things to our advantage in show-like spaces all the time. A store will control the smells and temperatures inside. A Broadway play might recreate some special effects of weather. A used car salesman might dress their lot like a warmer climate for an early bird summer sale when there is snow outside and convertibles are not in high demand. We build houses and buildings designed for humans to move through and utilize space in already. We have restaurants and grocery stores and cooking TV channels to address the human need and sense of taste. Unless experiencing no gravity, humans are always in contact with something, whether standing, sitting, sleeping. We can and do dress flooring to change the feel of a space. We have stores filled with various beds and bedding and couches and other furniture so we can have somewhere to sit. We already have all these methods, but when we design for immersive and interactive experiences, we don’t always think about what we’re already capable of. 17 Garlington had described how no matter how immersed you might be in a film happening on the screen in front of you, you’ll always know who you are. The tools that we use are ones that make it more efficient and easier to get into that fun place, despite the fact that we know that we’re not there. … This takes you there in a certain way, and if you were designing… to me to design a world that is best viewed through a telescope or a periscope or a peep hole is different than designing a world to stand in the middle of. … Somebody says why can’t you do this on an iPhone, yes you can, but then you want to develop the vision of that world differently because it’s more about exploring… I think … when you’re looking through a peep hole, it’s as important what you don’t see as what you do see. Because it’s the mystery of what surrounds the little view that you’re given. When you’ve got the world completely around you like when in an HMD, then it’s less about the mystery of what you can’t see than it is exploring the voluptuousness of what you can. (Garlington Conversation) Garlington continued on about the challenges of technology combined with the HMD and also advised that it can be up to the producer or project director to figure out which problems should be dealt with by engineers or by artists (Garlington Conversation). It seems imperative that in reaching for a Holodeck that designers, engineers, artists, and creators of worlds always keep everyone else in mind as a resource for problem solving and moving ever closer to a Holodeck. Chapter 2: Alternate Dimensions 2.1 Histories and Definitions, Part II A Head Mounted Display can be used in conjunction with digital worlds. Some HMDs have been paired with other technologies to enhance them. For instance, paired with a motion capture system, an HMD can be used as a 3D viewer where the user is able to walk around virtual scenes. In the Mixed Reality Lab, part of the Institute for Creative Technologies at the University of Southern California, the Wide 5 HMD has one of the 18 largest tracked spaces in the world. At a space of approximately 30 feet by 30 feet, it uses 52 motion tracking cameras from Phasespace to track the space. The lenses on the Wide 5 are so large, they allow for about a 150 degree angle, which is very close to the range of human vision. (Bolas Conversation) While experiencing the Wide 5, it is difficult to pay attention to where the image ends. With a system like the Wide 5 and its motion capture system, more than just the user can be tracked. This allows for “mixed reality” experiences, such as tracking a chair in the physical world that is paired to a virtual chair. Being able to actually sit on the virtual chair or see it virtually morph while sitting on it is an experience that “just a screen” couldn’t pull off on its own. 2.2 Dragon Slaying From The Living Room Why don’t we have these mixed reality experiences in our living rooms already? To some degree, we already do have mixed reality in our homes. The Microsoft Kinect is capable of taking images and blending them on the screen for a user, or matching a virtual avatar to hang out with you in a screen image of your living room. Currently, A Microsoft Kinect retails for about $150. A motion capture system, like Phasespace, is crushingly unaffordable for the average consumer. A single Phasespace camera retails for $5000 (Anant), plus all the other equipment important to make the system run with an HMD. While HMDs can range from hugely expensive like the Wide 5, to self-built for under $1000, to new consumer stereoscopic personal HMD viewers from Sony which also stay in a range below or around $1000 (Smith), the trouble is that 19 there is no standardization for HMD technology. The Wide 5 at the Mixed Reality Lab is actually one of few of its design still functional in the world. Some of the parts used to put it together are no longer being made, making repairing the Wide 5, if damaged, a difficult, if not impossible, task. The average consumer could not, and likely would not, deal with these issues. 2.3 We are Physical Beings! Say for a moment that financial and technical troubles were no longer a problem: Would we have holodecks in our living rooms without these problems? Probably not. While infinite funding and technical genius could certainly improve consumer grade products like a Kinect, making them more like a Holodeck in some ways, there’s still the pesky problem that to ensure that human beings are the controllers of computer-generated worlds in a physical space, human bodies need to be taken much more into account in the design of these alternate worlds. The only current way to change someone’s couch into a cliff and create a dragon that is touchable and able to correctly interact with the user’s preferred method of dragon-slaying (sword, crossbow, pebble, etc.) is to remove the couch, add a cliff prop, and add some sort of sophisticated physical puppet or robot. Human technology is still lacking the matter transformation that Mr. Sutherland sought in the “ultimate display,” which makes the Holodeck such a compelling science fiction technology. 20 2.4 Inspirational Alternate, Immersive, Interactive Worlds and Spaces Even an avid Star Wars geek like myself must admit that Star Trek’s Holodeck is inspiring, particularly to an interactive media designer. For me, the most appealing Star Trek series is Star Trek Voyager, which has many episodes whose stories are often integrated with the Holodeck technology. Though all of these episodes spring from the imaginations of the writers and creators of the series, some of the ideas are very good examples of what an “ultimate display” could be. One of the simplest examples of Holodeck potential is that of the Holodeck game “Velocity,” showcased in Season Four’s 26 th episode “Hope and Fear” in the episode’s teaser. Captain Janeway and Seven of Nine are playing a game on the Holodeck where a ball flies around and defies gravity. The rules aren’t all entirely apparent, but it is clear that the ball must be in turn shot at and hit by each player, bouncing off walls is either just okay or part of the game, by a laser-gun looking object to direct its path of motion. Points are gained by the opposing player if a player is hit by the ball, which does not appear to hurt at all – in fact, it vanishes completely when Seven of Nine is hit and discovers that she has lost the game. (Hope) Without the assistance some technology that can manipulate matter, or appear to manipulate it while simultaneously still addressing the human sense of touch, this game is not possible. In Voyager’s Season Five, Episode 5, entitled “Once Upon a Time,” Naomi Wildman, the only child who had been born on board and is currently around the age of 5, explores some “Holoprograms” for children in the series called “The Adventures of Flotter.” In the teaser of this episode, she explores the episode “Flotter and the Tree 21 Monster,” in which she finds Flotter, an elemental sprite-like humanoid made out of water and follows on his adventures in trying to capture the “Tree Monster.” The program is meant to help teach children about the elements by challenging them (Flotter appears naïve, saying that the Tree Monster would sink like a log, and Naomi has to explain that logs float) while also entertaining them. Flotter may be a bumbling character, but he is endearing and friendly with Naomi and adult crewmembers who join her later in the episode. Flotter lives in the “Forest of Forever,” which is a beautiful, other-worldly forest that the Holodeck creates. Quite a playground for any child. Later in the episode, Captain Janeway, whom Flotter somehow recognizes by sight from many years ago, actually reveals that when she was a girl she had flooded the entire forest in an effort to prevent a dry spell Flotter had suggested might occur, making it clear that human interaction can have deep and wide-spread affects on Holodeck worlds. (Once) Other notable science fiction examples of what an “ultimate display” could perhaps look like in future are the “SQUID” in the movie Strange Days and the Na’vi hybrid avatars in the movie Avatar. These are examples that seek to alter the user’s state, rather than the world around the user, but this method certainly merits some thought, since both seem to seek a similar result: achieving a feeling of being in an alternate space. In Strange Days, the “SQUID” is a neural recording device that can record what the wearer is seeing, feeling, hearing, smelling, and touching, but whoever is watching this record later cannot interact with any of the recorded space and must also keep their eyes closed. The simplest example of this intense feeling combined with a complete lack of agency is the clip of a man running barefoot on the beach being watched by a man who 22 had lost use of his legs and was in a wheelchair. When he experienced this clip, he was both delighted at the sense of something he couldn’t have and clearly saddened that what he is really experiencing is someone else’s memory. (Strange) In Avatar, humans have discovered natives on Pandora, a Jupiter moon, called the Na’vi. In an attempt to have better relations with them, avatars have been made from a cross of the intended human operator’s DNA and Na’vi DNA. Avatar operators are physically inside a “pod” that essentially shifts the consciousness of the operator into the avatar, allowing them to remotely control the avatar as though the avatar body was truly their own. (Avatar) 2.5 Theme Parks are Themed Worlds Ask a child, “What’s the most magical place on earth?” and it is likely that he or she will answer with a Disney theme park. While Walt Disney was not the first to create themed events or amusement parks, his creation of Disneyland is what truly defined a “theme park” and is what stands out from all previous endeavors that might have fallen into the category. Theme parks are really an evolution of outdoor picnic areas, or at least, anything that could be defined as a “theme park” or in a similar category to one pre-Disneyland. In the US, these morphed into “beer gardens” and were easy for companies to set up at the end of a transit line for affordable amusement for the average family. Expositions were also predecessors of theme parks, with the 1893 Chicago World’s Columbian Exposition being the first one to focus on rides, shows, and concessions. And, of course, 23 Coney Island, built in 1880, catered to the amusement needs of New York City dwellers and visitors alike by providing food, entertainment, and rides. (Amusement) Walt Disney was the man who changed it all. While addressing a need for an outlet for amusement, Disney focused on creating experiences surrounding a theme – his widely successful movies, television shows, animations, and popular characters. The name “Disney” was already a household brand. At Disneyland, this park had it all and then some. Food, shows, rides, but everything could be advertising for everything else in the park, plus the attractions reflected a Disney movie or character that was recognizable to create an adaptated, transmedia-like space to be immersed in. “Imagineers conceive of communicating identity [to guests] at three levels: symbolism, representation, and sensory information.” For rides like Peter Pan’s Flight, the experience is based on and closely resembles the animated movie, flying, which is symbolic of “adventure and daring in the story as a whole,” and the sensory experience of flying that each guest on the ride can experience for themselves from their seat as it flies over the world. (Hench, 39) The immersion into this magical place where previously 2D stories came to life was a large and distinct feature of Disney’s “theme park.” Since the original park opened, many more Disney theme parks have followed, upgraded, expanded, and changed. Rival theme parks have emerged, such as Universal Studios and recently, its new Wizarding World of Harry Potter park at Universal Orlando. The Harry Potter world has been built for visitors to walk through – recreated from the movies that were adaptations of the J.K. Rowling novels. Harry Potter’s world park focuses deeply on 24 immersion, getting everything to look and feel just right like a set on a film or stage. Visitors can go to the Wizarding World of Harry Potter and recognize Diagon Alley, the wizarding Main Street of London, where the shops featured in the books and movies come to life. (Wizarding) 2.6 Interactive Theater Shows that happen around an audience and also include at least someone from an audience are not a new concept. All forms of theater throughout human history are, at the very least, immersive in their storytelling. When an actor breaks the invisible barrier from stage to audience, that is the foundation of interactive theater. Live Action Role Playing games, or LARPs, are role playing games where participants will physically act out the actions of the character they are playing by using props, an agreed upon setting and rules, and imaginations. These first emerged in the 1970’s and have in this time adapted to many different styles, genres, and purposes. These can be entertaining or educational, small scale or fantastical large events. (Live) These events give participants less guiding and more of a role in the event as they are generally a more open world without some trained actors to provide most of the entertainment or move the story along. Comedians and improvisation groups will sometimes solicit audience participation in their shows. Dinner theater includes diners by giving them roles in a staged group event. Participants are aware that they are being included in the “play,” but the story intrigue is not spoiled by the information they are given. (Hamlet, 43) 25 In virtual public spaces, MMORPGs (Massively Multiplayer Online Role Playing Games) can add an even higher level of participation from the players on each server. Games like World of Warcraft (WoW) or Star Wars: The Old Republic (SWTOR) have two opposing factions that create player versus player enemy roles. Players may encounter other players anywhere in the world, particularly shared spaces, and can act accordingly, depending on whether they are friend or foe. In WoW, opposing factions do not speak the same languages and are unable to communicate directly with each other (World). In SWTOR, the storylines highly encourage players to mistrust the opposing side, Republic or Empire (Star Wars Old). Unlike WoW, SWTOR has a practically built- in opposing faction narrative: Jedi versus Sith. Any Star Wars fan recognizes this as a clear good versus evil setup and players have no trouble understanding that when encountering a member of the opposing side, it’s time for a fight. In all of these spaces, whether taking place in the real world with an imaginary setting or a virtual world, the audience are invited to become players in the illusion. “These are all holodeck experiences without the machinery.” (Hamlet, 43) 2.7 Public Interactive Installations “…Walt Disney, who wanted Disneyland to be a place where adults and children could experience together some of the wonders of life and adventure, and feel better because of it.” (Hench, 1) Disney inspired many people already with his wonderful films and wanted to keep inspiring and entertaining people in more formats. 26 “Walt was a visionary; he introduced people to a new way of experiencing a planned environment. To do this, Walt needed to assemble a new group of designers, the imagination engineers, or Imagineers.” (Hench, 1) At Disneyland, guests will find many things to do and many spaces to explore. Not every attraction is participatory, most are actually passive or highly directed, but each space is specifically designed to be immersive and direct guests through an experience, particularly spaces where Imagineers can utilize a wienie, any object or building or setting that is inviting. (Hench, 50) Interactive installations that are publicly accessible outside of a theme park have long faced challenges that can either enhance or hinder their design. Interactive installations can be artistic, perhaps users will pass them on a subway and hardly notice that they are interactive, and some can be inviting games. Sometimes an installation can handle very few people or very many and sometimes an installation may be more of a display rather than a more private (though still in public) experience. These are factors that can work to enhance or detract from an experience. 2.8 Histories and Definitions, Part III “In many ways, virtual reality is just an exercise in manipulating … perceptions.” (Blascovich, 14) “Augmented reality means to integrate synthetic information into the real environment.” (Bimber, 2) Spatial displays detach the user from the technology and use projectors to integrate real world and virtual world into the environment. A popular example is the CAVE, where most of the walls in a room are projected on. (Bimber, 7) “New display paradigms exploit large spatially-aligned optical elements, such as mirror 27 beam combiners, transparent screens, or holograms, as we as video projectors. Thus, we call this technological variation spatial augmented reality (SAR).” (Bimber, 8) There are so many definitions of different types of alternate or virtual realties that the specificity contained within can sometimes become too specific to include similar experiences. “Spatial Augmented Reality” seems to be defining the integration of technical illusions with a specific physical space, via objects, characters, or other scenery. Virtual Reality is about manipulation of what a user perceives, sees, and perhaps, can touch? Augmented reality adds a virtual illusion or aspect to whatever real environment a user is in. These all seem to be talking about a huge range of things, while blurring and blending into each other. These terms might indeed be interchangeable in some situations. What then, can we call a physical installation that mirrors real world aspects in the virtual world of an HMD? Currently, if the terms don’t seem to fit well, making up a new phrase seems the easiest option, or adopting a phrase that is equally broad. Paul Milgram has created a “Reality-Virtuality Continuum” that well represents the problem of distinguishing these experiences: Ultimately, when looking at this image, head mounted displays as well as anything not restricted to a 2 dimensional screen environment or an entirely real environment, blends into “Mixed Reality.” (Milgram) Figure 16: Paul Milgram’s Reality-Virtuality (RV) Continuum. 28 The real issue is that even leaders in the field of mixed reality experiences don’t have an answer for how to draw a clearer or more specific line. At the VES Immersive Experiences Conference on February 25, 2012, Professor Mark Bolas gave a presentation about mixed reality, based on the work from his Mixed Reality Lab, part of the Institute of Creative Technologies. He promised at the beginning of the talk that he would talk about virtual reality, mixed reality, and real reality, but at the end of the presentation, he admitted that “real reality” is very confused now. We’ve all got phones and computers and other personal gadgets that can blur reality when we’re just sitting on the couch, that can alter an image on the fly by removing people or objects from the scene at a finger’s touch and thereby alter a memory, and ways to augment what we’re seeing when we are looking out the window of a car. (Bolas Conference) The question may no longer be, “What is mixed reality?” The question may now be, “What is reality?” 2.9 Building a Holodeck From Scratch If unhindered by advancing technologies, what would be the design requirements for a Holodeck? Currently, the world of immersive technology seems fragmented. HMDs are really just screens: real-time movies with some calculations behind them. Other immersive media in existence range in many ways from theme parks to interactive theater. Since these all share a goal of immersion, what traits do we put together when designing for an “ultimate display” that could be made today, a Holodeck-like platform that does not require any more technical advancements? What can be made right now 29 that is very close to the feel of a Holodeck and what properties would we borrow from other media in this project? An “ultimate display” is a space. It doesn’t hinder the user with wearable technology, but rather has the technology manipulate the space to create illusion. The illusion is tangible and can be manipulated by the user. Chapter 3: Discover Shayd 3.1 Goal #1: Total Immersion After two years in the Interactive Media Division, I was itching to get as close to a Holodeck as possible – as my third year thesis project. After having my first experience in an HMD, the Mixed Reality Lab’s Wide 5, there was no altering the desire to make content for an experience that was as near to a Holodeck as possible. Much like Ivan Sutherland’s pursuit of an “ultimate display,” my pursuit of a Holodeck has spawned action and ambition. The first project goal was total immersion, which meant immersion for a human body’s senses - touch, sight, sound, perhaps even smell, seemed doable in a project, though taste was not a particularly important, or even relevant, goal. It was critical to make sure that whatever the experience was, the player was convinced that their whole body was really in the alternate space. This also meant a space that was not just a tiny space, it had to be a space that felt like a whole world with many stories. Full physical immersion clearly needs to draw on the realm of architecture, art, and theater combined with interactive design. This means that not only does this goal 30 seem life-sized and large, it absolutely requires a deep investment in multiple disciplines with interactive design holding the reins. 3.2 Goal #2: Alien World Science Fiction, Fantasy, Weird, Bizarre… these are all favored genres of mine to engage in as well as to write in. It shows in my work from the previous two years here at USC. Whether it’s a glowing world created in Unity3D or a card or board game (full of wind up toys, small animal figures, stuffed animals, and improbable creatures), it’s probably not going to seem like every day life on Earth. A general pursuit in my work the past two years has also been a “beautiful world.” I have said it often about a lot of my work and realized that I really wanted to Figure 17: Screenshot of Glow World, created by Juli Griffo in Spring 2011, which can be seen in 3D and played in conjunction with a Microsoft Kinect. 31 make a beautiful world that reflected some beautiful narratives one more time before I completed my degree at USC. My thesis project had to be a “beautiful world” as well as an alien one. The desire for an immersive beautiful world comes from my desire to make a better society to actually live in. My hope is that if an immersive world is a beautiful world, it may perhaps influence a player’s real life outside of the experience. 3.3 Goal #3: Seems Like Magic The third law of Arthur C. Clarke’s Three Laws is: “Any sufficiently advanced technology is indistinguishable from magic.” (Clarke’s) From Disneyland to great movies or books that pull an audience in, the allure of these story spaces is that they feel like magic. There’s an intangible thing behind narrative experiences that captivates the imagination – story, illusion, role playing, transportation to some other place – no matter what the experience, the reader, audience, or player should feel magically transported into that place. Something that does not appear to be alive, whether it is an object, a setting, or words on a page, can suddenly appear alive, via animatronics, virtual reality, theatre, or an incredible story in any format. The shift from being out of the illusion to entering the illusion and feeling part of the illusion can and should feel like magic. In the spring semester of 2011, in IMD’s Thesis Prep, I was challenged with performing a magic trick. It didn’t take long to decide what it was I wanted to do – the problem was how to get it to work. After several weeks of failing to make the trick work, 32 I finally got to stand up in front of the class and tell my little story. I have a house rabbit at home named Madelyyne. My classmates have all heard about her or seen pictures of her, and some have even met her, so it was not a long story to tell. I told them about how I’d found a way to do something similar to Frosty the Snowman’s magical hat, which makes him come alive when placed on his head (Frosty), and the magical hats used by the magician and Alec Azam the rabbit in Pixar’s short, Presto. The magician can reach into his top hat and pull out Alec, who is out of sight somewhere wearing a magical hat which acts as a portal to the top hat (Presto). I explained that I was going to channel Maddy’s spirit from home into a stuffed rabbit and she would come to life in the classroom. After inspection from classmates and professors alike, the lifeless stuffed animal was put to the test. I placed a magical bucket hat onto the stuffed animal’s head and suddenly Maddy’s spirit inhabited the stuffed animal and moved around. My classmates are all intelligent people and no one immediately identified the trick (in fact, I never revealed the trick to the whole class and no one that was not “in the know” ever guessed quite correctly), but everyone responded with interest when the far-fetched “magic” appeared to work. Figure 18: Stuffed rabbit that embodies Madelyyne's spirit from home. 33 That flicker of engagement with the magic of a story is absolutely a requirement in all of my future work. It was clear that day that I was right that interactive experiences of any kind (even not-so-interactive experiences like movies, books or passively immersive experiences like a theme park ride) always needed to feel like magic, no matter how clearly impossible the circumstance. It is necessary to get an audience, player, reader, user, or whatever name you want to call the person experiencing a piece of media to suspend disbelief and forget entirely about any technology involved. 3.4 Introducing Shayd Shayd is a project that attempts to merge all three of these goals to create an ultimate experience that is technically possible now. Shayd has total immersion in that you can see, feel, and hear everything around you. It is an alien world – and a beautiful world, at that! The experience of Shayd also seems like magic in that users don’t expect it to have so many senses covered and when operating correctly, all signs that Shayd takes multiple people to manipulate are hidden. Shayd is a physical installation mixed with a head mounted display, tracked by Phasespace motion tracking cameras. In the queue, guests are presented with a video about how they’ve left their broken, dying home world and traveled in space. They’ve come upon a planet that appears uninhabited and is healing from some ancient natural disaster. Only one person at a time can actually experience Shayd, so the video can be watched while someone is already inside. When the experience is cleared, a guest will 34 enter the cave, see cave paintings telling a story of what has happened on this strange planet, and discover alien technology (the HMD). A stagehand will be available to assist guests into the HMD, though they will be silent and dressed in black in order to be as invisible as possible. Upon putting on the HMD, the guest will discover a vibrant and alive world inside a cave. The main character, Syyrette, a four and a half foot tall symmetrical biped of the Tormee race, will come through a teleporter to meet the guest. This cave will also have cave drawings depicting a similar story. Syyrette will approach the guest warmly and Figure 19: Inside the Cave in the alternate dimension of Shayd. 35 encourage the guest by coming up and touching the guest’s arm to follow him through the teleporter. Once through the teleporter, the guest will be in a clearing, overlooking all of Shayd. Syyrette will introduce himself as “Syyrette” and another character will enter the clearing, though he will not come up close to the guest. Syyrette will introduce the new character as “Maubwe,” a generally disinterested asymmetrical three and a half foot tall, mostly upright creature. While Maubwe passively stays near Syyrette, he is generally just comic relief and plays with things nearby with his telekinesis ability. After introducing himself and his partner, Syyrette urges the guest to the edge of the clearing on a cliff side and introduces the planet, “Shayd.” The clearing overlooks water and streams, trees, mountains, a green sun low in the dark sky, two moons, and an aurora borealis. Figure 20: Syyrette on left; Maubwe on right. The main characters of the Shayd project. 36 After the guest has had a chance to take in the view and Maubwe’s antics, Syyrette will create a teleporter and guide you onto it, taking the guest to a circular area around a pool of Shayd’s life force. There is a small, insurmountable rock wall next the the teleporter exit. Syyrette guides the guest around the life force. Maubwe follows moments later and watches Syyrette guide the guest. Syyrette creates a teleporter and Maubwe floats up and across the life force, entering the teleporter before Syyrette and the guest. Figure 21: The Clearing in the alternate dimension of Shayd. Figure 22: The Hollow in the alternate dimension of Shayd. 37 Syyrette and the guest get on the teleporter and arrive in a small clearing on the mountainside. Maubwe is already looking to the sky, perhaps telepathically calling to the flying creatures that are arriving. Maubwe gets onto one and takes off and Syyrette helps the player onto another creature. As the guest’s creature takes off, Syyrette hops onto his own creature and joins the guest and Maubwe on a flyover ride across the Shayd world. After flying over all the places that have already been visited and seeing other parts of Shayd full of life, Maubwe and Syyrette create a large teleporter in the sky near a large crevice bursting with Shayd’s life force and fly into it, vanishing from their flying mounts. The guest soon follows and arrives back in the cave, sitting on a rock, Syyrette and Maubwe waving to the guest and the HMD turns off. The guest will take off the HMD and actually be sitting on a physical rock. There will be no signs of any person that might have been an actor puppeting Syyrette, just a stagehand dressed in black ready to help the guest out of the HMD and to exit the cave. Figure 23: Flying over the alternate dimension of Shayd near the Cave. 38 3.5 Syyrette & Maubwe Walk You Through The Uncanny Valley Unscathed A desire for everything alien makes for an easy and logical design decision to have aliens from another planet as the stars of any immersive experience. It also has an added advantage: Non-human characters are far more believable and way less creepy. Virtual human avatars have gotten very close to appearing to be human in recent years; but, as with anything that isn’t actually a carbon-based, bona-fide human, it’s hard to trick a human into accepting something that is just almost human. Wax museums, doll makers, movie-makers, painters, and artists of all kinds have all had lots of time to make something really seem human. Computer generated images and 3D models of humans have been around for less time, but have encountered the same problems. The closer it gets to looking human without being absolutely perfect, the more offensive and unrealistic it becomes to the audience. Nobody can be immersed in a story the same way once engrossed in how weird it is when a character’s mouth moves or how unsettlingly just shy of human something appears to be. “With high photorealism and perfect communicative realism, of course, nothing is uncanny and there is no repulsion. However, communicative realism is very difficult to achieve, from a technical standpoint.” (Blascovich, 78) Pixar is careful to keep their characters from being too photorealistic, to the success of their animated films. James Cameron also approached Avatar’s “Na’vi” aliens to be human-like, but not too photographically similar, to avoid unsettling the audience. (Blascovich, 78-79) There are two races on Shayd in the alternate dimension where there was never a natural disaster: the Tormee and the Wah-zay. The Tormee are upright, symmetrical 39 bipeds with two arms and there is a reason for them to be so. Even though Shayd is an alien world, it seemed natural to make sure that at least one race was recognizably “people.” The Wah-zay are asymmetrical, somewhat upright, and lack digits or opposable thumbs, something that seems to be a defining factor in human development and intelligence and is definitely important in the human ability to manipulate physical things in the world. This made the Wah-zay strange aliens and the Tormee the race that humans are most likely to identify with first. Shayd is a planet filled entirely with symbiosis. People, animals, and the planet itself must give freely to one another. Without two intelligent races flourishing together on the planet to compliment each other, something symbolic would be lacking in the world. It feels right that the two races who depend on each other look completely different. It is important to also ensure that they are recognizable and easy to accept as an intelligent species by visitors to the experiences as well as having different sets of abilities to fit into a utopian world that complimented each other. 3.6 The Making of Syyrette: Digital Puppetry The precise details of Syyrette as a digital puppet, portrayed by an actor being tracked by the Phasespace system, have not been completed at the time of this paper. The current scheme, which has been tested very little, is that the actor will have a small portable keyboard or controller to have control over facial expressions and speech, and the actor’s movements will be somewhat tracked in real time by several LEDs on their body. Some amount of guesswork by Lead Programmer Chao Huang and well-timed 40 puppetry on the part of the actor will have to fill in the rest of the movements of Syyrette’s body. At the same time as we complete the digital puppetry, we are trying to work out the best practices to teach our actors who will play Syyrette. At this time, we believe that stagehands and the actor will have several roles to play in ensuring that Syyrette’s movements and behavior function correction and that physical props match virtual elements relative to the visitor. The actors may have to use taping on the floor, practice, and quick thinking to properly maintain character while also keeping the visitor safe as they wear the HMD. 3.7 Hitting the Road… or Not: Why Can’t Shayd Be Installed Everywhere? Shayd’s setup is extensive. While some pieces of Shayd are portable, like the 10 camera set used for the thesis shows, this portable HMD system is expensive and requires a high level of technical knowledge to set up. The physical cave itself cannot easily be reproduced, either. Shayd requires a large amount of space in any setup to make the tracked space of the mixed reality experience. Shayd also needs to hide its actors and stagehands to keep the illusion alive. Everyone who works to run the project must be trained. These issues don’t prevent Shayd from being duplicable, but they do make it incredibly difficult to replicate. Arcades are generally the cheapest places for entertainment that can hold both digital games and small physical games; but Shayd would need a large chunk of an arcade. Shayd also might be able to find a home in an art 41 show, a science museum, or even a children’s museum. In a children’s museum, Shayd workers would likely need to be extra careful with expensive equipment made for adults. Perhaps a children’s museum should be given a smaller head mounted display, which could certainly be an interesting challenge to PR4 creator, Mr. Palmer Luckey, the creator of new HMDs from the Mixed Reality Lab. Shayd is using Luckey’s PR4 as its HMD, which is explained as “ancient alien technology” in the narrative of the project. Shayd has been specifically created and tailored to being presented during the May 5-10 Interactive Media Thesis Show, titled Other Worlds. Shayd would have to be somewhat redesigned to fit each other space that might want to feature the project in the future. 3.8 Designer’s Dream, Designer’s Nightmare Shayd is a fantastic project to devote an academic year to. As a thesis project, it has been nothing but wonderful to dream up this alien world and design the physical and virtual illusions of the experience. And as a thesis project, it has also been an incredible and often stressful undertaking. Shayd in its very nature is large scale. As long as the thesis project platform is a tracked HMD, it’s never possible to be small scale. The virtual world inside the HMD could contain a plain box and a light, but the production efforts would remain the same – moving equipment and setting up sensitive cameras to display anything, even that boring box, is a large scale task on its own. Making sure each piece of Shayd comes together is a daunting task, but a worthy one. Shayd has so many components – HMDs, tracking cameras, Kinects, virtual 42 models, physical props, physical installation and even a children’s book. It’s impossible to know if everything will come together exactly as planned until the show in May, but whether it succeeds or fails, it’s all a lesson in design… and project management. Chapter 4: It Was Just One Thesis Project, Wasn’t It? 4.1 Summer School After submitting my thesis proposal in April of 2011, I knew only that I wanted to use the HMD equipment that the Mixed Reality Lab offered to loan me. I had not yet figured out the world I wanted to create or the stories from that world that I would tell. While still coming up with the right world, throughout the summer of 2011, I ran Unity3D/HMD labs at the Mixed Reality Lab where I learned a lot about the HMD, its capabilities, and how to make Unity3D play nice with the physical aspect of the tracked HMD. We worked to test designs and come up with ideas for new content to put into the HMD. Several projects came out of these labs, including my thesis project as well as a project that even made it into the Sundance Festival. During that period of time, my biggest breakthrough experiments had some mixed results. While none of these have been well tested, there were some observations I made that appear to hold true and that are worthwhile to take note of. The longest running experiment of mine from last summer, largely because of the technical problems I had to overcome to make it work, was investigating whether I could fool HMD users into thinking that they were falling through a hole in the floor. The short answer is that no, no one suddenly truly believed they were falling. 43 The long answer, however, is that it is possible to get relatively close. The speed of the fall must be within an approximate range of what users believe it would be like to fall; if it’s too fast or too slow, it’s unrealistic and unconvincing. Some things help this problem, like falling along with objects or particles. The problem is that the user’s feet are still grounded. Sometimes a user might begin to fall in the virtual world with a foot off the ground still and putting it down as they fall just completely ruins the magic. Sometimes the timing of the fall itself was off; if a user was just looking over the edge of the hole, it might trigger a fall through the floor, not through the hole itself. Without an actual edge on the floor that a user might feel on their feet, this experiment proved more that visually the motion of falling could make sense, but that without addressing the physical body in a more tangible way, no one would ever truly believe that they were plummeting to some possible doom. HMD users are attracted to several things in the virtual world. Some things are just like in the real world. For example, a more lit area will tend to draw people just the same as in the physical world. Some of the things users are attracted to are unseen in the real world, like particle effects. In a virtual world, particle effects will generally draw any user’s eye. Sometimes users will stare at particle effects to the exclusion of all other things in the virtual world, without ever taking a step towards them! Two-dimensional elements in a 3D world are sometimes acceptable. In particle effects, it is difficult to tell that a particle is 2D if it is billboarded (in Unity3D). However, 2D leaves on a 3D tree body are not acceptable when the tree is within reach in the virtual world. Placing a 2D/3D tree out of the range of the HMD user is acceptable. 44 It is much harder to tell that the tree’s leaves are not 3D, and users generally will pay the discrepancies no mind when it is in the background. I have been calling the instant displacement of an HMD user from one space to another “teleporting.” What this means technically is simply that I am now tracking a different space in the Unity3D world and the user appears to be in a new place, while their feet have remained on the ground. Users understand what has happened, but don’t always understand the cues that were surrounding the teleporter that sent them to the new place. Visual special effects, sound effects, and particle effects are useful to drive users onto the teleporter and give them a sense of teleporting when they move to the next space; however, these must be done not too elaborately and not too simple, which can require lots of iteration and testing to get just right. Users have mixed reactions to characters that look at the users as they explore a virtual world. Some users are interested in the characters to the exclusion of all else; others seem confused by these characters and will avoid them, paying attention to other aspects of the world instead. There is something interesting about characters that are focused on the user, which I really wanted to pursue further by creating smarter and more responsive characters that could look at the user on cue instead of perpetually stare. A final highlight from my summer of exploration might be the most important of all my findings from the whole year. People will get into the HMD and do unexpected things, but there’s one thing that can always be expected: Once an HMD user is interested in something and gets the hang of looking and moving around in the HMD, they will look at anything and everything. They will do the unexpected. If they become 45 interested in something on the floor, or the floor itself, they will kneel down, roll around, or lay on the floor to satisfy their interest. 4.2 Geared up for Winteract In hindsight, the winter show (we called it Winteract this past year) was really an exercise in learning how to move the HMD and Phasespace system. As the show date got closer, I realized more and more that all my efforts and energy were going into figuring out the details of putting together a portable system, moving it to campus, and setting it up correctly once there. This prototype was a simple demo of teleporting between the cave and the clearing space. After several tests during the show, my team and I removed the teleporters partly because they were not working quite right, but most of all because people were not recognizing them as teleporters and were confused momentarily by the shift in space, since they didn’t expect it to happen. At this stage, Syyrette was AI only and we attempted to move him around and try touching the user at a correct moment. The AI behavior was meant to keep Syyrette out of range, which we had to turn off. This opened the door for more problems, because now users were able to just walk right through him, since our manual digital movements of him were not very precise or well timed. It was clear after this evening that Syyrette really needed to be portrayed by an actor or we needed to figure out some really incredible AI that could access more user feedback, the latter of which did not seem as possible as the first. 46 In our clearing space, we had our colorful, bubble trees all in a ring around the edge of the cliff with the view of the rest of Shayd. While users were interested in the strange, colorful bubbles, they didn’t read the trees as being trees. The issue with these trees was that they didn’t move at all, not even as though they were blowing naturally in the wind, and they had no texture on them to suggest the depth of their 3D models. They simply looked like colorful pills stuck together. We hadn’t built our full sized cave at this point, but we did wall off our section around the cameras in an effort to make it feel as though there was a different space visitors were entering. We used mostly c-stands and scavenged rolling white boards, so this did not read as a cave, nor did it read as a space that was not to be entered freely. People ended up milling into our space and being in the way of the person using the HMD. And finally, our biggest insight at this stage was that there was too much freedom to explore and no goals. This helped me to change the design from being geared towards freely exploring to a guided discovery. With no goals, users reported that they didn’t know what they were supposed to do. 4.3 Shayd Children’s Book A children’s book for the Shayd world was always in the works, however, after a few months into the academic year, some of the more difficult issues like overcoming technology issues, space problems, design issues, trying to learn to draw in 2D with the fine motor coordination of an animal with no fingers, and crewing up a dedicated team 47 with enough hands to take on a large-scale project like Shayd started to feel overwhelming. I thought the easiest way for me to feel productive while facing this daunting issues was to do something that I know, and, for me, that’s writing. The writing of the book started ahead of schedule to ease fears and frustrations with other aspects of the overall Shayd project by using this storyteller’s strongest skill (writing!). In hindsight, this was a really good thing since available hours for writing dwindled quickly by the second half of the academic year. Currently, the book is written for a young reader. The story could perhaps be read to little ones (preschool through first grade level) and read independently by an advanced 8 year old reader. The story itself isn’t so complicated and is being supplemented with illustrations (that are not done by me) to mirror the reader level of the Dinotopia book series, which is a great story for very young readers to be read to or for an 8-12 year old to read independently. The official reading level of Dinotopia varies depending on the source, though no source measures it outside a range of 8-12. After putting so much into the design of this world and the histories of the alien races surrounding Shayd and its dual dimensions, writing the book felt really good and also really difficult: one book didn’t seem like enough! So in true Dinotopia fashion, the book written is the origin story. It is the first peek into the huge world of Shayd. This project became like another iteration of the Shayd project and helped inform the rest of the story and design tweaks of Shayd the installation. 48 4.4 Shayd Mobile Shayd Mobile was featured at IEEE VR 2012. Using the FOV2GO utilities that Perry Hoberman and the Mixed Reality Lab worked to create, Shayd had a scene on the iPhone. I wanted to reaffirm my belief that Shayd needed to be on this large scale as well as prove to others that it was just not the same experience on an iPhone. As it turns out, I was right, though Shayd Mobile was a worthwhile effort and a nice take away after I graduate (and I have to give back all the equipment I’ve borrowed). The iPhone serves as a small window into a space that we are not immersed in physically. Since it can only look around from an ambiguous feeling point, it is confusing as to where you are in relation to the virtual space. On the bright side, it is portable and certainly easy to set up. Shayd Mobile seems to be much more like an interactive “poster” for Shayd rather than a replacement platform for the project overall. Chapter 5: From One Little Installation to What Lies Beyond 5.1 Possibilies for Shayd to Improve in Near Future With Shayd composer Jeremy Tisser’s beautiful music recorded live the last weekend in March, Shayd is set to sound fantastic, both in the virtual world and in the physical world. Future improvements in the sound design for Shayd would be to expand the world to a fifth area and to layer in some soft creature and nature sound effects. Shayd can also be expanded to have fully suited actors both inside the installation and out, assisting or just playfully interacting with people waiting in line, much in the 49 style of a Disney theme park. Some of the original ideas for Shayd included creating such large scale, real world characters, though it was quickly clear that it would be hard to manage so many large-scale tasks in just a single year without quite a few more team members. This idea remains a hope for the future. A clear and important goal for Shayd would be to find ways to make Shayd more portable. Some way to easily set up and tear down, much like a circus, would be a fantastic attribute for the project on the whole because then Shayd could really be shown off to many more people. Currently, since so much of the project’s technology is being borrowed, it is unlikely that the project could remain on exactly the same platform. It might be possible to use multiple kinects in future to track the same virtual world paired with a low-cost, self-made head mounted display, though this would still require a decent chunk of funding. 5.2 A Lifetime to Shayd on the Holodeck? How long until Shayd could be experienced on a “true” Holodeck, without the assistance of wearable technology, actors, or stages? Are holographic alien landscapes and companions possible in my lifetime? I am 28 years old, with less than forty years before retirement age (65). Could I potentially be designing ultimate “alternate life spaces” in my lifetime? It is my hope that content like Shayd and designers like me get noticed, and quickly, by engineers with imaginations and a similar passion for making imaginations come to life. By showing what can currently be done, like Shayd, and peaking the interest of people from all fields, investors, producers, engineers, and other 50 storytellers, more people might become interested in the same pursuit as me. That is my hope for Shayd and the future: that Shayd can help to shape the future of interactive media by creating an equal sense of importance for physical immersion in the media and virtual world as mechanics and story. 5.3 The Shayd Children’s Book: Vague Adaptation, Expansion, Transmedia? Shayd: The Discovery (tentative title) is the first installment in a series that will be completed in the future. In this origin story, our narrator – who remains genderless, ageless, and other than having an upright, stick figure type body with two eyes, largely alien race neutral – comes from a broken, ugly world. We follow the narrator from leaving home and traveling alone to finding Shayd, a seemingly abandoned world that appears to be recovered from some long ago natural disaster, and setting down outside a cave. Exploring inside the cave leads to the discovery of some ancient cave drawings that tell some secrets about the planet and the history of the race that lived there. Next to these paintings is a lit up piece of technology. Upon investigating, a hologram appears nearby, demonstrates how to put on the technology, and vanishes. Our narrator hesitantly follows the demonstration and puts on the technology. Instantly, our narrator is in another cave… no, the same cave. It is the same cave, but it is clear that the paintings on the walls of the cave have changed; and though it’s the same green sunlight softly glowing outside through the dark twilight, everything else is definitely much more alive and well than before. Suddenly bursting into existence just at the mouth of the cave, amidst sparkles and lights, is Syyrette. 51 From approximately this point on, the experience of the narrator in the book and the experience in the installation start to radically differ. Syyrette still shows the narrator around and introduces himself and Maubwe and Shayd, but the story gets much deeper in the book. The book has a solid day of being shown around, having visual stories from Syyrette (in the form of tiny holograms on the palm of his hand), during which the narrator gets the story of the Zoop, the race who created the technology the narrator is currently using, who are still nearby and watching out for Shayd from elsewhere in the solar system. The Zoop are roughly two and a half feet tall and share in the traits of Syyrette and Maubwe and their dimension of Shayd – round, colorful, they walk upright like the Tormee (Syyrette’s race) and are natural caretakers like the Wah-zay (Maubwe’s race). Syyrette also conveys the story of the Zoop’s conflict with the Nad’l, eight or nine foot tall giants with triangular, stick-like bodies with large block-shaped heads. The Nad’l have long tried to capture the Zoop, their technology, and the resources of Shayd, but the Zoop, though physically vulnerable to an attack from such giants, are more technologically advanced and are able to escape the reach of the Nad’l and defend their home world. At the end of a long day of exploration and history lessons, Syyrette and Maubwe are ready to part with the narrator for the night, though our narrator ends the book with plans to return to meet Syyrette and Maubwe the next day and perhaps in future to find the Zoop, while keeping watch for the Nad’l. Though the narrative surrounding the entering of the installation and the main characters are so similar and the book could easily be called an adaptation of the 52 installation up to this point, the other parts of the narrative in the book and the narrative in the installation are much more like transmedia. What, then, is this peculiar split? How does the book inform the installation and vice versa? In the same conversation with Mr. Joseph Garlington on March 12, 2012, we discussed this issue. I was feeling sure that the book was not a direct adaptation, having learned about and written some adaptations of my own while pursuing my undergraduate degree. I was less certain about whether or not this could fall into the category of transmedia. Part of the definition of transmedia is that it is not adaptation. The other requirement to being transmedia is that it “deliver unique pieces of content over multiple channels.” (Transmedia) If so much of the story starts out practically identical, it seems difficult to argue that this is transmedia. Garlington, however, had a good answer to my concerns that I was not going to be able to speak clearly about the book in relation to the installation without having a better grasp of what category this fits into. Garlington explained that it’s the same world being portrayed from different viewpoints. Strange as it sounds, I had not previously vocalized that, though it is exactly right. The spark that got me writing the book ahead of schedule was because I had this huge narrative world and so much effort put into the world-building of Shayd, that I was fearful would never be brought to life in the installation (as it would be impossible to do in a five minute experience). That’s when the viewpoint started to really shift. I wanted to give a first person experience that was very neutral so anyone reading it could make up their own narrator/avatar in their head. Images of the narrator that happen after entering the 53 alternate dimension of Shayd are kept to a shadowy stick figure, which the narrator quickly realizes and addresses the complete lack of previous self. But this narrator has thoughts that I wrote. This narrator has an opinion of the home world that had been left behind and traveled alone. In the physical installation, visitors are going to be clearly aware that they are at a thesis showcase and in line to experience an immersive, interactive installment. No matter how hard the Shayd team tries to dress the space, it will still be clear that there are humans manning the experience somewhere behind the scenes, even if we achieve the ultimate in magical illusion and visitors forget that during the experience. Most importantly, the narrator goes on to have more experiences and ends the book with a desire to come back to the alternate dimension as well as meet the Zoop in future. The narrator is able to develop more of a relationship with Syyrette and even Maubwe, whereas there isn’t too much time to bond with Syyrette or come close to cracking Maubwe’s indifferent shell in the installation. This makes Shayd the installation and Shayd: The Discovery operate in different narrative modes, which are “the set[s] of methods the author of a literary, theatrical, cinematic, or musical story uses to convey the plot to the audience.” (Narrative) Shayd is already a cross-platform affair, but defining the differences between them goes a little deeper. Narration, the process of presenting the narrative, occurs because of the narrative mode. It encompasses several overlapping areas of concern, most importantly narrative point-of-view, which determines through whose perspective the story is viewed and narrative voice, which determines a set of consistent features regarding the way through which the story is communicated to the audience. 54 Here’s the thing: in the installation, you, the visitor are in control of the narrative point-of-view. In the book, the narrator is controlled by the writer (me) and meant to be a representation of self for the reader who is not physically in control of the narrative point- of-view. Therefore, I submit that this situation accept a new name: cross-mode point-of- view. This term can be applied to cross platform media experiences that are narratively told from different points of view within the same world or narrative timeline, like Shayd’s installation and children’s book. 5.4 Syyrette & Maubwe: Smart Enough to be C-3PO & R2-D2 in Shayd? The Shayd mascots are definitely Syyrette and Maubwe in the installation as it is. But can Syyrette and Maubwe be made smart enough to interact in a longer-term and more meaningful way with visitors to Shayd? After the Other Worlds showcase finishes in May, pursuing ways to make Syyrette and Maubwe engage with visitors even more, whether through speaking to and responding to visitors or being able to virtually and physically navigate space with visitors (via actors or robotics), will be a reaching goal. It would be nice to have higher capabilities, but part of the pursuit would be the process of discovering what is possible and by what means. This would definitely need further research. Their original conception was to be a quirky pair that guide you through a narrative, immersive world. They are buddies and would never be parted by great distance or over any span of time, much like R2-D2 and C-3PO (Star Wars). They are even in a similar vein for communication abilities: Syyrette has vocal chords and can 55 talk, though he doesn’t know English, just like C-3PO is upright with arms and legs and does the talking. Maubwe is cute, short, awkward and somewhat less mobile, with no mouth to do any verbal communication with, like R2-D2 is cute, short, and awkward. R2 can only communicate through C-3PO by beeping, something that doesn’t translate well to humans, similar to how Maubwe must silently communicate with Syyrette to communicate through his telepathy in order to communicate. If more of their personalities could be present by creating more capable companions in Syyrette and Maubwe, Shayd would be an even more impressive experience. 5.5 Shayd Team, Fall 2011-Spring 2012: “Original” Though we came up with a number of things that had similar elements to Shayd, the Shayd team could come up with no example of an experience that they had ever seen before that was really like Shayd. Several team members used the word “original” to describe the project and their reasons for joining the team and devoting their time to it. I knew that along the way, I would clearly draw on elements from design, art, film, theater, architecture, or more; but when I first conceived of Shayd, I was really only looking to make the best narrative world I could come up with to use with the HMD I was borrowing from the Mixed Reality Lab. I like aliens and beautiful worlds. The HMD was a tool for magic, so it wasn’t a leap to create the narrative around the magic tool itself. And then it all just expanded out from there. After working with HMDs and this narrative world for nearly a year of my life, I can really see that this experience pulls 56 from so many elements that already exist in the world, though I still cannot find anything that feels quite like Shayd. 5.6 Is This Interactive Media? It is important to address this question. Perhaps the first question to answer is, “What is Interactive Media?” Interactive Media can describe a broad range of media. Interactive Media draws from multiple categories, some of which can be called reactive, responsive, or immersive. These can be muddled and debatably be categorized as Interactive Media. However, for the purpose of this paper, my broad definition of “Interactive Media” will mean “any platform that can be engaged with by a human with parameters for action or response to interaction.” Though it is broad, this definition can be easily applied to digital games, board games, role-playing, interactive cinema, theme park rides, interactive theater, and beyond. This broad definition easily encompasses Shayd. But is Shayd interactive media if there is a more narrow definition to interactive media? Shayd is not really a game. While it may have a virtual world made in a game engine and appear somewhat game- like in its playful and explorative adventure with its colorful alien characters, it really can’t be called a digital game, nor is it a physical game like a LARP. So what is it? Virtual reality seems to be widely accepted in the realm of digital games – many, if not most, virtual reality experiences are games. Explorative physical spaces might be accepted as a physical game with real world boundaries, and in that regard, a local children’s museum or even public library can house games. But mixed reality 57 experiences do exactly what they say – they mix realities and therefore they mix the rules up. They draw from many different platforms. It seems sure that these can also be games, but that not all mixed reality experiences will be games or perhaps can even be defined as “games.” Does this make experiences that don’t really fall into the more popular category of “game” less worthwhile? It shouldn’t. The field of interactive media is sensationalized as new. USC’s program is still younger than the students in it. While interactive media as a career field is indeed relatively new, interactive media in its broadest sense has been around for all of human history. Any time children play and make a story and make decisions together about the imaginary world of play, they are participating in interactive media in the broad sense. This means that not only is interactive media an old concept, but it can truly cover many interactive realms. This is where Shayd resides. Therefore, Shayd is indeed interactive media. 5.7 Why Do We Need This? As an industry, interactive media is a young field. Members of this community are young. Technology has pushed media forward quickly and continues to push at a rapid pace. It is general knowledge that keeping up and ahead in any field is a key to keeping an industry alive, thriving, and successful. Mixed reality experiences and highly immersive and engaging story spaces are hugely important in maintaining advancements in the field of interactive media. Storytellers of all kinds require a suspension of 58 disbelief, why wouldn’t interactive media designers remain open minded to broadening the realms of possibilities? Chapter 6: What’s Next? 6.1 A Year of Immersion in Immersive Technology is Not Enough The realm of mixed reality is huge and expanding constantly. There is no way a single year of work with immersive technology is enough to completely understand everything about it. Even though this academic year was full of discoveries about how head mounted displays worked, what it feels like to move around while wearing an HMD, and designing for a virtual and physical environment, there is certainly plenty more to learn. One year only just brushed the surface. The goals I set out with for this thesis project have been met in many ways. I knew that I would not be producing a true Holodeck in one year (that would be a tall order), but I did want to make an experience that felt “Holodeck-like” and was something that could be on the path towards a Holodeck. I do believe in this regard, that I have succeeded. I had several more detailed goals specific to the Shayd project itself. I would have liked to create more intelligent and interactive characters in Syyrette and Maubwe. As it stood, early on, I realized that making them both able to talk, gesture, respond to gestures, and touch the visitor was too lofty a goal for a smaller crew. I had also wanted to put more story elements into the experience while remaining a 5-7 minute virtual 59 reality experience, which also quickly proved to be too complicated and too big an undertaking for a crew of our size. While I certainly wish I did not have to iterate and change so much of the meat of the content to fit into such a small time frame, I do not believe this hurt my overall goals of creating a physically immersive environment in an alien world and having the experience seem like magic. At no time should the visitors see the technology at work, which is true of Shayd’s properly executed designs. It is still dangerously possible to break the illusion with a moment of bad timing from an actor or stagehand or any other potentially unforeseen circumstances. I would have liked to make Shayd simpler to execute correctly with room for fewer errors while still retaining the feeling of a high level of sophistication. 6.2 Shayd Needs to Be Tested on a Larger Scale for Better Design Feedback Due to its issues of portability, Shayd has not been tested as it will be shown in May at Other Worlds. It will also have more features in place in time for the show that are not currently in place at the time of this writing. Even then, when all the pieces are built and put together exactly right, it would require months and months of testing and iteration to improve the design of both the physical installation and the virtual world. For now, the pieces of the project have been worked on and tested. Often when some pieces are combined, unexpected things arise, so it is assumed that when every aspect of Shayd is put together, a clearer path for iterative design will suddenly be visible. 60 6.3 Shayd is Limited By Funds, Team Size, and Deadlines Shayd is currently limited by funds, team size, and deadlines associated with the thesis year. Were Shayd a project from a company with funding and a larger team assigned to it, with fewer academic restrictions (no thesis paper to write before the project’s completion), Shayd would be similar but probably still look and feel a lot different. The project would have had the time and man-power to iterate further. After the thesis project is finished and the thesis show has ended, the Shayd project will be seeking a new, permanent home where work can continue. 6.4 So Where’s My Holodeck? Should There Be Such a Thing and What Is It Now? The Holodeck from Star Trek, the “ultimate display” Sutherland was in search of, still is not here yet, but work has been made towards achieving such a thing. Tomorrow, we will not all wake up and see advertisements for the new living room-sized Holodecks now available for purchase. But, we have started down a path towards a Holodeck and the biggest fuel in moving towards success currently are the dreams of engineers, designers, artists, children, dreamers, thinkers, actors, builders, magicians, writers, moviemakers, futurists, butchers, bakers, and candlestick makers. Everyone has thought about what it might be like if imagination were able to run wild in the physical world, so everyone has imagined a Holodeck. The idea is not new. The technology advancing towards it is not new. It is the progress that is new. Every day, new screens are made, more capable and more beautiful than before. Every day, more powerful computers are created, able to talk to everything, faster and faster. There is a point where our 61 technology will improve just enough and suddenly, a Holodeck will be real. Perhaps, they will be for the super rich, the government, the military, and giant conglomerations at first, but just like with the dawn of the computer, they will eventually be available to all. We already have some consumer-level Holodeck-like things, like the Microsoft Kinect and the Hasbro my3D for the iPhone, which functions like a small Viewmaster and miniature head mounted display. The questions of whether we can have a Holodeck and whether we should are completely different. As humans, we will eventually be able to make more and more advanced technology. A Holodeck is definitely possible in future, we clearly have similar-feeling things already today. But this doesn’t address whether we should have Holodecks. The answer to this question is deep and similar to the answer to whether we should have video games. “Chapter 1: Mashing the Pleasure Button” in David J. Linden’s book discusses experiments on rats with electrodes implanted into their brains by Drs. Olds and Milner in the 1950’s. They discovered that there was a part of the brain that when stimulated by the electrode made rats feel pleasure and once they gave the rats the ability to stimulate themselves, they would do so to the abandon of all else, including self-preservation. An altered Skinner Box experiment, later researchers moved on from rats to humans. (Linden, 7-9) Rats and humans share similar brain structures, particularly this one which seemed to be in charge of pleasure – a pleasure center that “was much more powerful than any natural stimulus.” On moving on to humans, work done by Dr. Robert Galbraith Heath, humans exhibited similar obsessive behaviors to the exclusion of all else, one 62 subject even being stimulated by a woman in spite of having had homosexual preferences before the study. One woman had ulcerations on her finger where she would fiddle with the implant, trying to increase the intensity. She would also demand it be limited or her access to it be taken away, only to demand later that her control of the implant be restored. (Linden, 11-15) Some rats in the Olds and Milner experiments would press that lever thousands of times a day (Linden, 9). Thousands of times in a twenty-four hour period! Even though these are rats, and the humans studies in a similar vein later (that may seem to some nowadays as morally abhorrent) were small-scaled, clearly a pleasurable behavior is easy to become addicted to for rats, humans, and possibly all creatures. And one could easily argue that the pleasure of playing a video game can be just as addicting as having an electrode shock the pleasure center of a human brain. We have video games already and we certainly have people who cannot bare to tear themselves away from their favorite games. A large part of game design is built on creating pleasure and a desire from the players to return to play more. Similarly, people would likely have the same sense of desire for pleasure derived from the use of a Holodeck. The invitation to engage and immerse oneself in the Holodeck is inherent – it’s impossible (or at least silly) to talk about a Holodeck that humans would never be users of. “5 to 10 percent of Internet users compulsively access Internet applications and, arguably, can be said to be addicted. If Internet addictions are strong, addictions stemming from immersive virtual-reality experiences should be even stronger.” (Blascovich, 183) 63 The Star Trek series has several episodes referring to “Holoaddiction,” an addiction to Holodeck use. Most people in the Star Trek world function well with little or no Holodeck use, though sometimes people become obsessed with the fantasies and other worlds that they can be a part. In Star Trek Voyager’s Season 6, Episode 10, titled “Pathfinder,” Lieutenant Barclay struggles with a relapse of Holoaddiction while simultaneously using the addiction to further his research and make an advancement in communicating with Voyager, stranded far away in the Delta Quadrant, from his station in Earth’s solar system. Through his conversations with Counselor Troy, it is clear that his Holoaddiction revolves around a lack of self-esteem and a general depression. The Star Trek: Voyager series does not delve into Holoaddiction experienced by anyone else, nor does it allude to such a condition present in anyone who is not experiencing other psychological issues as well. While a desire for continued immersion in an alternate world is certainly a factor in video game addiction, it is also possible to play video games and not become an addict. Early content for demonstrations of Holodecks and Holodeck-like experiences should be designed with repeatability in mind, but focus should be less on a requirement to come back later or to continue to play until some number of points or achievements have been accomplished. This may not prevent an addiction to Holodecks or Holodeck-like experiences, but it may be the most ethical way to keep addiction at a minimum. In spite of the possibility of addictions to the pleasure of a Holodeck experience, there are many, many possibilities for Holodecks and Holodeck-like experiences. For example, most high schoolers (perhaps from my generation and before, when software 64 for it was less available or non-existent) have a science lab where they are required to dissect a frog as a learning experience. With a Holodeck program, students could have that exact experience in a simulated environment without ever having to actually acquire a frog. Students who are allergic to the formaldehyde can be a part of the experiment without allergy. This is a simple example, but everything from radical operations to testing new designs for space flight could be simulated realistically on a Holodeck for scientific purposes, not just entertainment. These reasons alone seem to imbalance a fear of potential addictions, since the Holodeck could help unlock many other advancements and betterments for humankind. 6.5 Content Gives Meaning to Technology As a student of Interactive Media, I have had the opportunity to discover that there is so much amazing technology already in existence. I have even come up with ideas for ways to combine those amazing technologies to make something different. They are not new ideas, though. It isn’t that no one has thought of crazy, interesting ideas, it’s that those ideas need funding, time and research, and lots of talented people. A lot of the amazing things I have been privy to seeing would probably not seem so interesting to someone outside of the Interactive Media Division; it is not because those things are not incredible, but because they are not always presented in an accessible way, an issue I have run into myself and am still struggling to understand. A short, thirty second pitch of Shayd is nigh-impossible to clearly tell to someone who hardly has played any games and has never seen or even heard of a head mounted 65 display. If those thirty seconds are spent on describing the technology to a layperson, the excitement of the content itself is lost. Some fantastic technological projects that advance the field do not even get to the engaging content part, they just try to show off the shiny new technical accomplishment with some very simple content for demonstration. But this is a mistake. Thirty second pitch aside, once anyone I have talked to about Shayd understands the concept of the HMD, how it is tracked around a physical space, and the narrative experience to be expected, everyone gets excited. Few people are excited just after learning about the HMD and its tracked physical space. People want something engaging to relate the technology to and understand an experience of. In the realm of advancement towards a Holodeck, much more engaging content needs to be made and made available to as many people as possible. It is the content that will revive the old Virtual Reality boom from decades past and drive more technological advancements towards the Holodeck in future. 6.6 Of Course, All Of This Requires Further Research To prove that content can drive technological advancements toward a Holodeck, more research (and more content) would be required. Content for films have driven technical abilities forward for more than a century, but without further research the same can not be definitely true for Holodecks. Shayd’s design is still in an iterative state and requires further research to continue to make progress in the realm of working towards a Holodeck. Digital puppetry 66 and physical installation user testing could require more design iteration and change the look and feel of Shayd several more times in future with even six more months of research. Shayd’s children’s book series will be continued and completed and might change the way the physical installation is viewed entirely. As Mr. Garlington put it in an e-mail on March 18: “…projects are easiest and generally most successful when they are based upon either a story that is already known … or an archetypal situation that guests have already imagined…” Shayd the book and Shayd the experience are both new stories or experiences and require several months of study and control groups to determine which order receiving one of Shayd’s platforms is optimal for narrative sense and enjoyment. (Garlington Message) Figure 24: Maubwe (left) and Syyrette (right) spend time together perched on a large branch in the alternate dimension of Shayd. 67 The only way to know for sure the effects of a true Holodeck experience is to create that ultimate in alternate reality space. The closer Shayd gets to feeling like a Holodeck, the more support for what a Holodeck could and should be will emerge. In future, more storytellers like myself will be needed to create content for these technological advancements and to help inspire more and more advancements until one day, we are there. 68 Glossary Adaptation – An adapted form of any work, from within the same platform or across platforms. Alien World – In this paper, an alien world is one of three goals in the making of this thesis project. Alien world means anything that is certainly not like Earth, though gravity must remain approximately the same to explain an HMD user’s ability to remain grounded in the same way a human is grounded to the Earth. “Alternate Life Spaces” – A general description by the author of this paper to describe the driving force of the desire to achieve an “Ultimate Display” or the Holodeck, pointing out that this would be a space where the rules of physical life still apply – it requires air, must be tangible, can be edible, must be visual, and obeys its own rules of physics while maintaining an environment that is not hazardous to the user. Augmented Reality - “Augmented reality means to integrate synthetic information into the real environment.” (Bimber, 2) Beautiful World – This author’s definition of a beautiful world is a world that is colorful, different from beauty seen on Earth, and filled with creatures and sentient beings who are peaceful and community-oriented. A beautiful world can also be described as a utopian world. Cross-mode Point-of-view – A term applied to multiple perspectives of the same narrative space across different modes of storytelling. For example, the Shayd children’s book and Shayd the installation are both set in the same story world, however the narrator of the book and the visitor to the installation are only experiencing similar stories with different levels of agencies and different points of view. Expansion – An extension of a work already in existence; this can include a sequel. Head Mounted Display (HMD) – A method of display, often stereoscopic, that is supported by the wearer’s head and close to the eyes of the user to display images that appear similar to the ability of the user’s own eyes. The Holodeck – The Star Trek series features a special room that is computer controlled which can manipulate photons to create tangible spaces and characters. This is essentially what Ivan Sutherland was seeking in an “Ultimate Display.” Imagineers – “Imagination Engineers” who are responsible for the design of the Disney theme parks, also referred to as “Walt Disney Imagineers.” 69 Immersive Illusion – An illusion of any kind that maintains physical immsersion, including tactile, visual, and aural feedback. Interactive Theater – Particularly in reference to an interactive mystery dinner theater or experiences like Medieval Times’ Dinner and Tournament, where the audience is called upon to cheer in participation, though rarely is any member of the audience brought into the stage space. Ivan Sutherland’s “Ultimate Display” – Sutherland wasn’t just seeking wearable technology to change perceptions of the real world, he was seeking a space that could actually change matter around the user. Magic – In this paper, magic does not mean drawing upon a pool of magic or using magic words, like the magic in a fantasy genre. Magic also does not mean simple card tricks from a magician, though this is much more similar. Magic in this paper refers to the elusive feeling one gets while experiencing a story or a space that has been specifically designed to hide its tools, whether technique or technology, and is so engrossed by the illusion of this story or space that the design goes unnoticed. The illusion feels believable and convincingly real. Maubwe – One of two main characters found in Shayd’s alternate dimension. Maubwe is indifferent to explorers, but very devoted to Syyrette. See Appendix H for more information. Mixed Reality – For the purposes of this paper, mixed reality is any virtual space or illusion that does extend in some way into the physical world. Nad’l – The race that often attempts to enslave and pillage the Zoop and the planet Shayd. See Appendix F for more information. Other Worlds – The M.F.A. Interactive Media Thesis Show in May 2012, which Shayd will be a part of. Shayd – Name of the planet explored in the thesis project Shayd. See Appendix A and B for more information. Shayd Mobile – A small, portable version of Shayd known as Shayd mobile in which there is little participation by the user and only one scene. Featured at IEEE VR 2012. See Appendix L for more information. Shayd (Project) – Name of mixed reality installation project, created in partial fulfillment of the requirements of the Master in Fine Arts in Interactive Media in the School of Cinematic Arts at the University of Southern California. 70 Shayd’s life force – Seen only in Shayd’s alternate dimension, Shayd’s life force is colorful glowing cracks in the world that expose the lifeblood of the planet, which helps to rejuvenate the world and its occupants. It is disrespectful to step into these life force pools. Shayd: The Discovery – The tentative title of the Shayd children’s book. Spatial Augmented Reality – Augmented reality combined with spatially-aligned optical elements in order to add three dimensional virtual elements to a physical space. (Bimber, 7-8) Stereoscopic 3D – Human vision supports seeing a three dimensional world because humans have two eyes. Two images that are viewed separately by each eye will be interpreted by a human brain as three dimensional objects with varying depth in the space they occupy. Stereoscopic 3D delivers, via wearable technology (glasses or HMDs), two separate images of scenes from slightly different vantage points on the same horizontal plane. Syyrette – One of two main characters found in Shayd’s alternate dimension. Syyrette is eager to be a guide to explorers. See Appendix G for more information. Teleporter – A method of transportation in Shayd’s alternate dimension. Syyrette and Maubwe have control over these. Theme Park – A space designed with a predominant theme, such as Disneyland. Tormee – One of two alien races found in the alternate dimension of Shayd. See Appendix D for more information. Transmedia – Not adaptation; it must “deliver unique pieces of content over multiple channels.” (Transmedia) Virtual Reality – “An exercise in manipulating … perceptions.” (Blascovich, 14) For the purposes of this paper, virtual reality is any virtual space or illusion that has no extension into the physical world. Wah-zay – One of two alien races found in the alternate dimension of Shayd. See Appendix E for more information. Wienie – Also can be called “the beckoning hand.” This is any object, setting, lighting, or spatial layout that draws the attention of someone moving throughout a space, particularly, but not limited to, a theme park, to subtly draw them to different spaces. Zoop – The race that previously occupied Shayd. See Appendix C for more information. 71 References Anant, Kan. "Re: Phasespace Camera Retail Cost." Message to the author. 20 Feb. 2012. E-mail. "Amusement and Theme Parks." Blue Ridge Online Sales. Web. 10 Feb. 2012. <http://www.goodmagic.com/websales/midway/amuse.htm>. Avatar. Dir. James Cameron. 20th Century Fox, 2009. Film. Bimber, Oliver, and Ramesh Raskar. Spatial Augmented Reality: Merging Real and Virtual Worlds. Wellesley, MA: K Peters, 2005. Print. Blascovich, Jim, and Jeremy Bailenson. Infinite Reality: Avatars, Eternal Life, Universal Consciousness, and the Dawn of the Virtual Age. New York: W. Morrow, 2011. Print. Bolas, Mark T. "Conversation." Personal interview. 23 Feb. 2012. Bolas, Mark T. VES Immersive Experiences. Institute for Creative Technologies, Los Angeles. 23 Feb. 2012. Lecture. "Chess." Wikipedia. Wikimedia Foundation. Web. <http://en.wikipedia.org/wiki/Chess>. "Clarke's Three Laws." Wikipedia. Wikimedia Foundation. Web. 3 Mar. 2012. <http://en.wikipedia.org/wiki/Clarke's_three_laws>. "Disneyland Resort | Welcome to the Magic." Disneyland Resort. Web. 3 Mar. 2012. <http://disneyland.disney.go.com/>. "Frosty the Snowman (TV Program)." Wikipedia, the Free Encyclopedia. Wikipedia. Web. 15 Feb. 2012. <http://en.wikipedia.org/wiki/Frosty_the_Snowman_%28TV_program%29>. Garlington, Joseph O. "Conversation." Personal interview. 12 Mar. 2012. Garlington, Joseph O. "Re: [Shayd] First Move Trailer." Message to the author. 18 Mar. 2012. E-mail. "Georges Méliès." Wikipedia. Wikimedia Foundation. Web. 11 Mar. 2012. <http://en.wikipedia.org/wiki/Georges_M%C3%A9li%C3%A8s>. Gurney, James. Dinotopia: A Land Apart from Time. Atlanta: Turner, 1992. Print. 72 Hamlet on the Holodeck: The Future Narrative in Cyberspace. United States: M I T (MA), 2000. Print. "Head-mounted Display." Wikipedia. Wikimedia Foundation. Web. 15 Mar. 2012. <http://en.wikipedia.org/wiki/Head-mounted_display>. Hench, John, and Pelt Peggy Van. Designing Disney: Imagineering and the Art of the Show. New York: Disney Editions, 2009. Print. "Hope and Fear." Star Trek: Voyager. UPN. Television. "L'Arrivée D'un Train En Gare De La Ciotat." Wikipedia. Wikimedia Foundation. Web. 20 Mar. 2012. <http://en.wikipedia.org/wiki/L%27Arriv%C3%A9e_d%27un_Train_en_Gare_de _la_Ciotat>. Linden, David J. The Compass of Pleasure: How Our Brains Make Fatty Foods, Orgasm, Exercise, Marijuana, Generosity, Vodka, Learning, and Gambling Feel so Good. New York: Viking, 2011. Print. "Live Action Role-playing Game." Wikipedia. Wikimedia Foundation. Web. 18 Feb. 2012. <http://en.wikipedia.org/wiki/Live_action_role-playing_game>. "Magic Kingdom." Wikipedia. Wikimedia Foundation. Web. 10 Feb. 2012. <http://en.wikipedia.org/wiki/Magic_Kingdom>. Milgram, Paul, Haruo Takemura, Akira Utsumi, and Fumio Kishino. AUGMENTED REALITY: A CLASS OF DISPLAYS ON THE REALITY-VIRTUALITY CONTINUUM. Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum. Web. 29 Feb. 2012. <http://etclab.mie.utoronto.ca/people/paul_dir/SPIE94/SPIE94.full.html>. Mulley, Romeo. "Frosty the Snowman." Frosty the Snowman. Dir. Jules Bass and Arthur Rankin, Jr. CBS. 7 Dec. 1969. Television. "Narrative Mode." Wikipedia. Wikimedia Foundation. Web. 19 Mar. 2012. <http://en.wikipedia.org/wiki/Narrative_mode>. "Once Upon a Time." Star Trek: Voyager. UPN. Television. "Pathfinder." Star Trek: Voyager. UPN. Television. Presto. Dir. Doug Sweetland. By Doug Sweetland. Pixar, 2008. Film. Animated short, played prior to the full-length animated feature film "WALL-E." 73 Smith, Daniel. "Why Sony's HMZ-T1 Might Be a Game Changer." The #1 Stereoscopic 3D Gaming (S-3D Gaming, 3D Gaming) Resource. Web. 16 Feb. 2012. <http://www.mtbs3d.com/index.php?option=com_content>. Star Trek: First Contact. Perf. Patrick Stewart. 1997. Film. Star Wars. Dir. George Lucas. By George Lucas. Perf. Carrie Fisher, Mark Hamill, and Sir Alec Guinness. Twentieth-Century Fox Corp., 1977. Film. Star Wars: The Old Republic. Bioware, 2011. Game. Strange Days. Dir. Kathryn Bigelow. Twentieth Century Fox, 1995. Film. Tetris. Bullet Proof Software, 1989. Game. "Transmedia Storytelling." Wikipedia. Wikimedia Foundation. Web. 13 Mar. 2012. <http://en.wikipedia.org/wiki/Transmedia_storytelling>. "View-Master." Wikipedia. Wikimedia Foundation. Web. 18 Mar. 2012. <http://en.wikipedia.org/wiki/View-Master>. "The Wizarding World of Harry Potter - Only at Universal Orlando." Universal Studios Orlando. Web. 22 Feb. 2012. <http://www.universalorlando.com/harrypotter/>. World of Warcraft. Blizzard Entertainment, 2004. Game. 74 Appendix A: Shayd Tormee/Wah-zay Side (Virtual World) The green sun in Shayd’s solar system casts rays that make the planet appear to always be in twilight. The two moons that slowly orbit each other and the planet cast bright reflections onto the planet so it rarely ever appears to be truly dark nighttime. The environment and creatures on Shayd are all rounded, colorful, and beautiful. The air moves gently through the world, calmly stirring the sparkles that waft around, both rising and falling and never quite managing to settle on the ground. Many creatures are small, furry, and symmetrical, move quickly and gracefully, and can be seen frequently catching rides with the Wah-zay. Several creatures are massive and round and move slowly. Feathers are less common and few creatures are capable of flying, though those that can are still very rounded and fly gracefully (more like a hawk than a bat or a hummingbird – there are no lines involved in flying for creatures on Shayd). 75 Appendix B: Shayd Zoop Side (Physical Installation) With the Zoop scattered into colonies elsewhere in the solar system, most of their most precious sites, like the caves with painted histories on the wall and holographic guardians watching over the use of the alternate dimension portals to the other side of Shayd. The world is healing, but still appears desolate. Most plant and animal life is barely coming back to life and the world smells of dust and dirt. Some glowing green light from the planet’s green sun can be seen through the entrance to the cave. 76 Appendix C: Zoop The Zoop are an incredibly technologically advanced species originally native to Shayd. After a meteor devastation many, many years ago, the Zoop branched away from Shayd, settling on nearby planets in the solar system and some colonies further away and concealing their new colonies by creating self-sustaining environments. The player, when roaming the universe, finds Shayd attractive because it looks clear that something happened to the planet but it is now on the mend. The player misses the Zoop in their travels because the design of the Zoop’s colonies don’t jive with the player’s detection methods. Part of the background information given to players before they enter the physical cave will be that they didn’t detect any nearby life through their known methods. The Zoop created these “portals.” Their artwork is all over the inside of the cave where the player finds the technology. Their cave drawing artwork depicts the concept of what the portals do – they send the Zoop into the dimension where the Tormee and the Wah-zay exist. It also shows the Zoop have the ability to project themselves into the other “side” of Shayd and are able to move freely and interact with objects freely. In the art work there is some vague reference to others trying to use the technology and having a less successful experience, but the predominant art work is instructions on using the technology and about who will be found on the other side. The Zoop stand approximately 2–2½ feet tall. 77 Appendix D: Tormee The Tormee stand between 4 and 4½ feet tall. The Tormee have four fingers, thin, curved, with rounded tips on each hand, one of which acts as a thumb and is not quite clustered with the other three. They are symmetrical with two arms, two legs, and two eyes. Their eyes are oblong with oblong irises that stretch up and down their eyes. They have filmy eyelids that blink but are mostly unseen. They have delicate, nearly invisible scales that make them look shimmery in intense light, but is otherwise hard to make out. The Tormee are the race on Shayd that reign supreme in technical manipulation – that is, they have fingers and control the world tangibly. They rely mostly on technology that they can have a physical interface with – whether it’s a simple tool or something more advanced. On Shayd, nothing creates pollution, so the Tormee’s technology does not appear metallic, shiny, or gas-powered the way human technology looks and moves. The technical advancements here look like magic to human eyes because they are so well integrated with the planet itself. 78 Appendix E: Wah-zay The Wah-zay stand between 3 ¼ and 3 ½ feet tall. They are asymmetrical beings with a single large eye. They have soft, incredibly short fur all over their bodies. They have two feet to help them keep their balance while they walk – upright – and two stubby arm/appendages that help them to come into physical contact with the world. Their lack of fine motor coordination and lack of a mouth is compensated for with their telepathy and telekinesis. They also have thick, wiggly “feelers” which are not antennae but do act as receivers for physical information about the world around them and a method of nourishing their bodies. The Wah-zay have more trouble interacting with races that are incapable of telepathy and so rely on the Tormee to do the vocalizing for them, though the Tormee are capable of telepathy as well. Normal interactions with the Zoop via the portals were slowed a bit because the Tormee needed to vocalize as mediators. The Wah-zay are gentle though they move awkwardly and are nature’s balance on Shayd – caring for and bonding with the animals of the planet. They are often followed by “familiars.” 79 Appendix F: Nad’l The Tormee and the Wah-zay know less about the Nad’l than the Zoop because they have never actually met the Nad’l. The Nad’l are a race that do not exist anywhere in the Tormee and Wah-zay dimension. They do not live in Shayd’s solar system, but come from nearby. They attempted to take Shayd and control the Zoop, but the Zoop are far more technologically advanced and the Nad’l didn’t even realize it. The Zoop consistently fended off the Nad’l for a thousand years. Then, a meteor (that did not occur to the Tormee and Wah-zay side) devastated Shayd and the Zoop took up primary residence on their colonies on other planets in the system. The Zoop are small, and their colonies are hard to detect by the Nad’l and other races not as advanced. The Nad’l want the Zoop’s technology, particularly the portals if they could get their hands on them. Particlarly while Shayd is towards the end of its healing and becoming life- sustaining again, the Nad’l’s efforts have increased and the Zoop frequently patrol Shayd for signs of them. The Zoop infrequently use the portals while the Nad’l are still on the prowl and while they are guarding Shayd instead of living on it, but they will send messengers sometimes to use the portals to let the Tormee and Wah-zay know of any danger. The Zoop have also made sure that the Tormee and Wah-zay know what a Nad’l looks like. 80 Appendix G: Syyrette Syyrette (Sur –eht) is a true blue (or perhaps slightly deep blue, just not purple) skinned Tormee. He is 4 feet 5 inches and has black hands/fingers, black feet, and black hair on his head. His eyes are a slightly different shade of blue than his skin. Syyrette is the first alien the player will interact with. Any aliens around will not really interact with the player except to be confused at the player’s presence. Syyrette is the “Ron Weasley” to the player’s “Harry Potter” upon entering into the world of Shayd. He’s likeable and highly intelligent, which is clear from his ability to correctly interpret the player’s communication attempts with only a reasonable amount of trouble upon two completely different races meeting. Syyrette’s going to also be the storyteller for the player. He will show players little “hologram images” of the “story” of what happened to the Zoop and perhaps a live- view of what’s going on when the Nad’l are attempting an invasion (which ends the experience for players). Syyrette’s closest Wah-zay friend is Maubwe, the grumpier of the pair, who Syyrette very nobly tolerates and understands. He is very patient and very upbeat. Excitement sparks from learning, interacting with Shayd technology, interactive with natural elements of Shayd, interacting with Maubwe, interacting with the player, and telling stories (communication). Syyrette will be frustrated and disappointed if the player refuses to interact with him or is not interested in his stories. Syyrette will also try very hard to understand communications from the player requesting the player’s story. 81 Appendix H: Maubwe Maubwe (Mahw –bwey) is a bright yellow skinned Wah-zay. He stands at 3 feet 4 inches. His head antennae are a bright orange and the iris of his single eye is also the same orange color. Syyrette is his best friend and dedicated interpreter for the player, as Maubwe cannot telepathically communicate with the player as he would with anyone else on Shayd. Maubwe is moody and will clearly interject in Syyrette’s attempts to tell the story of Shayd and Syyrette will pause momentarily while Maubwe telepathically explains himself. (This means to the player, everyone will be quiet and perhaps interact gesturally with each other, but momentarily not be attempting to do so with the player.) Maubwe tries very little to directly communicate via gestures to the player and gives up quickly as he cannot move as finely and as gracefully as this communication requires. Maubwe is a committed partner in the duo of his and Syyrette’s friendship and so when Syyrette is so immediately friends with the player – the new stranger – this friendship does actually extend to include the player as a member of a newly formed trio. Maubwe’s grumpier appearing mood is simply because communication is frustrating. Ultimately, Maubwe does get to happily showcase some of his animal friends to the player. He has a small, four-legged strong and quick little herbivore who never strays particularly far from him, but does spend a decent amount of time just out of the player’s reach (sort of in the background). This creature appears to be a rounded, rabbit-like animal with big eyes – three of them. 82 He’s sweet, appears clumsy and almost Frankenstein-like in his walk because of his asymmetrical body, and moves around and grips things similar to the way that a stuffed animal might be animated in a Pixar movie (think of the strawberry-scented, meant-for-hugging evil teddy bear in Toy Story 3). He’s also very gentle and has a quiet, shy-seeming way about him due to his lack of a voice box. While Syyrette is always interested in communication and being a storyteller, Maubwe is happiest passively being the “wing man” and occasionally enjoying the animals rubbing past him or interacting with him (a bit as an aside). 83 Appendix I: Shayd Children’s Book Excerpt, Pages 1-12 (--- represents a page break in the book – illustrations are forthcoming at the time of this writing) I come from a place far away from here. --- I come from a world that had no peace. There was so much violence and greed and many people were slaves in one way or another. --- I come from a world that would not, could not work together. I come from a world where “freedom” did not mean that we were really free. --- There were dark times and more dark times and it seemed never-ending. --- So I left my angry, hurting planet and I traveled. --- There I am, that’s me in my space ship. --- I traveled alone and I felt peace, but I held out hope that not everyone was like the ones I left behind. --- I traveled long and far until one day I found a planet that seemed to have once been like 84 my home planet was ages ago. This planet, too, was cloudy with dust and a sense of death, but did not seem inhabited. It seemed as if hundreds of years before it had suffered some great planet-wide tragedy. --- But now it was on the mend. The color was starting to return slowly under the dusty terrain. The air was clean again, enough for me to breathe, and so I landed my space ship near a cave and looked around. --- The planet was far from its green sun, which made the planet seem dark, in a perpetual twilight. It was easy to see the stars in the sky even though the sun was high above. The moon also reflected a warm green hue in the soft light. --- I found no one to greet me, so I explored further. The cave was quiet, with a very soft hum of what seemed like music – the planet’s music? – I wasn’t sure. --- On the walls were paintings. Many, many paintings. --- 85 Appendix J: Shayd Children’s Book Excerpt, Last 3 Pages (--- represents a page break in the book – illustrations are forthcoming at the time of this writing) After a long silence between us, I realized that Maubwe had come and sat beside Syyrette and was watching him. I thought they were having a conversation that I couldn’t be a part of, so I tried to wait patiently. Finally, Syyrette gestured to my head. I slowly moved to take it off, feeling that my time on Shayd must be over, but he stopped me and made a gesture for me to take it off, paused briefly, and then gestured for putting it back on. --- I would go back onto my ship tonight and sleep, and tomorrow, I’d come back to look for Syyrette and Maubwe. Perhaps after some time spent with Syyrette and Maubwe, I’d seek out the Zoop I now knew were nearby and try to fit in with them. I’d also be sure to be on the lookout for the Nad’l, and help to defend this beautiful planet and my new friends. --- I took off the technology and placed it back where it belonged. I stared at the story on the wall again, having a better understanding of it, and had a good night’s sleep on my ship, ready to return tomorrow. 86 Appendix K: Tormee Language Lyrics Lyrics written for live music recording in Tormee language, with English translations. Shayd is our shelter where we live together Kii’lii’ku dur colayha lii Shayd wehro This planet cares for us as we care for it Feyrata vett tor-ku pahsay dur zee-trow Tormee and Wah-zay symbiosis Tormee dur Wah-zay kii’lii’sore-ay Peacefully we cooperate Colay’ku dur kii’lii’zehr Our friends the Zoop found us long ago Shay’pee’na Zoop tor-weena teff pa too They watch and protect the pathways Col’ku’zehr lii dur Shayd kii’lii’colay And we welcome them with open arms Dur Zoop lii colayha 87 Appendix L: Shayd Mobile Shayd on iPhone using FOV2GO, featured at IEEE VR, March 2012 (For 2D viewing of FOV2GO version a.k.a. Shayd Mobile) (For 3D free viewing: This is how the image would look on the iPhone, which would be paired with a viewer from the Mixed Reality Lab.)

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Seymour Deeply: exploring stereoscopic 3D as a storytelling tool in interactive media

PDF

Montage of interaction: conceptual exploration and creative practice

PDF

Developing a playful situational creator for mixed reality: design analysis on Neon City, a city building game

PDF

Penrose Station: an exploration of presence, immersion, player identity, and the unsilent protagonist

PDF

American braves - total aesthetics: an opinion piece on game design for academics

PDF

Untitled: an exploration of virtual reality as a cinematic medium

PDF

Virtual production for virtual reality: possibilities for immersive storytelling

PDF

The illusion of communication

PDF

Creatively driven process: using a virtual production workflow to achieve a creative vision

PDF

Psynchrony: finding the way forward

PDF

Silent sparrows: exploration into augmented reality narrative frameworks

PDF

The ritual model: how to use the mechanics of ritual to create meaningful games

PDF

The return: a case study in narrative interaction design

PDF

Transference

PDF

Video Volume

PDF

Ascension: a look into context-dependent memory development in virtual worlds

PDF

Morana: explore healing potential of virtual reality storytelling

PDF

TCam: Touch driven communication for virtual moviemaking

PDF

Combiform: a console for the new communal casual game genre

PDF

Grandma Green: kindness in bloom

Asset Metadata

Creator Griffo, Juliana Christine (author)

Core Title Shayd: the pursuit of magic, illusion, and interactive worlds

School School of Cinematic Arts

Degree Master of Fine Arts

Degree Program Interactive Media

Publication Date 05/05/2012

Defense Date 03/26/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag digital puppetry,head-mounted display,illusion,Interactive Media,interactive worlds,magic,mixed reality,motion capture,OAI-PMH Harvest,Shayd,virtual reality

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Bolas, Mark T. (committee chair), Fisher, Scott S. (committee member), Garlington, Joseph O. (committee member)

Creator Email griffo@usc.edu,jshmans@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-32147

Unique identifier UC11288303

Identifier usctheses-c3-32147 (legacy record id)

Legacy Identifier etd-GriffoJuli-777.pdf

Dmrecord 32147

Document Type Thesis

Rights Griffo, Juliana Christine

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA