University of Southern California Dissertations and Theses

The extension of our reality

(USC Thesis Other)

The extension of our reality

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content THE EXTENSION OF OUR REALITY
by
Autumn Auriel Collins
A Thesis Presented to the
USC SCHOOL OF CINEMATIC ARTS
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF FINE ARTS
(INTERACTIVE MEDIA)
May 2024
Copyright 2024 Autumn Auriel Collins

Table of Contents
List of Figures.................................................................................................................................ii
Abstract......................................................................................................................................... iii
Chapter 1: What is Extended Reality?...........................................................................................1
Chapter 2: The Virtual Experience.................................................................................................2
Chapter 3: The History of Mining...................................................................................................3
Chapter 4: The Game Experience.................................................................................................5
Chapter 5: The Physical Experience............................................................................................. 7
Building the Installation............................................................................................................8
Chapter 6: Agency and Choice....................................................................................................18
Testing the Experience.......................................................................................................... 19
Chapter 7: How Art Evoke Emotions........................................................................................... 21
Chapter 8: The Future of Extended Reality................................................................................. 22
Bibliography.................................................................................................................................23
ii

List of Figures
Figure 1: Infinity optics system installation sketch.........................................................................9
Figure 2: Arduino sensor system sketch......................................................................................10
Figure 3: Foam core model..........................................................................................................10
Figure 4: Smaller scale prototype................................................................................................ 11
Figure 5: CNC router cutting wood.............................................................................................. 12
Figure 6: Frame assembly...........................................................................................................13
Figure 7: Mirror mylar attached to frame..................................................................................... 13
Figure 8: Frosted dome lens........................................................................................................14
Figure 9: Final installation set-up.................................................................................................15
Figure 10: Projection without frosted dome lens..........................................................................16
Figure 11: Projection using frosted dome lens.............................................................................16
Figure 13: Rear projection installation......................................................................................... 17
Figure 14: Taurus Rising projected - control panel...................................................................... 20
Figure 15: Taurus Rising projected - moon surface.....................................................................20
iii

Abstract
Our reality is evolving. With the rapid development and integration of technology,
our lives have become increasingly dependent on it. These expressions of our reality
tell us when to wake up, what to eat, who is communicating with us, where to go and
how to get there. However, these technologies reach their limits when trying to engage
with human emotions and empathy. In this paper, I explore the current and future
technologies of extended reality experiences and how these technologies can be used
to elicit emotion and help players make meaningful choices.
iii

Chapter 1: What is Extended Reality?
Extended Reality (XR) is a term that refers to immersive experiences that
combine the physical and virtual worlds. The realm of Extended Reality also includes
Augmented Reality, (real world environment that is digitally augmented),Virtual Reality
(full immersion in the virtual world), and Mixed Reality (digital objects interact with real
world environments).
Extended Reality (XR) brings the innovation and imagination of a virtual
simulated world while also allowing the present world to peer through. This type of
enhanced reality is better for our human brains than a total virtual environment like
virtual reality because our brain uses our five senses to develop situational awareness
and help ground us in the real world. Once that awareness is taken away, it takes away
our sense of comfort and easiness (Rosenburg).
For my thesis, I decided to create an Extended Reality installation experience
that combines the virtual and physical worlds in order to further the degree of immersion
in the experience.
1

Chapter 2: The Virtual Experience
For the virtual part of the experience, I created a 3D first-person adventure game
called Taurus Rising using Unreal Engine 5. Taurus Rising is about Orion, a planetary
geologist for the Space Frontier, who is on a mission to a faraway galaxy to harvest
minerals that can be sent back to an extremely resource-scarce, dying and depleting
Earth. Orion uses his harvesting equipment to scan rocks and asteroids for valuable
minerals. He then breaks these rocks and asteroids and vacuums these resources to
send home. Orion is accompanied by his AI companion Inanna and regularly is in
communication with the Space Frontier director, Zenia, and colleagues Aster and
Polaris who are also deployed on the mission.
All goes well in the mission until Orion and Aster discover that there is plant life
around the region he was sent to explore, the Centaurus A galaxy - something mission
planners hadn’t predicted in their studies and research. They soon realize that certain
minerals such as Calcite and Olivine are essential for life growing in this galaxy. While
having full agency, the player, asOrion, has to decide whether to continue the mission
and send the resources back to Earth or to end the mission and save all the life in
Centaurus A.
The inspiration for Taurus Rising comes from the real world concept of space
mining. In October 2023, NASA launched a spacecraft to land on the asteroid 16
Psyche in August 2029 that orbits the Sun between Mars and Jupiter. NASA wants to
research the metal-rich asteroid to explore planet formation. It is also possible that in
the far future, 16 Psyche could be explored for mining possibilities since that asteroid
alone has minerals worth $100,000 quadrillion dollars.
2

Chapter 3: The History of Mining
With space possibly being the next great frontier, it is concerning that the mining
exploitation done on Earth might be replicated extraplanetary. The earliest known record
of mining dates back to 40,000 to 20,000 years ago in southern Africa. The mine found
was a coal mine. A recent discovery even speculates that mining could even date back
to 300,000 years ago with the first humans mining for flint. However, the mining industry
didn’t become more advanced until about 10,000 to 7,000 years ago. Many ancient
civilizations were wealthy due to extractions from mines. Ancient Athens became
wealthy from its silver mines. Ancient Egypt was one of the first civilizations to mine for
gold on a massive scale (Coulson, 2012).
The history of mining isn’t complete without mentioning the other side of
extracting these minerals. Conflict, exploitation and abusive labor systems have arisen
due to the mining industry. Tin, tungsten, tantalum and gold have become known as
conflict minerals because these minerals are often tied to armed-conflicts and human
rights abuses. “Blood Diamonds” from Sierra Leone and Liberia even capture the issue
in their name. All of these minerals have to go through extensive review and due
diligence to determine that they were humanely extracted.
Though some minerals go through regulation, not every mineral has extensive
laws to protect them. One example of this is cobalt. Cobalt is used in lithium ion
batteries. These batteries are used in smartphones, laptops, tablets and electric
vehicles. A battery pack for an electric vehicle requires 1000x more cobalt than a
smartphone. Outside countries have taken over the cobalt mines in East Congo and
3

forcibly work the locals in these artisanal mining sites for as little as $1 a day (Searcey,
Forsythe, Lipton, Gilbertson 2021).
Mining has always been a means to take over foreign lands for resources and
use these resources to make the way of life in a wealthier country better. In my project, I
want to highlight that the extraction of minerals from outer space could be just as
exploitative as what is happening here on Earth.
4

Chapter 4: The Game Experience
I want players to understand both sides of the situation - their people need
resources to help the colonies thrive, however, it comes at the expense of the foreign
land and the surrounding civilizations that depend on these resources. Ultimately, I do
not want to tell the players what is the right decision to make. At the end of the
experience, I want each player to make their own choice of whether or not they want to
send the resources they’ve extracted back to Earth or send the extracted resources to
the surrounding civilizations who depend on them. It’s important for me to give players
the information and give them the opportunity for them to make their own decisions.
While developing the game, it was important for me to redesign, adjust and adapt
to the iterative process because the game has to feel immersive to humans. One key
design choice I made was to have the player play as Orion through first person
perspective so that the player can connect with Orion through his eyes.
Another important design choice that I discovered while iterating was to add
compound gameplay. In the first version of the game, the game mechanics of scan,
harvest and destroy each worked for certain rock types. In the final version of the game,
the player needed to scan and destroy each rock before they were able to harvest the
minerals. Adding compound gameplay mechanics, allows the player to feel more
immersed and engaged with the work needed in order to be successful in space mining.
A third important design choice that I added to the game was "voice over,”
dialogue and an extensive narrative. Games become more real when there are human
elements attached to the characters such as a human voice, a human way of speaking
and a way of conveying human emotions and problems. It was important to make the
5

characters Zenia, Aster and Polaris different from each other in order to represent
different spectrums of the human experience. It was also vital to be sure that Orion, as
the player character, remains neutral throughout the majority of the game so that
players can project their own way of feeling and their own thoughts into Orion so that
they can relate to him and see themselves as him. If Orion had a fully fledged
personality with strong opinions, then players wouldn’t feel that Orion was an extension
of themselves.
6

Chapter 5: The Physical Experience
For the physical aspect of the experience, I decided to create a physical infinity
optics installation. The installation consists of a roughly 5 foot by 8 foot frame with a
stretched piece of silvered mirror mylar that goes across the frame and a vacuum is
attached to the back of the frame. When the vacuum turns on, the air stretches the
mylar into a concave shape, ideally a section of a sphere. Hanging from the ceiling is a
video projector. It projects the video game onto a concave rear projection screen (RP
screen) also a spherical section that is mounted concentrically with the concave mylar
mirror noted above. The light then travels parallel through the RP screen, is reflected off
of the concave mirror and into the eyes of the player, who is seated between the RP
screen and the mirror. The player sits close to, but somewhat below the centerline of the
axis created by the projector, the RP screen and the folded beam path created by the
mirror. The player is centered, horizontally, in the middle of the installation right behind a
table. The table has the game controller and motion tracker camera built into it so that it
can read the player’s gestures when playing the game.
I chose to use the installation medium for a couple of different reasons. The main
reason is to invoke our senses. Being in an environment that allows the player to look,
feel and touch something tangible should invoke a more meaningful impact on the
player compared to playing a digital game behind a screen. Here, the “infinity optics
system” makes things feel big, outer space big, which helps the player feel they are truly
in this extraterrestrial place.
Another reason for choosing to create an installation project is because it can be
converted into a mixed reality experience. Mixed reality melds together the virtual and
7

physical worlds in a way that virtual reality can not do. Virtual reality helps players to
disconnect from the world. However, for this project it is important for the players to stay
grounded in reality so that it can invoke real feelings such as empathy.
This project uses infinity optics whose goal is to make far-away things feel truly
far away. Infinity optics do this by columnating the light coming from the projector so that
all the rays of light reaching the user’s eyes are parallel. Humans (and most other
animals, too) recognize columnated light as being light coming from a distant source.
This idea is great for the immersion of Taurus Rising because Taurus Rising is
set in different alien terrains in outer space and the player is often alone. The infinite
space will evoke vastness and eeriness as well as loneliness.
Building the Installation
Before the building of the installation can begin, a proof of concept prototype
must be made. The image below shows the initial sketch of the Infinity Optics system. It
shows a projector projecting an image through a rear screen and into a concave shaped
mirror which reflects the image into the player’s eyes. It is set up to fit within a 10 foot
by 10 foot space.
8

Figure 1: Infinity optics system installation sketch
Attached to the frame is the sensor system. The sensor system works like this:
The arduino and the vacuum is attached to an AC/DC control relay. The sensors let the
arduino know that the mylar has stretched just the right amount and to turn off the
vacuum. A bread board along with a 9V battery would be connected to the arduino to
ensure the sensors have enough power. When the vacuum stretches the mylar mirror
into the proper curve, the belly of the mirror blocks the view of the receiver from the
emitter. Once that is blocked, the arduino turns off power to the vacuum.
9

Figure 2: Arduino sensor system sketch
The first step of the installation process was to visualize the space without
making any commitments. This was done with a miniature foam core model built to
scale. The image below shows a mini foam core model of the installation with a piece of
mirror mylar attached
Figure 3: Foam core model
10

After constructing the foam core model, a small scale wooden model prototype
was created. Mirror mylar attached to it in order to test the vacuum stretching. After the
wood was cut, I took the pieces to the IYA Creator Space to carve out curves out of one
side per piece.
Figure 4: Smaller scale prototype
Once the prototyping and measuring was complete, it was time to build the actual
installation. We took the measurements of the frame and created an illustrator file of all
four sides. The file was then placed into the CNC router so that the machine could
perfectly cut the curves and bevels on each side of the frame.
11

Figure 5: CNC router cutting wood
When the CNC router was done, the next step was to assemble the all four sides
frame together supported by a flat piece of wood and wooden beams. A hole was drilled
into the back to create a space for the hose of the vacuum to sit. Two smaller holes
were drilled in the top and bottom of the frame to create a space for the emitter and
receiver light beam sensors. The mirror curve cycles slightly due to the off and on
suction provided by the vacuum, but the subtly changing radii of the curved mirror is not
enough to distract the viewer playing the game.
12

Figure 6: Frame assembly
When the assembly was done and the frame was sanded down, it was time to
carefully attach the mirror mylar using wood glue and duct tape. The type of mirror
mylar that I purchased was a bit transparent, so the inside of the frame was painted
black to stop the light reflection from going past the mylar.
Figure 7: Mirror mylar attached to frame
13

After the frame was finished, it was time to assemble the 4 foot dome onto the
stand. Once the dome was bolted on, the dome was then sanded until the plastic finish
was smooth and frosty and there was no reflection.
Figure 8: Frosted dome lens
The final result of the construction is the Infinity Optics system! As shown in the
photo below, the Infinity Optics is essentially a lens that shines through a concentric
surface and alters the rays of lights to create an illusion of infinite space and depth.
14

Figure 9: Final installation set-up
Shining the projection onto the mirror without the frosted dome wouldn’t give it
the same effect. The frosted dome is needed to create the illusion and to distort the
projection image in a way that is interesting and alluring to the player. Below is an image
of the projection without the dome next to an image of the projection using the dome.
After setting up the installation in place, it was quickly discovered that the
concentric circles of the frosted dome and the mirror needed to not be the same radius.
As shown in the below image, having the circles be the same radius results in a small
image. If the frosted dome had a bigger radius, it would result in a bigger image.
15

Figure 10: Projection without frosted dome lens
Figure 11: Projection using frosted dome lens
Because of this discovery, I changed gears and created a large rear projection
installation that consisted of a 9 foot by 6.3 foot curved frame and rear projection fabric
16

stapled across. This quick change from the Infinity Optics to Rear Projection was
needed in order to test if scale contributes to player agency, choice and immersion.
Figure 13: Rear projection installation
17

Chapter 6: Agency and Choice
Player choice stems from a myriad of factors. A choice can be made due to
personal preference, environment, or past history. Because this decision can be
affected by various elements, it was important for me to design an experience that
controls these factors and write a narrative that is meaningful.
In Taurus Rising, the player has to make a choice. The player isn’t aware of this
important choice until they come to the end. They spend almost the entire game
playing through a narrative that feels predestined. However, it is the goal of the
installation to make players feel empowered to explore the world in the game so that
when it’s time, they can make a choice that feels meaningful to them. “Agency is the
practical function of a character to serve as representation of the player in the game”
(Fullerton 109).
The meaningful choice in Taurus Rising comes down to making a decision
between continuing the mission or sabotaging the mission. The mission is to gather
resources and minerals from the Centaurus A galaxy to send back to a dying, depleting
and resource scarce Earth. However, throughout the game, the player will soon realize
that there is significant life on the planet that relies on these resources to live. The
player will need to decide whether or not to save Earth or to protect Centaurus A.
One of my design challenges is to invoke empathy in the player so that the
decision that they make would be a meaningful one. “Empathy is the potential for
players to develop an emotional attachment to the character, to identify with their goals
and, consequently, the game objectives” (Fullerton 109). Combining the idea of
empathy, agency and the unparalleled immersion that an installation experience can
18

bring, I believe that the player should make a choice that they feel most emotionally
drawn to.
Testing the Experience
With the installation fully assembled, it was time to begin user testing. While
testing the game using a PC, most players were concerned with getting the highest
score and collecting the most minerals they could find. This is because games typically
reward players for being the best at the game. A few players were able to notice that
Calcite would increase their score, but decrease the Life Radar. The ones that noticed
began to avoid Calcite, not because they were making a meaningful choice in response
to the game, but because they wanted to “win” and do good.
While testing the game in the installation, every player expressed how immersed
they felt in the experience. One player expressed how he felt like he was on the moon
and that the surface of the moon was wrapped around him. However, though the scale
contributed to the immersion for the players, it didn’t contribute to player choice and
agency for every player. Some players were concerned with protecting the plant life on
the moon surface while others wanted to collect as many minerals as they could.
Though the common player goals differed from players on the PC versus players in the
installation, the type of player a person may be, whether someone is more interested in
getting a high score or interested in story, definitely contributes to a player’s behavior.
19

Figure 14: Taurus Rising projected - control panel
Figure 15: Taurus Rising projected - moon surface
20

Chapter 7: How Art Evoke Emotions
Just like artworks of the past, immersive technologies can evoke emotions. Art
uses color, light, narrative, real world, and scale to convey different emotions. A study
by Lauri Nummenenmma and Riita Hari found that “Aesthetic emotional experiences
associated with encountering visual arts are strongly embodied and that visual arts can
elicit a broad range of emotional feelings that go significantly beyond the canonical
“basic” emotions” (Nummenenmma, Hari). If immersive technologies can use these
elements of art correctly, then they can evoke strong emotions in people which is what I
hope for in the Taurus Rising installation. Taurus Rising uses vivid color, physical and
digital light, solid narratives, scale and a real world physical installation to elicit emotion
in players and help them make meaningful choices.
21

Chapter 8: The Future of Extended Reality
The future is bright for XR. Companies and organizations like the United States
Air Force, Apple, Microsoft, Magic Leap and Unity have all been in the forefront in
ushering in mixed reality products. Most of these products consist of a headset that’ll
allow consumers to traverse the virtual and physical world with simple mechanics in the
search of increased productivity or brief entertainment.
However, besides productivity and brief entertainment, XR can be used as an art
form to convey and evoke certain emotions in the user. It can tell a story or send a
warning. Our human brains react to certain images and displays either favorably or
negatively depending on the subject. XR can go further than traditional artwork can by
using physical elements in order to physically influence our senses. Through the rapid
progression of immersive technology, our reality is evolving and extending and the
physical and digital worlds are melding together. With this in mind, it will be important for
us not to lose grasp of the reality that came before us so that we can have a better
future.
22

Bibliography
Nummenmaa, L., & Hari, R. (2023). Bodily feelings and aesthetic experience of
art. Cognition and Emotion, 37(3), 515–528.
https://doi.org/10.1080/02699931.2023.2183180
Fullerton , T. (2019). Game Design Workshop: A playcentric approach to creating
innovative games. CRC Press Taylor & Francis Group.
The history of mining: The events, technology and people involved in the industry
that forged the modern world. (2012). . Harriman House Publishing.
Louis Rosenberg, P. (2023, January 27). Mixed reality is our digital future.
Medium.
https://medium.com/predict/mixed-reality-is-our-digital-future-ec2d927ced01
NASA. (n.d.). Psyche - NASA science. NASA.
https://science.nasa.gov/mission/psyche/
Searcey, D., Forsythe, M., Lipton, E., & Gilbertson, A. (2021, November 20). A
power struggle over cobalt rattles the Clean Energy Revolution. The New York
Times. https://www.nytimes.com/2021/11/20/world/china-congo-cobalt.html
Sohn, R. (2023, October 12). Metal asteroid psyche has a ridiculously high
“value.” but what does that even mean? Space.com.
https://www.space.com/psyche-metal-asteroid-composition
23

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Silent sparrows: exploration into augmented reality narrative frameworks

PDF

Untitled: an exploration of virtual reality as a cinematic medium

PDF

Installation wizard: how communication makes or breaks a game

PDF

Cards of Heart: a cozy role-playing card game for promoting mental health

PDF

Jizo No Akumu: A psychological horror game exploration of generational trauma and mixed-race identity

PDF

Developing a playful situational creator for mixed reality: design analysis on Neon City, a city building game

PDF

Montage of interaction: conceptual exploration and creative practice

PDF

Penrose Station: an exploration of presence, immersion, player identity, and the unsilent protagonist

PDF

Free Will: a video game

PDF

The Glitch Witch

PDF

The mountain calls: an exploration of immersive storytelling through art and level design

PDF

Garden designing a creative experience with art and music orchestra

PDF

Exploring space literally & figuratively in Eight Moons to Eternity

PDF

Grandma Green: kindness in bloom

PDF

Hearth and home: interior design as tool and weapon in creating an atmosphere of dread

PDF

Oopz-Oofs: the creation of a dog simulator game

PDF

Virtual production for virtual reality: possibilities for immersive storytelling

PDF

A game called Paako: the challenge of an autobiographical video game

PDF

Past imagination, present creation and the reality of tomorrow: explore the impact of Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) design

PDF

Wandering light: an interactive visual novel

Asset Metadata

Creator Collins, Autumn Auriel (author)

Core Title The extension of our reality

School School of Cinematic Arts

Degree Master of Fine Arts

Degree Program Interactive Media

Degree Conferral Date 2024-05

Publication Date 05/17/2024

Defense Date 05/16/2024

Publisher Los Angeles, California (original), University of Southern California (original), University of Southern California. Libraries (digital)

Tag agency,augmented,Choices,extended,Games,installation,meaningful,Mining,Mixed,OAI-PMH Harvest,player,reality,space,technology,Unreal Engine,video,virtual

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Huntley, Jim (committee chair), Bilson, Danny (committee member), Lemarchand, Richard (committee member)

Creator Email aacollin@usc.edu,autumn.auriel@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113939772

Unique identifier UC113939772

Identifier etd-CollinsAut-12925.pdf (filename)

Legacy Identifier etd-CollinsAut-12925

Document Type Thesis

Format theses (aat)

Rights Collins, Autumn Auriel

Internet Media Type application/pdf

Type texts

Source 20240517-usctheses-batch-1152 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu