University of Southern California Dissertations and Theses

Aesthetic driven design of Space Maestro

(USC Thesis Other)

Aesthetic driven design of Space Maestro

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content i

AESTHETIC DRIVEN DESIGN OF SPACE MAESTRO

by

Jonghwa Kim

A Thesis Paper Presented to the
FACULTY OF THE USC SCHOOL OF CINEMATIC ARTS
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF FINE ARTS
(INTERACTIVE MEDIA)

August, 2013

Copyright 2013 Jonghwa Kim
ii
TABLE OF CONTENTS

List of Figures Iii
Abstract iv
Introduction
Aesthetic-Driven Design
Prior Art
1
2
5
Research and Development
Paper Prototype
Digital Prototype - Iteration 1
Digital Prototype – Iteration 2
7
7
9
11
Results
Setup Requirements
How to Play
13
13
14
Conclusions
Future Development & Areas to Research
Epilogue
15
15
16
Works Cited 17

iii
LIST OF FIGURES

Figure 1: MDA Framework

1
Figure 2: Palette

2
Figure 3: Songlines

3
Figure 4: For All Mankind 4
Figure 5: Testing paper prototype 5
Figure 6: Screenshots from first digital prototype 6
Figure 7: Experience curve graph 7
Figure 8: Physical setup and Orbit sequencer diagram 8
Figure 9: Final thesis version screenshots 9

iv
Abstract
Sometimes I think creating a game is continuous process of realizing how much you
didn’t know about the game you are making. This was particularly true with this project,
SPACE MAESTRO. It was constant conflict between contradicting ideas, and efforts to
merge these made it go through multiple prototyping stages and change over time.
This paper is a documentary of the process, as well as an assessment of aesthetic-driven
design approach.

1
Introduction
SPACE MAESTRO is an interactive Kinect experience that player creates her own
musical universe through free-form movement in physical space. By spatially moving
both hands like a maestro conducting an orchestra, player fuses gas clouds in space
into stars. During the process of the creation, as new stars born, collide with each other,
and die, it triggers musical stingers that make up a unique soundscape as well as visual.

The experience aims to position between interactive installations in gallery space and
performance art in public space. One session of the experience is meant to be played
through by one person, and it takes about 10 to 15 minutes to finish. In order to make it
accessible by potential players, the experience consists of four acts that gradually
introduce new interaction mechanics and audiovisual component.

The progression of interaction, music, and visual shapes the overarching emotional arc.
In the first three acts, player builds up the sense of control and scale of influence over
her creation, a galaxy. As camera keeps zooming out at the fourth act, the amount of
player’s influence to the galaxy slowly decreases over time until what is left on the
screen is the player’s galaxy on black emptiness of space. Then, on the border of the
screen, other galaxies appear, which other players’ have created.

The goal of the experience is to bring about the sense of power and wonder that
eventually induces the state of self-reflection through the overarching narrative.
2
Aesthetic-Driven Design
As stated in MDA framework, a formal approach to game design and game research,
mechanics lie at the core of any games. Designing a game usually starts from designing
the mechanic, which then, gives rises to dynamics, and then appreciated to players as
aesthetics.

Figure 1: MDA Framework
However, although I agree with that mechanic is the core of games, I think the process
of designing a game varies: It does not always start from the mechanic. In my case, I
tend to start from the other end, the aesthetic or the dramatic metaphor that usually
wraps up the system after the mechanic is established: A design approach that I call
aesthetic-driven design.

For instance, stained glasses in a cathedral have strong visual aesthetic that mesmerizes
its viewer. However, the visual aesthetic, such as patterns of color and shape, also
suggests some sense of system behind it. Then, we can ask questions like, what kind of
dynamics would result in this aesthetic, and what is at the core rules that give rises to
that dynamics.

Mechanics Dynamics Aesthetics
Designer Player
Aesthetics Mechanics Dynamics
Designer Player
3

Figure 2: Palette (2006)

The inspiration and design of one of my previous works Palette, a color mixing puzzle
game, followed this thought process. It was directly inspired the visual aesthetic of
stained glass, and the mechanic that fits with the aesthetic came after it.

With this approach, instead of starting from the core mechanic, I tried to come up with
a mechanic that fits with the desired aesthetic. This may not be an ideal approach for
designing games, since such games inherently highly lean on the mechanic. However,
when it worked, it could reverse the relationship, so that the aesthetic elements justify
the mechanic.

4
SPACE MAESTRO, in that sense, was an extreme case of aesthetic-driven design
approach. In this case, however, I was driven by multiple aesthetics simultaneously,
visual, music and the title itself. Images of deep space galaxies took by Hubble
telescope were not only beautiful, but also prompted some sense of musical
composition to me, and apparently many musicians. Listening to Nine Billion Names, an
instrumental music by a Russian postrock band Mooncake, I kept on imaging the
desired emotional state that I want this interactive experience called SPACE MAESTRO
would induce. This 6-minutes-long music, which has quite repetitive and simple structure,
invoked a mixture of emotional states to me: there was strong sense of immense scale
and being humbled by it, but also power and pride. I knew that that kind of emotional
state was what I wanted to induce through my thesis, but it was hard to pinpoint and
describe what that really is then.

The word maestro in the title suggested that music will be an integral part of the
experience, and player would have control over it, thus feel powerful. While searching
for the reference I could find the close relationship between space and sound in
conducting. “Basically the hands are there to describe a certain space of the sound
and to shape that imaginary material”, the music director of Philadelphia Orchestra,
Yannick Né zet-Sé guin, said during his interview with New York Times. Although he was
describing how the real conducting is like, I could see possibilities of experimenting with
the physical space and sound in interactive form.

5
Prior Art
Songlines

Figure 3: Songlines (2012)

A graduate student in IMD class of 2012, Samantha Vick’s thesis project Songlines had
many similarities with SPACE MAESTRO both mechanic-wise and subject-wise. Based on
an Australian creation myth, player becomes a fairy of music that shapes the world with
her musical power. Using Kinect, player moves both hands like a conductor to create
various terrains. The project brought about the sense of power well. However, although I
also wanted players to feel powerful, it was not the end goal for SPACE MAESTRO, but
part of the experience that leads to some mental state.

6
Astrophysics

Figure 4: For All Mankind (1989)
In For All Mankind, a documentary about Apollo mission to the moon, there’s this scene
where the astronauts look back at the earth as they travel to the moon. Suddenly, it
starts to question “Do you really know where you are at this point in time and space
and in reality and in existence?”

From his stunning monologue on the most astounding fact in the universe, Neil Degrasse
Tyson says it is the molecules cooked inside star and explode into supernova that
became the ingredients for life, thus the universe is in us.

There’s a sense of great connectivity between me and the universe in his quote. Then, I
realized it is this state of self-reflection that I eventually want from this project.

7
Research and Development

These prototypes are listed in chronological order. Prototypes that ended up not having
meaningful influence to the final experience were excluded from the list.
Paper Prototype

Figure 5: Testing paper prototype

In this paper prototype, we tested how music can be tied with experience of creating a
galaxy. While music is playing, player spreads a colored glitter of his/her choice on a
spinning board. The rotation and the centrifugal force of the rotation make an abstract
shape that resembles a galaxy.

This prototype told us about the experience more than what we originally expected to
get. First, we could find that the dramatic arc of the music doesn’t match with the one
of a player. When the dramatic arc of the music picked up before a player did, player
felt falling behind and became “overpowered” by the music. Since we wanted the
player to be the only powerful being in the experience, it was a big issue. Thus, instead
8
of a pre-structured music, we decided to make the music modular, so that it reacts to
the player’s action.

Also, taking morphological analytic approach, we could map out the list of variables
that we would tweak to get to the experience goal. Considering each variable as a
slider between two opposite ends, we had the list as below. This helped us a lot to
predict and plan out how the overall experience curve should be shaped.

Magnitude of Gesture : Small vs Large
Color: What color should we use in?
Control : Tight vs Loose
Player : Solo vs Group
Plane : 2D vs 3D
Mood : Melancholy vs Happy
Visual Experience : Obvious vs Reveals Itself Later
Music : Linear? Loop? Layered? Recombinant?

9
Digital Prototype - Iteration 1

Figure 6: Screenshots from first digital prototype
In this build, I put together basic components and interactions. To let players feel like
they are really putting their creative input, I needed some system that directly reacts to
player's action and actually shape the galaxy. I put together vector field with 10000 gas
cloud particles, so that player can color and shape the galaxy by interacting with the
vector field. In the 2D vector field prototype with mouse and keyboard, it clearly
resulted in galaxy-like shapes of particles even with minimal or chaotic input. However,
translating the input with Kinect was another challenge that was not answered yet.

The build had two interactions, charging and shooting energy photons to create stars,
and exploding all existing stars by holding both hands up and down. Seeds of stars were
formed randomly on top of gas clouds, and player had to shoot toward the spot by
holding both hands together and stretching one hand toward desired direction, same
gesture with the fairies prototype. Once they create a star, they could explode it, and
the number of stars explode progressed the experience to the next level.
10

However, it became clear that player did not understand any of the gestures intuitively.
Since a gesture is specifically designed set of actions, it had to be told specifically for
player to mimic them. Not only this was not intuitive, but also it conflicted with the sense
of control of player by telling them what to do that's predetermined by the designer.
Although some instruction would be inevitable, I wanted to convey that as subtle as
possible, letting the player figure out the rest.

Although most people picked moving around the physical space as their favorite part
of the experience, most of them didn’t try to do so at first. I assumed that people would
not expect it in Kinect experience because most commercially available Kinect
experiences focus on gestures and refrain from using large spaces, which would mean
reducing their target audiences. Therefore, we needed some prompt or reason for
players to move around.

Along with stereoscopic 3D, which greatly enhanced the perception of 3D space,
people generally liked the experience. However, most of the positive response was due
to the visual spectacle and was still lag of structure and meaning.

11
Digital Prototype - Iteration 2

Figure 7: Experience curve graph
We extrapolated experience into 4 sectors. In each sector, player created a visual
structure using one chunk of gas clouds at a time. In between every sector, camera
zoomed out incrementally, and each visual structure player created stacked on top of
the previous one, making a galaxy-like disc shape of particles. In each sector, the visual,
musical, and interactive components were changed and evolved, making an
overarching experience curve.

Taking the feedback from the last test, I decided to refrain from using too many
gestures and removed the charging and shooting gesture. Instead, I looked for
something pick-up and playable and made the continuous movement of hands in
space as the core action. Player could move both hands freely in space to touch and
drag gas clouds and hold hands close to create a star. The result was quite satisfying:
testers figured out the control much easier.
12

However, we still could not remove all the gestures. The star explosion gesture, holding
hands up and down, still lacked of instruction, and player could not figure it out or
accidently triggered it and got confused. Although I have been reluctant to use texts
for the sake of aesthetic subtlety, it became clear that there is no way player would get
what to do naturally. I had to tell what to do specifically and clearly.

Interactive music system was implemented at this stage, and it worked quite well. We
had a bank of musical stingers, and each stinger was quantized to the base loop track.
Although players enjoyed how it sounds for the first few minutes, they soon figured out
that the system is quite static: whatever they do, it sounds similar. Also, we still have not
dig deep enough to the possibility of using the whole body in physical space to
generate and manipulate sound. We set our next goal to use the spatial position and
whole body data to make the aural experience more diverse and dynamic.

As an interactive experience, it had clear discernibility: how player moved hands in
space deformed the gas clouds, giving direct feedback. However, it lacked of integrity:
it was not clear how the player’s input affect the outcome in the long run. Showing how
player’s input shape the visual and audio of a galaxy became our next goal.

13
Results

To show how player’s input affects the buildup of a musical experience, we applied the
analogy of musical sequencer. A galaxy consists of multiple concentric circles, orbits.
When a player creates a star, it snaps into the nearby orbit and revolve around the
center of the galaxy. Each time a star revolves around the galaxy, it emits musical tones
of its own. As a player creates multiple stars in each act, it builds up a unique musical
composition that is driven by the player’s input.

Setup Requirements

Figure 8: Physical setup and Orbit sequencer diagram
-Microsoft Kinect Zigfu OpenNI
-2.5m x 3.5m physical space
-Dark quite personal environment
-5.1 channels surround speaker system or wireless headphone
-Stereoscopic 3D TV

14
How to Play

Figure 9: Final thesis version screenshots
Holding two hands together creates a star’s core. Gently moving both hands around
the core transmits energy to it, transforming its color. Making a fast motion ignites the
core and creates a musical star.

When a star is created, it follows nearby orbit. Touching a musical star makes musical
sound – a galaxy sings to player’s movement, making a unique visual and music.

Moving around the physical space moves the viewpoint.

There is no goal, but to just enjoy creating stars and building your own galaxy.
15
Conclusions

Future Development & Areas to Research
There are many potential improvements that we could not implement due to technical
restrictions and lack of resources and time.

Designing Finger Gestures
Detecting and applying finger gesture to the creation of visual and sound using next
generation of Kinect will greatly increase the interactivity and solve many design
problems we had.
3D Vector Field Integration
Currently, we use 2D vector field to make gas clouds dynamically react on player’s
motion. Integrating 3D vector field that models gravity will not only increase the joy of
freeform movement in space, but also will make it possible for players to deform the
shape of the galaxy even more dynamically by their creative input.
Designing & Implementing Social Feature – Saving Your Galaxy & Sharing with Others
Epilogue-like part of the experience where player can navigate the whole universe
filled with galaxies other people have created. Same as the previous acts, player will be
able to move around the virtual space mapped to the physical space. This time,
however, player moves intergalactic space faster than the speed of light, which then
changes the look of other galaxies as player move around. I have prototyped this
subjective perception of time in space at the Spime prototype.
16

Epilogue
Personally, it has been my dream project since I got the first inspiration about 3 years
ago. Over time, I have struggled to figure out what the project is really about and how
to convey that authentically. Over the history of mankind, the human-centered view of
the universe has been constantly degenerated through new discoveries. We are not at
the center of the universe, nor began with the universe itself. All we know seems like a
speck of dust suspended in the pale blue dot in the vast emptiness of the universe.
However, as an astrophysicist Neil Degrasse Tyson said, the most astounding fact about
the universe is that the origin of the stuffs that made us are traceable to the molecules
that made through the cycle of born and death of stars, thus we are made of stardust:
we are all CONNECTED to the universe.

I find that statement a giant step to regain our self-esteem, and I wanted to convey
that through interactive media form in a way that has been never told before. SPACE
MAESTRO, in that sense, is a paradoxical title. It begins as a SPACE MAESTRO, but later it
reveals that you are not. However, it suggests another way of accepting that, through
showing the connection between all of us, which each has a universe in it. This is the
level of reflection that I want to induce from the players through this project.

17
Works Cited

Hunicke, R, M LeBlanc, and R Zubek. “MDA: a Formal Approach to Game Design and
Game Research.” Northwestern University. Evanston, IL, 2004. Lecture Slides.

Palette. HandMade Game, 2006. Game

Songlines. USC Interactive Media Division, 2012. Game

Nine Billion Names. Mooncake, 2008. Music

Daniel J. Wakin. “The Maestro’s Mojo” New York Times, 2012. Print.

For All Mankind. Al Reinert, 1989. Video

Neil Degrasse Tyson. “10 Questions for Neil Degrasse Tyson” Time Magazine, 2008, Video

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Songlines: combining music and gesture to create a mythic experience

PDF

Historicity and sociality in game design: adventures in ludic archaeology

PDF

The moonlighters: a narrative listening approach to videogame storytelling

PDF

SomeDay: designing a game about different thought processes

PDF

Paralect: an example of transition focused design

PDF

Manjusaka

PDF

544

PDF

Revisions: an exploration of metafiction and metaphors in game design

PDF

GSML: Game System Modeling Language—spatializing play structures and interaction flow

PDF

Outer Wilds: a game of curiosity-driven space exploration

PDF

Light at the End of the Tunnels: level design and its relationship to a spectrum of fear

PDF

duOS

PDF

Bottles

PDF

Intimation and experience of the self in games

PDF

The Toymaker’s Bequest: a defense of narrative‐centric game design

PDF

The Star Wanderer

PDF

Ascension: an analysis of game design for speech recognition system usage and spatialized audio for virtual reality

PDF

Kinesthesia: a multi-sensory gesture driven playground of the future

PDF

The ritual model: how to use the mechanics of ritual to create meaningful games

PDF

Psynchrony: finding the way forward

Asset Metadata

Creator Kim, Jonghwa (author)

Core Title Aesthetic driven design of Space Maestro

School School of Cinematic Arts

Degree Master of Fine Arts

Degree Program Interactive Media

Publication Date 07/23/2013

Defense Date 03/08/2013

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag game design,interactive art,kinect,OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Gibson, Jeremy (committee chair), Chen, Jenova (committee member), Diamante, Vincent (committee member), Hobberman, Perry (committee member), Wixon, Dennis (committee member)

Creator Email ambitiousk@gmail.com,jonghwa@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-296261

Unique identifier UC11292694

Identifier etd-KimJonghwa-1819.pdf (filename),usctheses-c3-296261 (legacy record id)

Legacy Identifier etd-KimJonghwa-1819-1.pdf

Dmrecord 296261

Document Type Thesis

Format application/pdf (imt)

Rights Kim, Jonghwa

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA