University of Southern California Dissertations and Theses

Songlines: combining music and gesture to create a mythic experience

(USC Thesis Other)

Songlines: combining music and gesture to create a mythic experience

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

SONGLINES:
COMBINING MUSIC AND GESTURE TO CREATE A MYTHIC EXPERIENCE

by

Samantha Vick

A Thesis Presented to the
FACULTY OF THE USC SCHOOL OF CINEMATIC ARTS
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF FINE ARTS
(INTERACTIVE MEDIA)

May 2012

Copyright 2012 Samantha Vick

ii

“In all chaos there is a cosmos, in all disorder a secret order.”
– Carl Jung

iii

Acknowledgments

There are many people without whom the creation of Songlines
would have been impossible. These people deserve my deepest thanks
and the accolades of legions innumerable, but might have to settle for
the former.
First and foremost, I want to thank my wonderful development
team, who took my vision and made into something real and beautiful.
Engineers Alejandro Villagomez, Sahil Ramani, Rambod Kermanizadeh,
and Jonghwa Kim; composer Gareth Coker; artist Alice Li; animators
Carolyn Chrisman and Ryan Chen; designer Michael Sennott; sound
designer Baihui Yang, filmmakers Freddy Gaitan, Moriah Burton, and
Alejandro Martinez; dancer and maker Tiger Brooke; and voice actors
Holly Stevenson and Michael Johnston. Without you, Songlines would
literally not exist.
Thanks to my thesis chair Jeremy Gibson, for his patience and sage-
like wisdom, and to my thesis committee members: Kathy Smith for her
sharp eye for animation and insights into Australian culture; and Vincent
Diamante, for his experience with dynamic music and encyclopedic
knowledge of game history.

iv

Thanks to professors Mark Bolas and Laird Malamed, who really
helped me learn who I was through what I was trying to create. In fact,
thanks to all the professors of the Interactive Media Division, for the world’s
best education in the field I love.
Thanks to my cohort, who always provided someone who was way
better than me at any given task, yet completely humble and kind, for
role model purposes.
Thanks to the Annenberg Graduate Fellowship Program, for
awarding me one of their amazing fellowships and supporting me for my
first two years at this school.
Thanks to my parents for their unwavering support for all of my
strange life choices, and for their sharp edits of this paper. Thanks to all of
the amazing classmates and friends I’ve made over the course of my
three years here at USC. And the biggest thanks of all to Mike Sennott.
Without his love and support, I’m sure I would have gone crazy months
ago.

v

Table of Contents
Epigraph ii
Acknowledgments iii
List of Figures vi
Abstract viii
Introduction 1
Game Premise 1
Core Gameplay 3
Exploration of the Mythological Setting 7
Mythology and the Mind 8
Mythology and Games 10
Mythology and Songlines 12
Themes of Songlines 14
Joy 14
Prior Art 15
Joy in Songlines 19
Music 20
Prior Art 22
Music in Songlines 25
Gesture 26
Prior Art 29
Gesture in Songlines 32
Power 33
Prior Art 35
Power in Songlines 38
Challenges 40
Conclusion 43
Bibliography 46

vi

List of Figures

Figure 1: The Songlines poster............................................................................ 2
Figure 2: Kinect gesture guide........................................................................... 4
Figure 3: Terrain and instrument pairings.......................................................... 5
Figure 4: Kratos overpowers a chimeric enemy. ............................................. 7
Figure 5: Campbell's seminal work.................................................................... 8
Figure 6: Campbell's theories distilled............................................................... 9
Figure 7: Flower (Sony, 2009)............................................................................ 15
Figure 8: Katamari Damacy (Namco, 2008).................................................. 17
Figure 9: LocoRoco (Sony, 2006) ..................................................................... 18
Figure 10: A successful combination is found in Bejeweled......................... 20
Figure 11: Star Wars: X-Wing (LucasArts, 1993)............................................... 22
Figure 12: Patapon (Sony, 2007)...................................................................... 23
Figure 13: Vib-Ribbon (Sony, 2009).................................................................. 24
Figure 14: The Whirling Dervishes practice their worship. ............................. 27
Figure 15: Pope Benedict administers the Eucharist. .................................... 27
Figure 16: Mother Nature (Diane Tucker, 2011) ............................................. 29
Figure 17: Child of Eden (Ubisoft, 2011) .......................................................... 30
Figure 18: The Gunstringer (Microsoft, 2011) .................................................. 31

vii

Figure 19: From Dust (Ubisoft, 2011)................................................................. 35
Figure 20: Spore (EA, 2008)............................................................................... 36
Figure 21: The Sims (EA, 2000) .......................................................................... 37
Figure 22: Songlines' previous controller-based interface............................ 40
Figure 23: An example of Aboriginal art......................................................... 41
Figure 24: A still from Songlines' opening animation. .................................... 42
Figure 25: The world’s oldest musical instrument........................................... 44

viii

Abstract

Songlines is a playable musical creation myth in which the player
shapes the world with gesture via the Kinect platform. The experience
synthesizes a simple, universal allegory with the emotional power of
gesture and music. This paper uses the theories of psychologist Carl Jung
and comparative mythologist Joseph Campbell to establish a case for the
importance of subconscious archetypes in explaining the emotional
power of myth, music, and gesture in games. It summarizes the core
themes behind the interactive experience, recounts other advances in
those areas, and describes the innovative systems behind this joyful
gameplay experience.

1

Introduction

Game Premise

Songlines is a gesture-based musical experience, built in Unity and
featuring motion control via Microsoft’s Kinect platform. The core concept
is based on an Aboriginal Australian creation myth in which totemic spirit
beings created the world through song and dance. The player takes on
the avatar of a primordial creator spirit who flies through a blank and
formless landscape, using gestural control via the Kinect and a unique
recombinant music system to directly shape and populate the world
around her. Songlines provides a sensory experience that is at once
relaxing, energizing, and transcendent, as the player watches life burst
into being all around her.

2

Figure
1:
The
Songlines
poster.

Songlines is meant to be played as an interactive experience,
rather than as a true game. There is no ‘lose’ state; the player is free to
pick and choose any terrain types and to sculpt any kind of landscape
she desires without penalty. Every aspect of the experience is meant to
impart a sense of power and joy to the player. Using the Kinect to harness
the emotional power of gesture, Songlines allows the player to create a
personal landscape from nothingness.

3

Core Gameplay

The core mechanic of Songlines is the ability to deform and
retexture terrain through gesture. The player can have up to two terrain
types active at a time, with one mapped to each hand. Most of the time,
this takes the form of raising and lowering terrain by moving both hands at
roughly chest height. Because Songlines takes place in a 3d space, the
player must be able to move and create terrain using all three axes: the X
axis spans the width of the screen, the Y axis spans the height, and the Z
axis spans the depth of the environment. Movement in XZ space will select
what patch of terrain one wishes to transform, and movement in XY space
will then raise or lower the chosen piece of terrain into the desired type.
Additionally, the player can deform terrain along a vector with a flick of
the desired hand, raising a whole range of mountains or a whole swath of
desert at once. A charge-up modifier, created when the player places
her hand at waist-level, can increase the size and intensity of the terrain
created through the two deformational gestures.

4

Figure
2:
Kinect
gesture
guide

XYZ
movement

XY
movement

Vector
deformation
start

Vector
deformation
end

Charge-up
modifier

Songlines features three full terrain types and two terrain modifiers
from which the player can choose to populate her world: mountain,
desert, grassland, forest, and water. The first three are true terrain types;
they deform the entire selected area and replace each other completely
if the player chooses to overwrite terrain that has already been modified.
The other two are terrain features that modify the first three. Forest, for
example, consists of broad swaths of trees that change species based on
the terrain upon which they’re placed. Thus, the same action that results

5

in a patch of acacia trees when placed in grassland will result instead in
cacti when dropped into a desert biome. Similarly, the water feature will
react differently depending on environmental context.

Figure
3:
Terrain
and
instrument
pairings

Desert
/short
strings

Forest/plucked
strings

Grasslands/digeridoo

Mountain/drums

Water/woodwinds

Each terrain type is linked to a suite of instrumental loops, which are
strung together via a custom recombinant music system, based on a
hierarchy of rules provided by the composer. Each instrument will increase
or decrease in complexity depending on the degree of deformation.
Because the player can choose to have either one or two terrain types
active, there can be either one or two instrumental suites featured at any

6

given time. The end result is that each terrain, or combination of terrains,
has a distinct sound which ebbs and flows throughout the course of the
experience.

7

Exploration of the mythological setting

At first glance, an Aboriginal Australian creation myth might sound
like an odd choice for the basis of an interactive experience. Most
videogames that have roots in mythology choose to explore a much
more classical and
action-heavy premise; not
only are those stories easy
to write and familiar to
most audiences, but they
lend themselves naturally
to a traditional, combat-
based method of play.
Sony’s God of War franchise (Sony, 2005 – 2011) is a good example: many
people are familiar with the Greek pantheon, and the gameplay is
satisfying hack-and-slash: gaming comfort food. From a psychological
standpoint, however, a creation myth is the perfect vehicle for an
experience such as Songlines: a rich and vibrant setting, steeped in
allegory and based on a primal, universally shared experience.

Figure
4:
Kratos
overpowers
a
chimeric
enemy.

8

Mythology and the Mind

Joseph Campbell, world-famous comparative mythologist, wrote
The Hero with a Thousand Faces in 1949,
a book that revolutionized the way we
think about mythology. He drew upon
psychologist Carl Jung’s theory of the
collective unconscious, which posited
that “the forms or images of a collective
nature which occur practically all over
the earth (are) constituents of myths and
at the same time…autochthonous,
individual products of unconscious
origin.”
1
In other words, the repeated themes and symbols present in the
dreams of people who otherwise had nothing in common were actually
universal archetypes, the result of a primal and numinous psychological
heritage that all humans possess by the very fact of their shared humanity.
Campbell took this theory a step further, claiming that all of mythology —
its very concept — bubbles up from the same fundamental, magical
place: our shared experiences as human beings. He writes, “Religions,

1
Jung, C.G. Psychology and Religion (Collected Works, vol. 11; New York
and London, 1958), par. 88.
Figure 5: Campbell's seminal work.

9

philosophies, arts, the social forms of primitive and historic man, prime
discoveries in science and technology, the very dreams that blister sleep,
boil up from the basic, magic ring of myth.”
2
He claimed that the myths of
man are simply expressions of the internal energies that struggle for
dominance within all of us. All myths are, in fact, one myth: the
Monomyth, also referred to as the Hero’s Journey. Although the
exploration of the Hero’s Journey comprises
only one small part of Campbell’s seminal
work, The Hero with a Thousand Faces, it has
achieved recognition and resonance
beyond its source text. Campbell’s theories
have been co-opted by the storytelling
community as a creative technique, a tool
to tell impactful, lasting stories through
fiction, film, and games. Christopher Vogler’s
bestselling educational text The Writer’s Journey is taught in screenwriting
classes the world over, as a method of teaching classic story structure and
character archetypes.
3

2
Campbell, Joseph. The Hero with a Thousand Faces (New York: MJF
Books, 1949), p. 3)
3
Vogler, Christopher. The Writer’s Journey: Mythic Structure for Writers
(California: Michael Wiese Productions, 1998)
Figure 6: Campbell's theories
distilled.

10

Mythology and Games
Mythology is a very popular theme in interactive media, and has
been since its conception. From humble beginnings in the forms of games
like Populous (Bullfrog, 1989) and Altered Beast (Sega, 1988) to the
modern retellings of the antics of the Greek pantheon Nintendo’s new Kid
Icarus (Nintendo, 2012), the landscape of mythology has long held a
fascination for game designers. Most of these games, however, focus on
only one aspect of any culture’s mythos: the hero’s journey, Campbell’s
Monomyth. Almost every adventure story, interactive or otherwise, follows
this myth to some degree. Less attention is given to a myth that is more
interesting, more far-reaching, and more revealing of a culture’s view of
itself: the creation myth.
There have been countless versions of the creation myth, carried
through the generations and spread across the globe. Perhaps at one
time they were all the same myth, endlessly embellished and distorted.
What’s certain, though, is that they share a universal purpose: to declare
the truths of the world, to show mankind their place in it, and to attempt
to explain the unexplainable. A creation myth is a commentary on the
nature of mythology itself. Only in their details do they differ, and these
eccentricities reveal interesting facts about the cultures that spawned
them. The myths of cultures that grew along the sea often feature water;

11

matriarchal societies often feature a female creator; tribes that live in
heavy jungle cover consider the world to be much smaller than tribes that
live in the high desert. Each culture’s myth is a reflection of the earthly
circumstances and beliefs of that culture.
There is a reason why game designers choose to draw from these
ancient stories. No matter from which area of the world one draws
inspiration, disguised under the various figures of mythology lie constant,
universal truths. Over and over, the same patterns emerge, the same
heroes arise, and the same monsters are vanquished. The myths of man
are part of our cultural heritage, and eloquently document our ancestors’
attempts to explain and categorize the world around them. “For the
symbols of mythology,” writes Campbell, “are not manufactured; they
cannot be ordered, invented, or permanently suppressed. They are
spontaneous productions of the psyche, and each bears within it,
undamaged, the germ power of its source.”
4
By drawing from these
archetypical stories and exposing them to a new audience, the modern-
day game designer is continuing the tradition of the shaman, the epic
poet, and the bard. Within this dynamic new medium there is amazing

4
Campbell, Joseph. The Hero with a Thousand Faces (New York: MJF
Books, 1949), p. 4)

12

opportunity to share these stories with an entirely new audience, and an
ability to impart their ancient lessons in a powerful, emotional new way.

Mythology and Songlines
The Aboriginal Australian creation myth of the Dreaming, upon
which Songlines is based, is fascinating in the way that it draws a parallel
between music and nature. There are many different versions of this myth,
just as there are many individual tribes of Aboriginal people that populate
Australia. One of the most popular, however, can be summarized like this:

In the era of the Dreaming, in the time before time, all
possibilities existed simultaneously, but none were realized.
The land was flat and amorphous. Into this silent world came
the creator beings, who used the power of music and dance
to breathe life and form into it. They travelled the lands along
sacred paths called songlines, and everywhere their song
touched, the earth sprang up in joy. After they were finished,
they were absorbed into the rocks and cliff faces of this fresh
new world, leaving behind paintings of themselves as a
reminder that they are always watching over the land that
they created.
5

Perhaps no other myth is so explicit in its connection between
music, movement, the natural world, and the sublime. It equates music
and dance directly with the sacred act of creation. The myth affirms that

5
Songlines, 2012

13

there is some unknowable element present in music and dance so
powerful that it cannot be explained rationally. Following Campbell’s
theories that all mythological expressions of external action are analogous
to the interplay of internal psychological forces, the existence of this myth
asserts that music and dance are fundamental to early man’s
understanding of the universe. This myth, due to both its nature and its
content, also serves as a compelling argument for the universality of the
link between music, dance, and a sense of the divine within the human
psyche. When viewed in this fashion, there seem to be few better
subjects for an interactive experience that seeks to engage a wide
audience with the simple themes of music, gesture, power, and joy.

14

Themes of Songlines

The following section is an examination of the four main themes of
Songlines, relating them back to the game-myth structure. It also takes a
look at prior art in each of these fields. Many of the games on these lists
could fit comfortably into multiple categories, but for the sake of brevity
they are each listed only once. At the end of each themed section, this
paper recounts the steps that the Songlines team took to instill each
theme into the experience as much as possible.

Joy
The concept of joy is a bit trickier to relate back to the realm of
myth than others on this list. Few myths are concerned with inspiring or
relating a feeling of joy; perhaps because the human condition is
understood to be flawed, or perhaps because advocating happiness
doesn’t relate socially useful lessons nearly as well as other emotions, like
fear or pride, to which many myths more commonly appeal. Whatever
the reason, joyfulness is a fairly rare theme in mythology – which makes
the Aboriginal Australian myth upon which Songlines is based all the more
special.

15

This myth is steeped in joy. Everything about it, from the method and
means of creation to the imagery of the land’s response, speaks of
playfulness and wonder. It doesn’t attempt to define a caste system, like
the Hindu creation story, nor establish a gender hierarchy, like the
Cherokee creation story, nor introduce the concept of original sin, like the
Abrahamic creation story. It simply seeks to explain how the world came
to be in terms that are powerful in their simplicity and universality.

Prior Art
Flower

Figure 7: Flower (Sony, 2009)

16

The second game released by USC’s own thatgamecompany,
Flower (Sony, 2009) gives the player the powers of the wind. By tilting the
gyroscope in the Playstation 3 Sixaxis controller to gather up flower petals,
the player can effect positive changes upon the environment. According
to Jenova Chen, the game’s designer, Flower was created explicitly to
evoke feelings of power, freedom and joy.
6
The game also features a
critically acclaimed dynamic music system created by Vincent Diamante.
Rightfully, Flower has a place on all four lists in this section, but since all of
its elements work together in harmony toward the singular goal of
producing positive emotion, it belongs above all in the section discussing
joy.

6
Chen, Jenova. Interview by Matthew Reynolds. Digital Spy. Digital Spy,
2009.
http://www.digitalspy.com/gaming/interviews/a182429/jenova-chen-
flow-flower.html

17

Katamari Damacy

Figure 8: Katamari Damacy (Namco, 2008)
Katamari Damacy (Namco, 2008) is an unapologetically silly and
nonsensical game. As the miniscule Prince of All Cosmos, the player’s job
is to roll up a succession of larger and larger objects into a ball, collecting
enough mass to replace the stars that had been destroyed by the
character’s father, the flippant King of All Cosmos, on a drunken bender.
The premise may seem incredibly odd, but the relentlessly upbeat music
and the weird satisfaction of being able to roll up larger-sized objects as
your katamari increases in size has made this cheerful game a cult classic,
spawning sequels across many platforms.

18

LocoRoco

Figure 9: LocoRoco (Sony, 2006)
Since happiness is such an ephemeral concept, it can be difficult to
sum up exactly what elements of a game serve to evoke a feeling of
happiness in the player. In the case of LocoRoco (Sony, 2006), this might
be attributed to its simplicity. LocoRoco is controlled using only the left
and right shoulder buttons of the PSP, which tilt the entire landscape left
and right. The game’s core conceit is that mercenary aliens have
descended on the peaceful world of the happy, jelly-like LocoRoco, and
its pacifist denizens are powerless to stop them. It’s up to the planet itself
to guide the LocoRoco by tilting itself, helping them get rid of the
invaders. LocoRoco features a bright and colorful art style, a dynamic

19

music soundtrack, and endearing characters. All of its elements come
together to form a game that is single-minded in its purpose of inspiring
happiness in the player.

Joy in Songlines
From its conception, the core goal of Songlines has been to inspire
a feeling of joy. The team was very inspired by games like Fl0wer, in which
the explicit aim was to evoke positive emotion in a way that was serene
and non-violent. Hearing Jenova Chen talk about his desire to change
the way the industry thought about emotion in games was extremely
motivating.
To that end, every stylistic choice in the creation of Songlines has
been toward evoking a feeling of joy. The music created by Gareth Coker
had its roots in the traditional music of the native Australian people, and
was specifically composed to evoke a feeling of excitement and
grandeur. All of the art was also based off the traditional dot-and-line style
of many Aboriginal tribes, but we also tried to make it colorful and vibrant.
Even though the concept of joy is hard to pin down, we’ve tried very hard
to capture it in this experience.

20

Music
Humans love finding patterns, thanks to a phenomenon known as
the priming effect, by which we interpret stimuli according to an
expected model.
7
This
fundamental trait is evolutionary in
origin, ensuring that early man was
able to spot the tiger lurking in the
undergrowth before he became its
dinner. Many games are about
pattern systems, from the simple
color matching of Bejeweled (Popcap, 2001) to the most complicated
character interactions of Bioware’s sprawling role-playing games. There is
a fascinating intersection between games, music, and mythology, which
all spring from the human brain’s proclivity to create these patterns, to
search for order within chaos. A music-based game is the perfect form for
a game about mythology, and even more so in this case because the
specific myth upon which it’s based is itself musical in nature.

7
Shermer, Michael. “Patternicity: Finding Meaningful Patterns in
Meaningless Noise”, Scientific American, 2008.
http://www.scientificamerican.com/article.cfm?id=patternicity-finding-
meaningful-patterns
Figure
10:
A
successful
combination
is
found
in

Bejeweled.

21

Even the most primitive games have some form of music. Many
people are passionate and opinionated about videogame music; game
soundtracks, concerts, and remixes are a hugely popular and diverse form
of media. Symphonies that showcase videogame music, such as Other
Worlds or Video Games Live tour all over the world and garner massive
ticket sales. Websites dedicated to creating and sharing remixes of game
soundtracks, such as ocremix.org, have a large following. Music games
are also their own pervasive genre, with one of the best-known examples
being Dance Dance Revolution (Konami, 1998). This type of game,
commonly known as a beat match game, usually finds the player
attempting to duplicate the rhythm of a song of set length that always
proceeds the same way, regardless of the player’s input. In these games,
interactivity is a one-way street; the player either conforms to the game’s
demanding rhythm, or fails.
Rather than delve deeply into either of these two traditional forms,
this section is dedicated to games in which the developers took the
concept of music a step further than mere soundtrack and instead
attempted to craft a music system that interacted with the player and
changed based on her actions.

22

Prior Art
Star Wars: X-Wing

Figure 11: Star Wars: X-Wing (LucasArts, 1993)
One of the first forays into the area of adaptive music was Star Wars:
X-Wing (LucasArts, 1993). This DOS computer game features the
revolutionary iMUSE dynamic music system, which allows the intensity of
the music to adapt to the changing intensity of the situation during
gameplay. The iMUSE system includes special themes to announce
specific gameplay events, such as the successful completion of a mission
or the arrival of enemy backup, which could be blended seamlessly into
the main score. Even though the game’s music takes the form of simple
MIDI files, it was an extraordinarily sophisticated system for its time.

23

Patapon

Figure 12: Patapon (Sony, 2007)
Another game that utilizes music in an unusual way was Patapon
(Sony, 2007). Patapon is a rhythm game for the PSP in which the music is
reactive to the player, turning her musical input into marching orders for a
small army of tribal warriors. Using the face buttons on the PSP, the player
issues commands to her minions in the form of drum beats, with different
sequences of beats signifying different orders. Although the player is
required to keep rhythm with herself, the game is completely reactive to
this rhythm: if she enters commands off-rhythm or stops entering them at
all, the music cuts out completely and the tribe mills about in confusion,
leaving them vulnerable to enemy attack.

24

Vib-Ribbon

Figure 13: Vib-Ribbon (Sony, 2009)
Vib-Ribbon (Sony, 1999) was unique in that the game loaded
entirely into the Playstation’s RAM, allowing the player to insert any CD in
her collection and use any track to generate a unique level. Tracks with a
faster beat were more challenging than more serene songs. Thanks to its
super simple line-based visuals, the entire gameplay experience was able
to be generated on-the-fly using only the music track selected by the
player. Different elements in the game translate as various obstacles that
the character must overcome by pressing buttons at the right time. The
player evolves or devolves depending on her performance. In Vib-Ribbon,

25

the game’s music defines its landscape, which marks it as an important
piece of prior art for Songlines.

Music in Songlines
Songlines features a recombinant loop-based music system created
by composer Gareth Coker and engineer Alejandro Villagomez. The goal
of the music system is to create musical landscape that shifts and adapts
in real time to reflect the physical landscape being created by the player.
The system encompasses five separate suites of musical instruments, each
bound to one of the five terrain types: drums/mountains, short
strings/desert, digeridoo/grassland, woodwind/water, and plucked
strings/forest. Each suite is composed of dozens of individual three-second
music loops, separated into simple, medium, and complex categories.
These loops are then strung together according to rules laid out by the
composer, using a custom music engine. The type and intensity of the
music is dictated by the type and intensity of the terrain being created by
the player; for example, if a player uses the charge-up modifier and then
unleashes a chain of snow-capped mountains, the drum section will shift
to become more complex. Since there can be two active terrains at any
given time, two suites of music can also be showcased. In addition, there

26

are embellishments and stings that are added to the track when the
player accomplishes an in-game goal or completes a certain number of
deformations within a given time-period. These exist to add flavor to the
soundtrack and serve as aural rewards and gameplay cues. The end
result is that the Songlines soundtrack is in a constant state of flux, and
every player will get a different result to reflect their personal landscape.

Gesture
With the recent popularity of motion-based gaming on platforms
like the Nintendo Wii, Playstation Move, and Xbox Kinect, the role of
gesture in games has become a pertinent topic of inquiry by game
designers and researchers. A previous graduate of this university’s
Interactive Media Program, Diane Tucker, delved deeply into the scientific
underpinnings of emotional gesticulation in her thesis and its
accompanying game, Mother Nature. Rather than retread the same
ground, this section will discuss instead the role of gesture in ritualistic and
mythic capacities.
The widespread use of ceremonial dance in ancient cultures as a
form of communion with the divine is well documented. The Whirling
Dervishes of 13
th
century Turkey, for example, practiced dhikr, or

27

remembrance of God, in the form of a whirling ceremonial dance in
which their spinning motion is
symbolic of a mystical journey of
spiritual ascent. In some
Aboriginal Australian traditions,
the act of performing their
sacred dances was analogous
to reaffirming and making present the reality of the Dreamtime. Many
native Australians view the dances of their particular tribe as secret, not to
be divulged to outsiders lest they lose their power.
These tribes are far from alone in continuing the use of ritualized
gesture for spiritual purposes into the modern day; consider the common
Catholic ceremony of the
Eucharist, in which both
worshipper and priest follow
a scripted set of lines and
gestures and in which a
wafer or a sip of wine act as
symbolic stand-ins for the
blood and body of Christ. Similarly, the Muslim tradition of five daily
prayers to Mecca, during which the worshipper physically prostrates
Figure 14: The Dervishes practice their worship.
Figure 15: Pope Benedict administers the Eucharist.

28

himself in the direction of the sacred city, is a clear example of symbolic
gesture as a reminder of faith in Islam, also represented in this case by a
physical symbol. The aforementioned Whirling Dervishes are still practicing,
led by the 22
nd
generation descendant of the original founder. The ability
of gesture to influence belief and emotion has been known to mankind
for nearly as long as mankind has existed, and its power is not diminished
by our modern analysis of it.

29

Prior Art
Mother Nature

Figure 16: Mother Nature (Diane Tucker, 2011)

Mother Nature (Diane Tucker, 2011) was created specifically to
explore the intersection between gesture and emotion. The
corresponding thesis paper is steeped in scientific research and studies
about the provable psychological connections between the two. The
game itself, a garden simulation utilizing gesture via the Kinect, certainly
features many different movements and gestures in the player’s quest to
raise and protect a beautiful garden.

30

Child of Eden

Figure 17: Child of Eden (Ubisoft, 2011)
Child of Eden (Ubisoft, 2011) is a prequel to 2001’s Rez. Like its
predecessor, Child of Eden is a rail-shooter with adaptive music elements,
presented by its creator as an experiment in synesthesia. Unlike Rez,
however, Child of Eden is designed for motion control via the Kinect.
Players switch between a lock-on missile weapon and a cannon that fires
a continuous stream of bullets. It’s an extremely simple setup that feels
fluid and natural, and the simplicity is greatly tested in later levels as the
difficulty ramps up. To date, it is considered by many to be the finest and
most creative Kinect games on the market.
8

8
Steimer, Kristene. “Child of Eden Review.” Rev. of Child of Eden. IGN,
2011.
http://xbox360.ign.com/articles/117/1176177p1.html

31

The Gunstringer

Figure 18: The Gunstringer (Microsoft, 2011)
Released in 2011, The Gunstringer (Microsoft, 2011) puts the player in
the role of literal puppet-master. The entire game is framed as a puppet
play, with all the events taking place on a stage and a curtain drop
between each act. The player controls the invisible strings of the
protagonist with one hand and aims and shoots his gun with the other,
adding up to a remarkably satisfying experience. The gestures feel

32

natural, not forced, as many early examples of motion control gaming
tend to be.
9
A clever, self-aware plot adds to the game’s merits.

Gesture in Songlines
Songlines started life as a traditional controller-based game. The
gameplay focus during this early prototype was entirely on the music
aspect of the game, with the terrain creation occurring as a result of the
player’s actions in the musical sphere. However, playtests indicated that
players were more interested in controlling the deformormation of the
landscape directly, rather than as a byproduct of an otherwise-unrelated
rhythm game. They wanted to be able to ‘conduct’ the landscape as if
they were conducting an orchestra. To that end, the team decided to
switch control schemes to take advantage of new Kinect SDKs that were
being released for use with Unity. We ended up using the OpenNI
framework because it allowed for development on both Windows and
Mac operating systems.
Our first pass at a Kinect control setup was very simple. We simply
took the data for the distance between the wrist and elbow and between
the elbow and shoulder joints, and added them up. The more movement

9
DeVries, Jack. “The Gunstringer Review.” Rev. of The Gunstringer. IGN,
2011.
http://xbox360.ign.com/articles/119/1194381p1.html

33

there was, the more intense the created terrain would be. The player
switched between instruments by pressing her hands together. In this
prototype, movement was handled entirely by the rail system that we
created. The system dynamically generated an overlapping path across
a section of terrain, and changed height smoothly and dynamically as
terrain rose or fell beneath the player. We were sad to put this system
aside to make room instead for Kinect-controlled flight, but one of our
most consistent points of feedback was that the player wanted to be able
to control her own direction. Currently, we’re in the process of taking
some of what we learned in making this system and applying it to a ‘mini
rail’ system that will serve the same height-change purpose as the last
system while still allowing players autonomy over their flight.

Power
Power fantasy is a common theme in games. Many games allow
the player to run faster, jump higher, or otherwise accomplish much more
than most humans would be able to do in real life. Many popular games
give magical or supernatural abilities to the player, further satisfying their
desire to feel powerful. The god-game genre takes the concept of power
in games particularly seriously. These games put the player in the position

34

of an omnipotent, omniscient god-like being, ruling her world exactly how
she sees fit.
The concept of power is also found, in one form or another, in
nearly every myth ever told. Elemental forces that were beyond the
comprehension of early man, such as the sun or the rain, were attributed
to gods or spirits. In some animistic traditions, such as Shinto, every discrete
facet of nature – individual rocks, trees, mountains, animals, as well as
more ephemeral concepts such as death – has a corresponding spirit that
commands it. Over time, these simplistic constructs often evolved into
fully-realized deities, who then often struggled and squabbled for power
with others in their pantheons. These gods and goddesses, now far
removed from their humble origins as personifications of natural
phenomena, were assigned superhuman powers by their human creators.
By giving a human face and human foibles to these constructs,
worshippers were able to assuage their ineffable terror and create a
being with whom they could reason. The antics of these humanized gods
led to the creation of story traditions with recurring characters, themes,
and lessons.

35

Prior Art
From Dust

Figure 19: From Dust (Ubisoft, 2011)
From Dust (Ubisoft, 2011) is the single game that has most directly
influenced the development of Songlines. The brainchild of famous French
designer Eric Chahi and more than five years in the making, From Dust
puts the player in the role of a creator spirit, protector of a tribe of vaguely
Aboriginal-looking tribesmen. Using a real-time dynamic terrain
deformation with realistic physics, the player must sculpt and manipulate
different types of matter (water, sand, lava, etc.) to save your helpless
charges from the vagaries of a beautiful and dangerous landscape.

36

Everything about the game, from the core concept to the snippets of
narrative the player unlocks during the course of gameplay, is steeped in
Aboriginal mythology. Although extremely challenging at times, few
games can match From Dust in terms of that feeling of sheer
omnipotence.

Spore

Figure 20: Spore (EA, 2008)
Will Wright’s flawed masterpiece Spore (EA, 2008) was magnificently
ambitious in scope: the player was tasked with the evolution and survival
of a species, from a single-celled organism all the way up to a
sophisticated space-faring culture. Incorporating elements of several
different game genres, the final product was somewhat less than its

37

creator had promised. Instead of a single, fluid experience that morphed
and shifted as your race of creatures did, the end game was split into
several discrete stages, with the core gameplay of each being somewhat
shallow.
10
Still, no other game before or since has taken the concept of
the ‘god game’ so far.

The Sims

Figure 21: The Sims (EA, 2000)

10
Ocampo, Jason. “Spore Review.” Rev. of Spore. IGN, 2008.
http://pc.ign.com/articles/907/907454p1.html

38

Another Will Wright creation, The Sims (EA, 2000) and its subsequent
sequels are some of the best-selling and most popular games of all time.
The Sims gives the player complete control over a household of semi-
autonomous virtual people. Players can choose to help their Sims
accomplish their goals and lead happy, comfortable lives, or they can
choose to hinder their Sims and make their lives miserable. There’s a
definite element of voyeurism at play in the vast popularity of this
franchise; even though the people involved are fake, the sense of
titillation the player feels in peering at and poking into their personal lives is
very real.

Power in Songlines
Power is a very important theme in Songlines, a game which puts
the player into the role of a creator spirit. Few myths exhibit as raw an
expression of power as shown in most creation myths. From its first stages,
great emphasis has been placed on making the player feel powerful. In its
current form, the ability to raise a hand and have the landscape deform
in response is the game’s main mechanic. We wanted the player to be
able to control and create his own personal terrain in a way that felt
natural and fluid.

39

Of course, the concept of power is highly subjective. What does it
mean to have power over the landscape? We took care in the creation
of Songlines not to fall into the trap of ethnocentrism; of a purely Western
vision of power. This is especially important when dealing with the singular
way in which the Aboriginal Australians view themselves in relation to their
land. They believe that the land owns them, not the other way around;
that the people are the lens through which the land views itself. Keeping
this in mind, we wanted to be sure that the direction of influence in
Songlines wasn’t purely one-way. The player doesn’t just influence the
land; the land also influences the player. This is one of the main reasons
why we chose to implement a dynamically-changing music system. The
music is the method by which the game affects the player, for the
emotional reasons listed in the “Music” system above.

40

Challenges

Over the course of development, many difficult decisions were
made in order to maximize Songlines’ four themes. For example, the
gameplay originally
consisted of a rhythm-
game like reaction to
colored balls that flew
toward the screen,
emphasizing the player’s
reaction and rhythmic
ability. This setup was scrapped because of its intrinsic reactiveness. We
wanted the player to feel like she was in control, not subject to the timing
and whims of the system. We wanted an experience that was proactive,
rather than reactive. Ultimately, it was decided that the music aspect of
Songlines would become subject to the terrain creation, which in turn was
reactive to the player’s input. Similarly, the decision to switch from a rail-
based flight system to a player-controlled one was made with the goal of
increasing the player’s feeling of power and control.
Figure 22: Songlines' previous controller-based interface.

41

Another challenge in development was the delicacy inherent in
using Aboriginal Australian mythic tradition, music, and art. The Aboriginal
community has a long and tragic
history of being marginalized and
mistreated by the dominant
culture. In the past, this has taken
the form of stolen land rights and
the removal of Aboriginal children
from their families. As Aboriginal
culture has become more popular
worldwide, this exploitation has taken a subtler form: the co-opting of
traditional Aboriginal art for profit. Thus, many Aboriginal Australians are
very protective of their tribes’ individual artistic traditions, which they
consider sacred. The Songlines team made every effort to respect these
sensitivities, taking inspiration from these traditions without copying them
too closely.
We also wanted to make sure that the player understood the myth
she was playing. The traditions of the Dreamtime are not necessarily
familiar to many players the same way that classical mythology might be.
In order to ground a Western audience in a mythology that might seem
Figure 23: An example of Aboriginal art.

42

alien, I created an opening animation that explains the concept of the
Dreamtime and introduces the player character of the creator spirit. It
also introduces the five
spirit animals that are
keyed to the different
types of terrain. My
hope is that the
animation will prime the
player for the upcoming experience, but not dictate how they should
view it or detract from the emotions created by the experience itself.

Figure 24: A still from Songlines' opening animation.

43

Conclusion

Too often, motion-based games merely seek to replicate a real-
world activity – sports games being the most egregious example. These
games are simple to design and easy to pick-up; it’s not hard to see why
so many of them exist. However, motion gaming has the potential to be
so much greater, to inspire emotions and create experiences that can’t
be replicated just by going outside, or even on a conventional controller.
Songlines is an interactive experience that uses the Kinect for creative
purposes – that is, to create something. It uses its motion controls and its
music system to mirror the player’s desires. It’s proactive, rather than
reactive. The Kinect is a singularly powerful platform for this type of
experience, and I hope that, over time, more game creators will come to
realize this.
Songlines’ combination of music and god game mechanics is a
natural fit when viewed through the lens of comparative mythology. It is
well-established that music and the divine are closely linked in countless
mythic traditions all over the world. Music, in fact, is one of the closest
things that can be considered a global human constant. Carl Jung

44

regarded it as the gateway to the collective unconscious, and it is this
premise that underlies our work in creating Songlines.
A love of and passion for music is found in virtually every culture,
across the world and throughout time. The oldest musical instrument ever
found, a delicate bone flute carved from a vulture bone, is an astounding
thirty-five thousand years old.
Connecting the force of music on
human states of mind and
consciousness itself has been our
greatest challenge in the
development of Songlines. Equally
daunting has been our effort to
evoke the feeling of ecstatic timelessness and the oneness of the human
experience.
With this goal in mind, the Songlines team has attempted to
combine the evocative power of music with the emotional power of
gesture to create an experience that appeals to the player on a very
basic level. In the process of the game’s creation, the team made every
effort to leverage the four themes discussed in this paper – joy, music,
gesture, and power – to create an appealing experience. Through
Figure 25: The world’s oldest musical instrument.

45

playtesting, the team hopes to refine Songlines until it fully encapsulates
these emotions.

46

Bibliography

Altered Beast. Sega, 1988. Game, Arcade.

Bejeweled. PopCap, 2001. Game, Personal Computer.

Campbell, Joseph. The Hero with a Thousand Faces (New York: MJF Books,
1949), p. 3)

Campbell, Joseph. The Hero with a Thousand Faces (New York: MJF Books,
1949), p. 4)

Chen, Jenova. Interview by Matthew Reynolds. Digital Spy. Digital Spy,
2009.
http://www.digitalspy.com/gaming/interviews/a182429/jenova-chen-
flow-flower.html

Child of Eden. Dev. Q Entertainment. Ubisoft, 2011. Game, Xbox 360.

DeVries, Jack. “The Gunstringer Review.” Rev. of The Gunstringer. IGN,
2011.
http://xbox360.ign.com/articles/119/1194381p1.html

Flower. Dev. Thatgamecompany. Sony, 2009. Game, Playstation 3.

From Dust. Ubisoft, 2011. Game, Xbox 360.

God of War. Sony, 2005. Game, Playstation 2.

Jung, C.G. Psychology and Religion (Collected Works, vol. 11; New York
and London, 1958), par. 88.

Katamari Damacy. Namco, 2004. Game, Playstation 2.

Kid Icarus: Uprising. Dev. Project Sora. Nintendo, 2012. Game, Nintendo
3DS.

LocoRoco. Dev. Japan Studio. Sony, 2006. Game, Playstation Portable.

47

Mother Nature. Diane Tucker, 2011. Game, Personal Computer and
Microsoft Kinect.

Ocampo, Jason. “Spore Review.” Rev. of Spore. IGN, 2008.
http://pc.ign.com/articles/907/907454p1.html

Patapon. Dev. Pyramid. Sony, 2007. Game, Playstation Portable.

Populous. Dev. Bullfrog. Electronic Arts, 1989. Game, Personal Computer.

Shermer, Michael. “Patternicity: Finding Meaningful Patterns in
Meaningless Noise”, Scientific American, 2008.
http://www.scientificamerican.com/article.cfm?id=patternicity-finding-
meaningful-patterns

Spore. Dev. Maxis. Electronic Arts, 2008. Game, Personal Computer.

Star Wars: X-Wing. Dev. Totally Games. LucasArts, 1993. Game, Personal
Computer.

Steimer, Kristene. “Child of Eden Review.” Rev. of Child of Eden. IGN, 2011.
http://xbox360.ign.com/articles/117/1176177p1.html

The Gunstringer. Dev. Twisted Pixel Games. Microsoft Studios, 2011. Game,
Xbox 360.

The Sims. Dev. Maxis. Electronic Arts, 2000. Game, Personal Computer.

Vib-Ribbon. Dev. NanaOn-Sha. Sony, 1999. Game, Playstation.

Vogler, Christopher. The Writer’s Journey: Mythic Structure for Writers
(California: Michael Wiese Productions, 1998)

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Historicity and sociality in game design: adventures in ludic archaeology

PDF

Aesthetic driven design of Space Maestro

PDF

Intimation and experience of the self in games

PDF

The moonlighters: a narrative listening approach to videogame storytelling

PDF

Nevermind: creating an entertaining biofeedback-enhanced game experience to train users in stress management

PDF

Timension

PDF

Grayline: Creating shared narrative experience through interactive storytelling

PDF

Paralect: an example of transition focused design

PDF

Quicksilver: infinite story: procedurally generated episodic narratives for gameplay

PDF

The voice in the garden: an experiment in combining narrative and voice input for interaction design

PDF

Seymour Deeply: exploring stereoscopic 3D as a storytelling tool in interactive media

PDF

The ritual model: how to use the mechanics of ritual to create meaningful games

PDF

Exploring empathy through negotiation

PDF

Come with Me: a cooperative game focusing on player emotion

PDF

The Toymaker’s Bequest: a defense of narrative‐centric game design

PDF

A second summer

PDF

Lost together

PDF

Toward a theory of gesture design

PDF

Bottles

PDF

Psynchrony: finding the way forward

Asset Metadata

Creator Vick, Samantha (author)

Core Title Songlines: combining music and gesture to create a mythic experience

School School of Cinematic Arts

Degree Master of Fine Arts

Degree Program Interactive Media

Publication Date 05/04/2012

Defense Date 05/04/2012

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag Aboriginal,Australian,Campbell,Creation,Gesture,interactive,Joy,Jung,kinect,Music,Myth,OAI-PMH Harvest,Power,video game

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Gibson, Jeremy (committee chair), Diamante, Vincent (committee member), Smith, Kathy (committee member)

Creator Email samanthavick@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c3-30620

Unique identifier UC11290160

Identifier usctheses-c3-30620 (legacy record id)

Legacy Identifier etd-VickSamant-769.pdf

Dmrecord 30620

Document Type Thesis

Rights Vick, Samantha

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA