Animal trainers for many years now have been using what is known
as positive training, a system which uses only rewards and
never punishment for teaching behaviours to animals. The technique is
effective, and has been used to train everything from champion
showdogs to the orcas at SeaWorld. The same technique could be
used as a system of game design – but what would such positive
game design be like, and what would it be good for?
Positive Animal Training
Positive training, as practised by a great many dog trainers such
as Pamela Dennison, Pat Miller and Gwen Bailey, is based upon the
principles of operant conditioning (Skinner etc.) to develop
reward schedules that reinforce desired behaviours and eliminate
undesired behaviours. As I have discussed previously in my
exploration of why people play games, the same reward mechanism in
the brain that makes these kinds of training techniques effective
lies behind the different kinds of enjoyable play – thus there is a
ready-made connection between this kind of teaching method and
videogame design.
It is very easy to understand the basics of positive animal
training, and one of the simplest summaries of the method is Pamela
Dennison's “Three Laws of Learning”:
Rewarded behaviour gets repeated.
Ignored behaviour stops.
Once a behaviour is in place, variable rewards will
strengthen the behaviour.
Positive game design would incorporate these three rules into the
game design process,
It may seem strange to be suggesting using a technique intended
for training animals on humans –surely humans are significantly
different than their animal cousins, and surely games are very
different than animal training methods? Well, yes, but the idea of
games as tools for learning is actually very well established (Raph
Koster etc.) and, as I have previously observed, “games as
learning” has as its flipside “games as rewards”, since
learning and operant conditioning are intimately related.
The Purpose of Positive Game Design
Before proceeding, it's worth explaining that I am not proposing
positive game design as a replacement for other kinds of game
design – there is not, nor can there be, a single method to game
design as I have discussed in the article Zen Game Design and
the book 21st Century Game Design.
Instead, positive game design is proposed as one particular method of
approaching game design. In principle it should be good for at least
three different things:
Educational games whose express goal is to teach
Mass market friendly “casual games” looking to reach an
audience who do not enjoy punishment in games
Tutorials in games expected or intended to reach a wide
audience.
Reward, Don't Punish
The highest principle of positive game design (as in positive
animal training) is reward not punishment. The player should
thus be rewarded for their successes, but not punished for their
failures. This concept may be difficult for some people to get their
heads around, especially given the ubiquity of punishment in
videogames, particularly time-penalty punishments, and punishments
used to generate fear (usually of death and hence failure) in order
to enhance the rewards of play.
Yet there is no reason that a game need punish players, and it is
perfectly possible to design games that reward the player without
punishment (although of course such games look very different from
many of the videogames we usually see). In some respects, we can
already see something akin to positive game design in games such as
the hugely successful Animal Crossing (which has no
punishments, and has already sold 10 million units on DS), or in
advanced tamagotchi's such as the phenomenally successful Nintendogs
(which has no punishments, and has already sold 22 million units on
DS). These titles may not have been designed with positive game
design as a foundational principle, but they demonstrate how
successful this approach can be in appealing to a wide audience not
so willing to endure punishments to enhance their rewards (as with
many gamer hobbyist titles).
The first of Dennison's “Laws” is that rewarded behaviours are
repeated. So, for instance, experienced videogame players always
smash crates (even when they are expressly told there is nothing to
find) because previous games have rewarded them for doing so by
placing power ups and items (i.e. rewards) inside crates. The
behaviour is very hard to break in most videogame players because
they have been heavily rewarded for smashing crates and barrels in
the past.
In designing games for mass market players (those who may not have
the game literacy of experienced hobbyists), it is thus necessary to
ensure that the things you want the player to do will be rewarded.
Now a particular design challenge here is that different players
enjoy different things, and thus there are no guarantees that
something that is offered as a reward will be received as such.
However, there are a few things that can be counted upon to be
interpreted as a reward by any player (although whether they are seen
as a big or a small reward will depend upon the player in question):
Currency rewards: any game with an in-game currency
easily provides rewards; simply pay out some of that cash! This kind
of reward is intuitively understood by all players and is likely to
be a foundational reward scheme in practical positive game design,
but it is important that the currency earned can be spent on
something, else it becomes meaningless.
Avatar Improvement: players love to improve their
abilities in games – in fact, role-playing games have their
success rooted in this kind of improvement. However, when
considering what appeals to a wider audience the kind of
improvements that will work as rewards must be immediately
appreciated. Three strong examples are increasing jump height,
faster speed of movement, and improved weapon damage. Improvements
that are essentially mathematical in basis (+1 Strength, 1%
interest) should be avoided, since these kinds of improvements
appeal to a minority of people (although they work well among gamer
hobbyists).
Gifts: giving the player new things is always
rewarding, even if the gift that is given out isn't of particular
interest. These are great rewards, but can also be very expensive to
develop, so they must be used carefully. Gifts can be purchased
using a currency system, or alternatively can constitute a
structured reward scheme in their own right.
Fanfare: from an ascending scale or an uplifting
chord, you can count on music to create the emotional resonance of a
reward, but of course the more often you rely on the same reward
music the less enjoyable it will become. However, because they are
cheap to implement they are an essential tool for positive game
design.
SFX: a good sound effect can also be extremely
rewarding, especially if it fits the representation of the game
well, or is particularly amusing. And as with fanfares, the cheap
development cost makes fun SFX essential to positive game design.
Eye Candy: just as sound effects and music are
universally enjoyable, explosions, lightshows, colour ripples,
particle systems and anything else that is visually showy and
impressive will work as a reward for almost all people.
Stamps: having a “stamp collection” that fills up
as the player finds or completes things is another reliable reward
(although some players will find this more rewarding than others). A
reassuring 'stamp' sound effect and pleasing graphics all add to the
appeal of stamp collections. This is a possible alternative to a
currency scheme, or a useful additional reward mechanism.
Narrative progress: the vast majority of players enjoy
a story, and advancing that story becomes a reward in itself.
However, there is a risk in this kind of reward in that if the
player is enjoying the story, failing to progress becomes a form of
punishment. Care must be taken when using narrative progress as a
reward.
There are many other kinds of rewards, but the value of these will
vary more from player-to-player, and from game-to-game. For instance,
some players are perfectly happy to chase high scores, even though
these numbers have no real meaning beyond an arbitrary measure of
performance.
When using positive game design, you want to reward the player for
doing whatever it is you want them to be doing. So in a positive
shooter, for instance, the player is supposed to be hitting enemies
and avoiding being hit. You want to reward the player for both these
activities. Blowing things up is usually intrinsically rewarding
(especially with good SFX), and can also provide points/currency, but
normally in a shooter being hit is punished (your ship takes damage
and then eventually blows up). A positive shooter cannot punish, so
being hit can produce a sound effect or show the ship being beaten
up, but it can't actually blow up (this could be perceived as
punishing).
To reward not being hit, a positive game designer can draw upon
ideas used in games such as Crazy Taxi and Burnout,
which give bonus points to the player when they narrowly slip past
other cars. (A parallel could also be drawn with “bullet scraping”
in vert shooters such as Radiant Silvergun). However, this
might train the player to produce near-misses rather than simply to
avoid being hit. An alternative approach would be a combo of missed
shots that is broken when the player is hit. (Although there is a
risk here that a broken combo will feel like a punishment; this idea
is discussed below).
Ignore Instead of Punishing
In positive game design, undesired outcomes are to be ignored and
not punished, which is to say, failure should produce neutral
outcomes, not negative outcomes. This runs counter to most gamers
expectations as to what a game should be, but this deviation from the
norm should not be considered a barrier. (After all, most gamers do
not consider Nintendogs, for instance, to be a game, yet it is
still enjoyed by a great many mass market players).
What constitutes a punishment is an important aspect of
understanding positive training, as there are two very different
kinds of punishment in operant conditioning. The first is known as
positive punishment, meaning “punishment where something bad
happens” and corresponds to what we normally think of as
punishment. Examples in games (in rough order of the degree of
punishment) include:
Damage (thus taking a step towards dying)
Temporary disadvantage (such as reversed controls)
Dying (and thus having to repeat a portion of gameplay)
Game Over (and thus having to repeat a large amount of
gameplay)
Loss of a progress resource (such as experience points)
Permanent loss of equipment or abilities
“Permadeath” (i.e. having to start again from the
beginning).
In general, positive punishments can be interpreted as forms of
time penalty – since whatever is lost could be regained by the
player in a certain amount of time. The most punishing outcome is
thus all progress lost (permadeath, as in Fire Emblem or Steel
Battalion), and the least is a temporarily applied disadvantage,
but it’s important to remember that “punishment” in this
context isn’t a game theory term, it’s about how the player
feels, and thus what is a punishment will vary somewhat from
player to player.
The other kind of punishment in operant conditioning is known as
negative punishment, which means removing something that the
individual in question wants. This could include temporarily
disabling an ability, or the disruption of a scoring mechanic the
player is benefiting from (such as the aforementioned breaking of a
combo that gives increasing score as it advances). Both kinds of
punishment – positive and negative – risk frustrating the player,
and anything that may frustrate a player is a potential punishment of
some kind.
In positive training, positive punishment is never permitted,
and negative punishment is allowed only to prevent entirely
unacceptable behaviours (never to encourage the desired outcome).
Thus in positive game design, the player should never be actively
punished (everything in the bullet list above can never be allowed to
occur) and cases where desirable aspects of play are withdrawn (i.e.
negative punishment) should be minimised wherever possible.
Dennison's second “Law” is that ignored behaviours disappear.
This can easily be seen in any game – if it doesn't work, players
will eventually stop doing it (although some deeply embedded
behaviours, like crate-smashing, may be impossible to remove from an
experienced gamer).
Instead of punishing an undesired outcome, positive game design
should ignore anything but success wherever possible, aiming to
design the game such that failure on the part of the player has as
neutral an effect as possible. It’s acceptable for failure to
result in no gain, but it is best to avoid failure resulting in a
loss of time or resources.
Perhaps the key to making the “ignore don't punish” concept
work is to find ways to replace punishments with neutral outcomes. In
the previous positive shooter example, the death of the player's ship
was replaced with it becoming beaten up (although there is a risk
here of watching it get toasted being rewarding); since the positive
shooter rewards the player for avoiding being hit, the penalty for
being hit is not being rewarded – there is no need for an
additional penalty. In a positive
platformer, a difficult punishment to avoid is the time penalty for
missing a jump, but this is easily fixed by a “safety line”
system – as the player passes various generously positioned points,
they clip on a rope that reels them back to that point if they fail
(not dissimilar to the ‘rewind time’ ability in Prince of
Persia: Sands of Time, but with unlimited uses). In a positive
fighting game, perhaps the closer the avatar gets to death the
stronger it becomes – victory is assured, it's only a question of
how beaten up you get first.
Positive game design might also have to resort to deux ex
machina to make the avoidance of punishment workable, bringing in
an extra element that comes into play if the player struggles for too
long. In a positive shooter, their mothership appears, kills the
enemy and repairs the player's ship before leaving. In a positive
platformer, an ally makes the jump and then runs a line across. In a
positive fighting game, an ally rescues the player, drives off the
foe and heals the player. In ways like these, failures (which risk
feeling like punishment) are recast as neutral outcomes to some
degree.
Some care needs to be taken with design elements intended to
reward which might instead frustrate. In the positive shooter example
above, the issue of broken combos was raised – building up a combo
is exciting because a mistake breaks the chain, but having the chain
drop to zero feels very punishing to many mass market players. To
avoid this, positive game designers can deviate from conventional
videogame logic and simply ameliorate the penalty. Perhaps breaking a
combo subtracts 10 from the counter, rather than zeroing it, or the
combo could remain the same but be temporarily blocked from
increasing. (These are still forms of punishment, but heavily
reduced).
Pragmatically, the needs of gameplay may make it impossible to
entirely eliminate punishment from all kinds of games, so reducing
the extent of the penalty may have to suffice. In general, however,
since success is rewarded there is no need to punish failure
when conducting positive game design. Anything that could frustrate
or annoy the player should be eliminated where possible, and
minimised otherwise.
Variable Rewards
Predicable rewards cease to be rewarding – the secret of keeping
rewards effective is to vary them to ensure maximum effect. In
positive animal training, fixed schedules (rewarding after every
success) are recommended when first training, but in order to ensure
that a specific behaviour persists trainers recommend switching to
variable schedules. Random elements can be used to determine when to
reward, and how much to reward, and the actual rewards themselves can
be varied.
Dennison's third “Law” is that once a particular behaviour is
established, you can strengthen the behaviour by using variable
rewards. Videogames design already uses random reward schedules to
great effect – the random treasure tables in games such as
Castlevania: Symphony of the Night and Diablo (and it's
cousin World of Warcraft) epitomise the compulsiveness that
can come from a variable schedule of reinforcement. Once the player
has learned that killing monsters gives treasure (something common to
most RPG designs) it's easy to use random treasure to make the game
extremely compelling. Positive game design can learn from these
successes, and also take it further.
Animal Crossing, which effectively uses positive game
design techniques, is packed full of variable rewards. Firstly, every
day there are different things on sale in the shop – players thus
come back over and over again, looking for specific things they want,
or seeking something unusual and interesting. Furthermore, apparently
random events occur all the time – a rare fish is seen in the
stream, a rare insect appears, or a new fossil is dug up – and
specifically staged events occur in ways that appear to the player
(at least at first) to be entirely random. It is this density of
variable rewards (and variable events that provide rewards) that help
make this game so compelling for its audience.
With animals, trainers are encouraged to vary what is given as a
reward and to provide additional rewards at random – this can
easily be incorporated into game design. Imagine a positive game with
collectibles: the player picks them up for the rewards inherent in
doing so (including the minor reward of the collection SFX, and the
major reward of whatever benefit is conferred), but they can be made
more rewarding by triggering additional random rewards. Imagine that
sometimes when a cash pickup is collected by the player it is worth
double – this variable reinforcement will make cash pickups even
more desirable to players, and will increase the player's desire to
find and collect them.
The frequency of reward can be made entirely variable as well, and
a system which 'retreats' can help to ensure that the player doesn't
burn out on too generous a reward. A positive platform game might
reward the player every time they jump when they are being trained
(perhaps with a tiny in-game currency reward), then later give them a
modest in-game currency reward 10% of the time they complete a long
jump. Later, it might give them a giant in-game currency reward 5% of
the time they complete a difficult jump. In this way, the player is
encouraged to keep jumping because they know that new rewards will
come, but not when they will come or how big they will be. (This is
the same trick that makes slot machines so compulsive to gamblers).
This is very different to a typical platform game (in which pickups
are almost incidentally collected as the player jumps in specific
places), but positive game design if pursued should be expected to
produce unusual approaches to gameplay.
Existing games have already made use these kinds of randomly
triggered rewards. For example, Front Mission 3 features
special abilities that trigger with random frequency under certain
preset conditions. It might seem that having random effects trigger
would make the combat less engaging, yet Front Mission 3 is
considered to be one of the best games of its kind, and the
compelling nature of these randomly triggered advantages helps
contribute to that popularity.
A word of caution concerning frequency of rewards is warranted.
There is no point in rewards coming so rarely that the player can no
longer expect to encounter any additional reward except by
persevering for ridiculous lengths of times. This happens all the
time in conventional videogame design – there is always some
treasure that is so legendarily rare that players strive to attain –
but as effective as this might be with the gamer hobbyists, who are
often willing to commit a great quantity of time to the games they
enjoy, it should be largely resisted in positive game design.
Conclusion
Positive game design will certainly work because the principles
upon which it is based apply to all mammals, and many other animal
species besides, plus videogames are already capitalising on rewards
and reinforcement schedules in many different ways. Furthermore,
games such as Animal Crossing and Nintendogs arguably
already use positive game design, although perhaps not by intent, and
have achieved success that equals and exceeds the most popular FPS
and RPG titles.
For games seeking the mass market “casual” gamer, and Serious
Games looking to train an arbitrary individual in a certain skill,
positive game design represents an intriguing new possibility worth
exploring. And even in games seeking the conventional gamer hobbyist
(“hardcore”) market, there may be merit in exploring positive
game design in the design of tutorials.
I would like to extend my thanks to the people whose work
influenced this piece, including John Hopson (who first convinced me
that applying Skinner’s models to videogames was not insane, but in
fact inevitable), Raph Koster (who more than anyone challenged me to
credibly consider “games as learning”) and of course Pamela
Dennison (from whom I have learned the most about positive dog
training).
The opening cartoon is Dog Video Game by Mitra Farmand, which I found here as part of her fuffernutter comic blog. As ever, no copyright infringement is intended and I will take the image down if asked.