Animal trainers for many years now have been using what is known
as positive training, a system which uses only rewards and
never punishment for teaching behaviours to animals. The technique is
effective, and has been used to train everything from champion
showdogs to the orcas at SeaWorld. The same technique could be
used as a system of game design – but what would such positive
game design be like, and what would it be good for?
Positive Animal Training
Positive training, as practised by a great many dog trainers such as Pamela Dennison, Pat Miller and Gwen Bailey, is based upon the principles of operant conditioning (Skinner etc.) to develop reward schedules that reinforce desired behaviours and eliminate undesired behaviours. As I have discussed previously in my exploration of why people play games, the same reward mechanism in the brain that makes these kinds of training techniques effective lies behind the different kinds of enjoyable play – thus there is a ready-made connection between this kind of teaching method and videogame design.
It is very easy to understand the basics of positive animal training, and one of the simplest summaries of the method is Pamela Dennison's “Three Laws of Learning”:
Rewarded behaviour gets repeated.
Ignored behaviour stops.
Once a behaviour is in place, variable rewards will strengthen the behaviour.
Positive game design would incorporate these three rules into the game design process,
It may seem strange to be suggesting using a technique intended for training animals on humans –surely humans are significantly different than their animal cousins, and surely games are very different than animal training methods? Well, yes, but the idea of games as tools for learning is actually very well established (Raph Koster etc.) and, as I have previously observed, “games as learning” has as its flipside “games as rewards”, since learning and operant conditioning are intimately related.
The Purpose of Positive Game Design
Before proceeding, it's worth explaining that I am not proposing positive game design as a replacement for other kinds of game design – there is not, nor can there be, a single method to game design as I have discussed in the article Zen Game Design and the book 21st Century Game Design. Instead, positive game design is proposed as one particular method of approaching game design. In principle it should be good for at least three different things:
Educational games whose express goal is to teach
Mass market friendly “casual games” looking to reach an audience who do not enjoy punishment in games
Tutorials in games expected or intended to reach a wide audience.
Reward, Don't Punish
The highest principle of positive game design (as in positive animal training) is reward not punishment. The player should thus be rewarded for their successes, but not punished for their failures. This concept may be difficult for some people to get their heads around, especially given the ubiquity of punishment in videogames, particularly time-penalty punishments, and punishments used to generate fear (usually of death and hence failure) in order to enhance the rewards of play.
Yet there is no reason that a game need punish players, and it is perfectly possible to design games that reward the player without punishment (although of course such games look very different from many of the videogames we usually see). In some respects, we can already see something akin to positive game design in games such as the hugely successful Animal Crossing (which has no punishments, and has already sold 10 million units on DS), or in advanced tamagotchi's such as the phenomenally successful Nintendogs (which has no punishments, and has already sold 22 million units on DS). These titles may not have been designed with positive game design as a foundational principle, but they demonstrate how successful this approach can be in appealing to a wide audience not so willing to endure punishments to enhance their rewards (as with many gamer hobbyist titles).
The first of Dennison's “Laws” is that rewarded behaviours are repeated. So, for instance, experienced videogame players always smash crates (even when they are expressly told there is nothing to find) because previous games have rewarded them for doing so by placing power ups and items (i.e. rewards) inside crates. The behaviour is very hard to break in most videogame players because they have been heavily rewarded for smashing crates and barrels in the past.
In designing games for mass market players (those who may not have the game literacy of experienced hobbyists), it is thus necessary to ensure that the things you want the player to do will be rewarded. Now a particular design challenge here is that different players enjoy different things, and thus there are no guarantees that something that is offered as a reward will be received as such. However, there are a few things that can be counted upon to be interpreted as a reward by any player (although whether they are seen as a big or a small reward will depend upon the player in question):
Currency rewards: any game with an in-game currency easily provides rewards; simply pay out some of that cash! This kind of reward is intuitively understood by all players and is likely to be a foundational reward scheme in practical positive game design, but it is important that the currency earned can be spent on something, else it becomes meaningless.
Avatar Improvement: players love to improve their abilities in games – in fact, role-playing games have their success rooted in this kind of improvement. However, when considering what appeals to a wider audience the kind of improvements that will work as rewards must be immediately appreciated. Three strong examples are increasing jump height, faster speed of movement, and improved weapon damage. Improvements that are essentially mathematical in basis (+1 Strength, 1% interest) should be avoided, since these kinds of improvements appeal to a minority of people (although they work well among gamer hobbyists).
Gifts: giving the player new things is always rewarding, even if the gift that is given out isn't of particular interest. These are great rewards, but can also be very expensive to develop, so they must be used carefully. Gifts can be purchased using a currency system, or alternatively can constitute a structured reward scheme in their own right.
Fanfare: from an ascending scale or an uplifting chord, you can count on music to create the emotional resonance of a reward, but of course the more often you rely on the same reward music the less enjoyable it will become. However, because they are cheap to implement they are an essential tool for positive game design.
SFX: a good sound effect can also be extremely rewarding, especially if it fits the representation of the game well, or is particularly amusing. And as with fanfares, the cheap development cost makes fun SFX essential to positive game design.
Eye Candy: just as sound effects and music are universally enjoyable, explosions, lightshows, colour ripples, particle systems and anything else that is visually showy and impressive will work as a reward for almost all people.
Stamps: having a “stamp collection” that fills up as the player finds or completes things is another reliable reward (although some players will find this more rewarding than others). A reassuring 'stamp' sound effect and pleasing graphics all add to the appeal of stamp collections. This is a possible alternative to a currency scheme, or a useful additional reward mechanism.
Narrative progress: the vast majority of players enjoy a story, and advancing that story becomes a reward in itself. However, there is a risk in this kind of reward in that if the player is enjoying the story, failing to progress becomes a form of punishment. Care must be taken when using narrative progress as a reward.
There are many other kinds of rewards, but the value of these will vary more from player-to-player, and from game-to-game. For instance, some players are perfectly happy to chase high scores, even though these numbers have no real meaning beyond an arbitrary measure of performance.
When using positive game design, you want to reward the player for doing whatever it is you want them to be doing. So in a positive shooter, for instance, the player is supposed to be hitting enemies and avoiding being hit. You want to reward the player for both these activities. Blowing things up is usually intrinsically rewarding (especially with good SFX), and can also provide points/currency, but normally in a shooter being hit is punished (your ship takes damage and then eventually blows up). A positive shooter cannot punish, so being hit can produce a sound effect or show the ship being beaten up, but it can't actually blow up (this could be perceived as punishing).
To reward not being hit, a positive game designer can draw upon ideas used in games such as Crazy Taxi and Burnout, which give bonus points to the player when they narrowly slip past other cars. (A parallel could also be drawn with “bullet scraping” in vert shooters such as Radiant Silvergun). However, this might train the player to produce near-misses rather than simply to avoid being hit. An alternative approach would be a combo of missed shots that is broken when the player is hit. (Although there is a risk here that a broken combo will feel like a punishment; this idea is discussed below).
Ignore Instead of Punishing
In positive game design, undesired outcomes are to be ignored and not punished, which is to say, failure should produce neutral outcomes, not negative outcomes. This runs counter to most gamers expectations as to what a game should be, but this deviation from the norm should not be considered a barrier. (After all, most gamers do not consider Nintendogs, for instance, to be a game, yet it is still enjoyed by a great many mass market players).
What constitutes a punishment is an important aspect of understanding positive training, as there are two very different kinds of punishment in operant conditioning. The first is known as positive punishment, meaning “punishment where something bad happens” and corresponds to what we normally think of as punishment. Examples in games (in rough order of the degree of punishment) include:
Damage (thus taking a step towards dying)
Temporary disadvantage (such as reversed controls)
Dying (and thus having to repeat a portion of gameplay)
Game Over (and thus having to repeat a large amount of gameplay)
Loss of a progress resource (such as experience points)
Permanent loss of equipment or abilities
“Permadeath” (i.e. having to start again from the beginning).
In general, positive punishments can be interpreted as forms of time penalty – since whatever is lost could be regained by the player in a certain amount of time. The most punishing outcome is thus all progress lost (permadeath, as in Fire Emblem or Steel Battalion), and the least is a temporarily applied disadvantage, but it’s important to remember that “punishment” in this context isn’t a game theory term, it’s about how the player feels, and thus what is a punishment will vary somewhat from player to player.
The other kind of punishment in operant conditioning is known as negative punishment, which means removing something that the individual in question wants. This could include temporarily disabling an ability, or the disruption of a scoring mechanic the player is benefiting from (such as the aforementioned breaking of a combo that gives increasing score as it advances). Both kinds of punishment – positive and negative – risk frustrating the player, and anything that may frustrate a player is a potential punishment of some kind.
In positive training, positive punishment is never permitted, and negative punishment is allowed only to prevent entirely unacceptable behaviours (never to encourage the desired outcome). Thus in positive game design, the player should never be actively punished (everything in the bullet list above can never be allowed to occur) and cases where desirable aspects of play are withdrawn (i.e. negative punishment) should be minimised wherever possible.
Dennison's second “Law” is that ignored behaviours disappear. This can easily be seen in any game – if it doesn't work, players will eventually stop doing it (although some deeply embedded behaviours, like crate-smashing, may be impossible to remove from an experienced gamer).
Instead of punishing an undesired outcome, positive game design should ignore anything but success wherever possible, aiming to design the game such that failure on the part of the player has as neutral an effect as possible. It’s acceptable for failure to result in no gain, but it is best to avoid failure resulting in a loss of time or resources.
Perhaps the key to making the “ignore don't punish” concept work is to find ways to replace punishments with neutral outcomes. In the previous positive shooter example, the death of the player's ship was replaced with it becoming beaten up (although there is a risk here of watching it get toasted being rewarding); since the positive shooter rewards the player for avoiding being hit, the penalty for being hit is not being rewarded – there is no need for an additional penalty. In a positive platformer, a difficult punishment to avoid is the time penalty for missing a jump, but this is easily fixed by a “safety line” system – as the player passes various generously positioned points, they clip on a rope that reels them back to that point if they fail (not dissimilar to the ‘rewind time’ ability in Prince of Persia: Sands of Time, but with unlimited uses). In a positive fighting game, perhaps the closer the avatar gets to death the stronger it becomes – victory is assured, it's only a question of how beaten up you get first.
Positive game design might also have to resort to deux ex machina to make the avoidance of punishment workable, bringing in an extra element that comes into play if the player struggles for too long. In a positive shooter, their mothership appears, kills the enemy and repairs the player's ship before leaving. In a positive platformer, an ally makes the jump and then runs a line across. In a positive fighting game, an ally rescues the player, drives off the foe and heals the player. In ways like these, failures (which risk feeling like punishment) are recast as neutral outcomes to some degree.
Some care needs to be taken with design elements intended to reward which might instead frustrate. In the positive shooter example above, the issue of broken combos was raised – building up a combo is exciting because a mistake breaks the chain, but having the chain drop to zero feels very punishing to many mass market players. To avoid this, positive game designers can deviate from conventional videogame logic and simply ameliorate the penalty. Perhaps breaking a combo subtracts 10 from the counter, rather than zeroing it, or the combo could remain the same but be temporarily blocked from increasing. (These are still forms of punishment, but heavily reduced).
Pragmatically, the needs of gameplay may make it impossible to entirely eliminate punishment from all kinds of games, so reducing the extent of the penalty may have to suffice. In general, however, since success is rewarded there is no need to punish failure when conducting positive game design. Anything that could frustrate or annoy the player should be eliminated where possible, and minimised otherwise.
Variable Rewards
Predicable rewards cease to be rewarding – the secret of keeping rewards effective is to vary them to ensure maximum effect. In positive animal training, fixed schedules (rewarding after every success) are recommended when first training, but in order to ensure that a specific behaviour persists trainers recommend switching to variable schedules. Random elements can be used to determine when to reward, and how much to reward, and the actual rewards themselves can be varied.
Dennison's third “Law” is that once a particular behaviour is established, you can strengthen the behaviour by using variable rewards. Videogames design already uses random reward schedules to great effect – the random treasure tables in games such as Castlevania: Symphony of the Night and Diablo (and it's cousin World of Warcraft) epitomise the compulsiveness that can come from a variable schedule of reinforcement. Once the player has learned that killing monsters gives treasure (something common to most RPG designs) it's easy to use random treasure to make the game extremely compelling. Positive game design can learn from these successes, and also take it further.
Animal Crossing, which effectively uses positive game design techniques, is packed full of variable rewards. Firstly, every day there are different things on sale in the shop – players thus come back over and over again, looking for specific things they want, or seeking something unusual and interesting. Furthermore, apparently random events occur all the time – a rare fish is seen in the stream, a rare insect appears, or a new fossil is dug up – and specifically staged events occur in ways that appear to the player (at least at first) to be entirely random. It is this density of variable rewards (and variable events that provide rewards) that help make this game so compelling for its audience.
With animals, trainers are encouraged to vary what is given as a reward and to provide additional rewards at random – this can easily be incorporated into game design. Imagine a positive game with collectibles: the player picks them up for the rewards inherent in doing so (including the minor reward of the collection SFX, and the major reward of whatever benefit is conferred), but they can be made more rewarding by triggering additional random rewards. Imagine that sometimes when a cash pickup is collected by the player it is worth double – this variable reinforcement will make cash pickups even more desirable to players, and will increase the player's desire to find and collect them.
The frequency of reward can be made entirely variable as well, and a system which 'retreats' can help to ensure that the player doesn't burn out on too generous a reward. A positive platform game might reward the player every time they jump when they are being trained (perhaps with a tiny in-game currency reward), then later give them a modest in-game currency reward 10% of the time they complete a long jump. Later, it might give them a giant in-game currency reward 5% of the time they complete a difficult jump. In this way, the player is encouraged to keep jumping because they know that new rewards will come, but not when they will come or how big they will be. (This is the same trick that makes slot machines so compulsive to gamblers). This is very different to a typical platform game (in which pickups are almost incidentally collected as the player jumps in specific places), but positive game design if pursued should be expected to produce unusual approaches to gameplay.
Existing games have already made use these kinds of randomly triggered rewards. For example, Front Mission 3 features special abilities that trigger with random frequency under certain preset conditions. It might seem that having random effects trigger would make the combat less engaging, yet Front Mission 3 is considered to be one of the best games of its kind, and the compelling nature of these randomly triggered advantages helps contribute to that popularity.
A word of caution concerning frequency of rewards is warranted. There is no point in rewards coming so rarely that the player can no longer expect to encounter any additional reward except by persevering for ridiculous lengths of times. This happens all the time in conventional videogame design – there is always some treasure that is so legendarily rare that players strive to attain – but as effective as this might be with the gamer hobbyists, who are often willing to commit a great quantity of time to the games they enjoy, it should be largely resisted in positive game design.
Conclusion
Positive game design will certainly work because the principles upon which it is based apply to all mammals, and many other animal species besides, plus videogames are already capitalising on rewards and reinforcement schedules in many different ways. Furthermore, games such as Animal Crossing and Nintendogs arguably already use positive game design, although perhaps not by intent, and have achieved success that equals and exceeds the most popular FPS and RPG titles.
For games seeking the mass market “casual” gamer, and Serious Games looking to train an arbitrary individual in a certain skill, positive game design represents an intriguing new possibility worth exploring. And even in games seeking the conventional gamer hobbyist (“hardcore”) market, there may be merit in exploring positive game design in the design of tutorials.
I would like to extend my thanks to the people whose work influenced this piece, including John Hopson (who first convinced me that applying Skinner’s models to videogames was not insane, but in fact inevitable), Raph Koster (who more than anyone challenged me to credibly consider “games as learning”) and of course Pamela Dennison (from whom I have learned the most about positive dog training).
The opening cartoon is Dog Video Game by Mitra Farmand, which I found here as part of her fuffernutter comic blog. As ever, no copyright infringement is intended and I will take the image down if asked.
"The behaviour is very hard to break in most videogame players because they have been heavily rewarded for smashing crates and barrels in the past."
I think this might have more to do with the fact that smashing a crate is more fun than leaving it alone (if only slightly). So even if you know a power up won't ever come out, you might as well break it anyway just for kicks. Now if a game actually punished you for breaking crates, I'm sure no one would do it.
"Now a particular design challenge here is that different players enjoy different things, and thus there are no guarantees that something that is offered as a reward will be received as such."
Good point. Another thing that should be taken into account is cost:reward. For example, getting a bigger house in Animal Crossing may technically be a reward, but if it means I have to play that garbage then the cost:reward is way too unbalanced.
"In a positive fighting game, an ally rescues the player, drives off the foe and heals the player."
Haha, in a way that would be more punishing than losing the fight. Getting rescued by an ally, healed and him taking care of the opponent sounds more humiliating than getting knocked out. At least if I fought hard and lost I still have some of my dignity!
Posted by: Sirc | Wednesday, 19 August 2009 at 22:53
Some of the 'deus ex machina' ways suggested to enable progression sound like rewards - it certainly sounds like an interesting animation (at least) that I'd want to see again.
Dragon's Lair's death screens were so varied that they almost acted as 'compensations' - you are given a 'time-punishment' but treated to a new animation each time you die in a new way. Maybe this helped the game endure despite its unforgiving nature and lack of interaction - the bittersweet mix of a restart and a 'rewarding' animation providing an addictive mix and almost training people to play the game to die in new ways.
I think a 'positive fighting game' could simply forego any possibility of your character dying. Maybe exclude your own life bar, have infinite life bars or have the character stay alive when the life bar reaches 0, but then recieve no bonus points for energy remaining.
Rewards could then be given for time taken, high combos and lack of damage taken.
This might actually be a fun, interesting high-score game though the variability of rewards is at odds with the understandability that I think a good high score system should have.
Would a decent compromise be to multiply all points by the level number? Or is unpredictability the key element of 'variable rewards'?
I know that mice hit the lever more if there's % chance of getting food than if food is certain, but what if it follows a pattern? Has that experiment ever been done?
Anyway, great article. Lots of food for thought. I hope you don't mind my stream-of-consciousness comment.
Love.
Posted by: Behrooz 'Bezman' Shahriari | Friday, 21 August 2009 at 00:27
Sirc: I think it's fair to say that positive game design isn't intended for a player such as yourself you is already an established gamer! :) We're talking about ways of making less punishing games for the purpose of training or entertaining a wider audience, after all.
And regarding the crates, "The Legend of Zelda: The Wind Waker" actually *did* have a section which punished you for smashing pots. But only in one area of the game world. I don't think it stopped people for long. :p But I take your point that there's something intrinsically entertaining in smashing things up for many people.
Bezman: yes, you wouldn't necessarily want the deus ex machina to seem directly rewarding... it's a fine line to walk.
As for Dragon's Lair, this was such a punishing arcade game, and at the price they were charging at the time I'm not sure dying really was good value! :)
"Or is unpredictability the key element of 'variable rewards'?"
It's certainly an important element of variable rewards - if people know exactly how the rewards work, they can game them, but if they are truly variable it is (according to positive training theory) more re-enforcing.
And no, I never mind stream-of-consciousness comments. :)
Cheers for the comments!
Posted by: Chris | Friday, 21 August 2009 at 10:59
I am too tired right now to write a coherent comment, but suffice it to say that I am *extremely* intrigued by this 'positive game design' approach and its potential for educational and mass market game design. Thank you.
The one example that really demonstrates this to me is Music Catch, a surprisingly popular Flash game created by Reflexive, a casual game studio. Its popularity compelled me to look closer at its design decisions.
Reading your book 21st Century Game Design with its discussion of Wanderer play styles helped me think about this further, but this article really provides some startlingly clear perspective.
I'm not sure yet how this will impact my approach, but I'll write back again when I've had more time to think about it. Thanks again.
Posted by: axcho | Thursday, 15 October 2009 at 09:56
axcho: glad to hear this has sparked something useful in your head! :) Please do come back and let me know if anything tangible comes of it.
Posted by: Chris | Thursday, 15 October 2009 at 10:55