Given the tremendous volume of psychological research that has been conducted on reward schedules, why is it that the most basic form used in digital games – that behind the grind in every computer RPG, and many other games beside – does not seem to exist in the literature?

I’ve written before about reward schedules in games (also known as schedules of reinforcement in the psychology literature), and as many will know these are considered for the most part to have four basic kinds: fixed ratio, variable ratio, fixed interval and variable interval. (If you are unfamiliar with these terms, please follow the previous link for an explanation, given in the second set of bullet points). In the past, I’ve suggested that the levelling mechanics from RPGs (now present in a growing majority of blockbuster commercial games) can be understood as a fixed ratio schedule. But this overlooks an important question.

If, as psychologists maintain, these four schedules are the basic effective forms of operant conditioning (that is, training for behaviour), why is it that the RPG levelling structure – which originated with Dungeons & Dragons – doesn’t use a fixed ratio at all, but in fact a ratio that gradually increases over iterations? For instance, a typical D&D experience point curve goes 0, 1000, 3000, 6000, 10000, 15000, 21000 etc. Let’s call this an escalating ratio schedule for convenience, or an escalating reward schedule. If psychologists are correct that the four basic schedules listed above are essentially fundamental, it seems quite anomalous that games (which are the example par excellence of reward schedules in action) should prefer an escalating schedule, and equally queer that the academic paperwork seems (as far as I have been able to ascertain) to completely overlook this form as significant.

What’s more, it’s not that the effectiveness of escalating schedules is unknown outside of game design, since animal trainers (as I discuss in the post on Positive Game Design) use escalating schedules as a matter of course to teach behaviour, giving jackpots for the first success, then rewarding every instance, then rewarding with decreasing frequency, before eventually rewarding at random. In the context of animal training, this is largely seen as starting with a fixed ratio (e.g. reward every time) then changing to variable ratio (i.e. random reinforcement). This fits with the literature, but it doesn’t fit so well with what is common in game design. Either the way escalating schedules function in games is maintained solely for reasons of tradition, and games would do better to use fixed followed by variable schedules (which I doubt, except from a certain perspective I’ll discuss shortly) or the escalating schedules are themselves significant forms of operant conditioning that are overlooked in the literature because the prior paradigm (established by B.F. Skinner et al) encourages psychologists to interpret in terms of its pre-existing ‘boxes’.

From my own experience of both animal training and game design, the escalating reward schedule is the ultimate form of operant conditioning when you want to motivate continued interest. It produces the same “post-reinforcement pause” (e.g. slightly reduced interest after levelling up) of a fixed schedule, hence the tendency to consider them under this banner, but they don’t allow the player to just repeat the same activity, because of the escalating requirement for the next reward (e.g. level). This helps maintain interest, since the player is encouraged to actively think about what they’re doing, which is itself rewarding. (It’s less clear to me why this should matter to my dog, who is less likely to be thinking in these terms, but still learns behaviour far more effectively with an escalating schedule).

It seems as if the merit of the escalating schedule is that in the early stages, the rewards come regularly, which helps the habit set in, while later on the rewards come further apart which gives more time to focus on the nature of the activity rather than having the activity overshadowed by the reward. Hence, in the case of teaching my dog to fetch, the rewards at the outset get him interested in what’s going on, then when the rewards come less frequently the activity itself (running after a ball or frisbee in this example) starts to become its own reward, and the food treats are merely the icing on the cake. In the case of players, the low experience point requirements for levelling at the beginning of a game (or the low target values in the case of Achievements) generate a high awareness of the rewards, then the escalating schedule increases the space between rewards to provide more time to think about how to achieve the goals or enjoy the process of attaining them.

It’s striking that almost all games using escalating schedules use an end point. Hence Dungeons & Dragons, World of Warcraft and almost any other RPG you’d care to mention with an experience system has a maximum level and/or a point at which the next level has a fixed value rather than an escalating value. The escalating schedule becomes counter-productive beyond the point that the player maintains interest in the next reward (e.g. level) – if, for instance, the amount of work required to reach the next reward seems insufficient to motivate proceeding. It is at at this point that the experience of escalating schedules ceases to be addictive fun and becomes grinding. Precisely how this term is used is as a pejorative to refer to conscious awareness of the ratio schedule, and hence the onset of boredom with respect to the process in question.

It’s equally striking that once the end point kicks in, games have supplemental systems based upon variable reward schedules at the back end. In the case of World of Warcraft, for instance, random rare treasure acquisition (“rare drops”) kick in near the maximum levels to maintain interest – which is exactly what animal trainers advocate when they talk about switching to rewarding randomly once the desired behaviour is established. This isn’t surprising since variable schedules are the most addictive, as any manufacturer of slot machines will tell you. The literature states that variable ratio schedules produce both the highest rate of responding and the greatest resistance to extinction, which is psychologist speak for how to make a slot jockey, or, for that matter, a WoW addict.

So despite the lack of discussion in the psychology literature concerning escalating schedules, the general way reward schedules are used in games does fit with the academic perspective on them: fixed ratio schedules to generate a high level of activity to begin with (to establish the habit) followed by variable ratio schedules to power up the degree of addiction. All that’s missing from the papers discussing this is the importance of the escalating ratio schedule (as opposed to the strictly fixed schedule), but from the point of view of game design this is a highly significant omission, since this pattern is utterly de rigueur, and not (I would suggest) for purely traditional reasons, since game designers would rather reinvent the wheel than borrow an existing wheel any day of the week. Game design patterns that persist do so primarily because they are successful in some manner pertinent to the games they are used in, and only rarely because of a culture of momentum.

The grind mystery, then, is why the escalating reward structures that produce the experience of grinding (which, despite the pejorative, are the basis of the core fun for an ever increasing number of games) are not considered in the psychological literature at all, given their effectiveness in games (and, I am positing, their greater effectiveness when compared to fixed schedules). Does this reflect an oversight on the part of psychological researchers? Or perhaps, as I have suggested here, it is because the gamer experience involves thinking about what to do and how to do it, and this parallel source of reward (i.e. fun) can be strangled out of existence by a game that doesn’t provide enough slack to allow for it. That’s the grind. And that’s why gamers complain about it, even though they continue to buy those games whose designs depend upon it for their success.