Over at Ice Water Games, Kevin Maxon provides another glorious rebuttal to my firestarter. Here’s an extract:

In some sense, ignorance might be an appropriate word for what I’m advocating: for creators to intentionally ignore with greater diligence the pressures to be similar, to follow fashion or money or power, pressures to use objective, scientific methods of art production. And similarly, I think part of what I’m advocating for could be called dogmatism: for creators to hold firm in their values and goals in order to create works that are more distinct, more filled with themselves, more honest and interesting and worth talking about.

Please rush over to his blog to read the entirety of The Ignorant Dogmatist right away!

The original firestarter makes one of its targets the kind of self-focussed indie game design method Kevin defends here. Yet I cannot do anything but respect Kevin’s commitment to exploring his own creative vision in games. For me, what Kevin is doing is making what I call artgames, and the moment you’re committed to art you are no longer practicing a commercial craft. You’ve gone down a marvellous rabbit hole, one where money may be tight but that worthwhile things get made. Almost everything I’ve thought worthwhile in games in the past five years has been an artgame… This is largely what I choose to play these days.

Why sell out artists in The Craft of Game Design Cannot Be Measured By Any Metric, then? When I look at Kevin’s output, which includes Eidolon and The Absence of Is, I see someone pursuing their vision for its own sake, which is the mark of an artist – a way of life I greatly respect, not least because it now feels closed to me. But when I look at the indie market, I see people pursuing a similar kind of self-focussed process and making yet more-of-the-same violent, repetitive ordinariness. Such indies are, I presume, trying to make a living – and they’re doing it badly. It was these indies I wanted to lambast.

If my piece in any way discourages someone from accepting the role of the starving artist, with all that entails, I apologise unreservedly. Art is one of the greatest ways to add value to life beyond money. But most indies aren’t making art. They’re masturbating into a codebase and thinking they’ll hit big doing so. Maybe I should respect that as a kind of art, but I just see it as bad commercial practice.

With my philosopher-hat on (I wear many, conflicting hats), I can only smile with an inner warmth at this line:

I think that often, the non-mechanical components of a game are more important than the mechanical ones, and so I tend to work on visuals and writing at least as early as mechanics.

I wrote Imaginary Games in part to defend this philosophy, and next week I’ll present to a hundred game academics about how games are more than their merely artefactual machinery. Kevin describes himself as willingly ‘ignorant’… his ignorance, though, is closer to the kind praised in Jacques Rancière’s The Ignorant Schoolmaster – it is a freedom from stultifying conformity. I could never oppose this, especially not when it is done in the pursuit of art. Everyone must discover who they are, sometimes over and over again… and never let someone like me tell you otherwise.

Delighted to announce that Dr Richard Bartle and I came the closest yet to actually meeting as we were both guests on Afzal Ahmed’s Asia Wired TV show. We’re discussing Pokémon Go, including the reasons that the game has been so late to release in Asia, why it's garnered such attention, and a little bit of the history behind it. Despite Richard writing a chapter for one of my books, and our occasional email exchanges about player modelling, we’re never actually met in person – this is our first real time conversation. Check it out!

Airs this Sunday (31st July) at 11 am on the Islam Channel in the UK. Also available via YouTube.

In a comment replying to The Craft of Game Design Cannot Be Measured By Any Metric, game designer and Chief Creative Officer at Spryfox Dan Cook gave such a sterling, thorough rebuttal that I’ve reposted it here in full.

Empiricism in Game Design

When I design I have a mental model of how I imagine my game will be played by players. This includes predictions about player emotions, learning, buying behaviors and a dozen other factors necessary to make a self-sustaining game in one of today’s various markets. I also make predictions about how markets will act. Platform desires, player designers, press desires.

Then we build the game, or at least we build an initial version of it.

Then we playtest the game to see if the my predictions worked out. Most of the time they don’t. In the best cases I’m only off by a factor or two. In the worse cases I’m off by several orders of magnitude. However, I may also find that players behaved in a manner that was actually more interesting than I predicted.

So we build another iteration of the game. Somehow, we need to connect the empirical reality of what the playtest suggests with what we predict will happen. This usually involves updating our models, sometimes radically. Often incrementally.

For some designers, this process can be frustrating. The reality of player behavior imposes constraints on their mostly imaginary vision. But I tend to see constraints as necessary to the process of design. And constraints based off observing real people playing the game tends to more often than not yield opportunities to impact the real shared world of many people vs the isolated imaginary world of a single person. We find new ways of playing that are more vibrant and interesting.

How are metrics useful when iterated on a game?

Game designers are information starved. With writing, we have an imperfect but competent mechanism for imagining how someone might feel reading a bit of text. In order to write, you must read. And thus you are forced to process a work in a somewhat similar fashion to how a potential reader might process. Game developers do not have this luxury. We build systems multiple times removed from a player’s experience. Write some code. Do a dozen other steps. Build an executable that someone somewhere runs. Knowing how people with react to what we make is hard.

So we use crutches. We create complex models of how players think. We use ‘proven’ patterns. We watch players and try to imagine what they are feeling. Then we try to backtrack all far removed information to whether or not a number in the bowels of a broken machine should be 2 or 4.

There are certainly classes of information we can extract more easily. Surface player emotions on individual playthroughs. Awesome. We can do that. But human behavior is broad. We see the need to sample behaviors across populations and discover central tendencies or outliers.

So metrics or analytics are that tool. They let us understand statistical patterns of behavior. Do they let us see inside the minds of our players? No. Nothing does yet. Do they replace in person playtests? No. Smart designers use multiple sources of insight.

But metrics do provide an amazing range of insight by allowing us to look at hard problems from a different direction. If players in an MMO are flooding forums with complaints about a change, how many people are impacted? How did playstyles change?

When balancing economies and progression systems, metrics are essential. You can’t do an in-person playtest of someone playing a game for 90 days. The old tools don’t work. And various forms of data collection do.

Maybe all this doesn’t need to be said. Maybe you are worried about something else entirely.

Are you worried about how metrics shines a light on bullshit design? Because a lot of design is unsubstantiated bullshit. We imagine people will play a game a certain way and then they don’t. Such an ego buster. Metrics beat us with bully numbers. They bluntly state our initial idea was flawed. Or even worse, the thing that people have been praising us for years doesn’t actually apply to anyone but some weird elite group of outliers that happens to give out chintzy feel good awards. Reality can be cruel when you live in a fantasy. But it also acts as a constraint that forces us to up our game and make something that works. Versus wandering blindly off a cliff in a feel good haze. Which I’ve done. (Lovely until you fall).

Are you worried that Bad Men use metrics in a reductive fashion to emphasize making money over art? Bad Men have been emphasizing making money over art for a very long time. For any golden era of games there were penny pinchers micromanaging creative decisions at a level that destroyed souls. Might I suggest that a new tool for getting data is not the actual problem. The team sets their goals. The tools just get them there.

Are you worried that we are using Dumb Metrics? That the dumb patterns dumbly followed by dumb practitioners result in dumb ideas and dumb games? Well it is true. And the solution is one that applies to all complex instruments used in the pursuit of art and beauty: Get Good.

I actually see metrics, competent design and building something positive that meets player needs as three complementary pursuits. I’ve asked “Well, what do players want and how does that align with business? And how does that align with art or craft?”

Here’s one answer. Many players want connection with meaning and community. They want mastery and agency. This leads to them enjoying an activity for a long period of time. That results in great retention metrics. And when deep needs are being met, people are willing to spend. Will I spend a buck on Pokemon lures to enhance a relaxing afternoon with my wife at the coffee shop? Yes. It makes for joyful light conversation. The game improves our relationship by creating a shared playful space.

Metrics track and tune all this. Is that evil? Just the opposite. I consider it doing great good for the world through competent design practices.

I have made minor edits to the text to make it read as a standalone post: the original comment is still available under the original post.

Brian Green (AKA Psychochild) has a piece responding to last week’s firestarter and arguing that there is a purpose for metrics in a game. Here’s an extract:

I dislike the absolutist nature of the argument, and prefer the more nuanced version. As a creative person, I still like things like food, a roof, and perhaps air conditioning when the temperature and humidity get high outside. But, I think it is important to realize that there is a decision to be made. One can choose to pure creative energy to create experiences on one extreme, pandering to tastes and maximizing for profit on the other, and a lot of room between the two extremes. And, as much as we might lionize the indie iconoclasts, the reality is that sometimes it takes a lot of work and understanding what people actually want to survive as an indie.

The argument Brian refers to here is art vs. commerce. Personally, I don’t accept a significant divide between art and commerce here… the vast majority of art is commercial in the sense that this term is used today: music recordings and performances are sold, paintings are auctioned, theatre and cinemas charge an entry fee. Knowing that games are artworks doesn’t mean the people who make them don’t deserve to be fed. I absolutely agree with Brian that game developers are no different in this regard: part of my argument in The Craft of Game Design Cannot Be Measured By Any Metric is precisely that indies, in rejecting commercial design considerations, are gambling on their livelihood.

So I accept Brian’s point that metrics can be used responsibly, at least in principle. My argument is only that there is a tension between the craft of game design, and engineering systems for commercial exploitation. Developers who can use metrics to assist their game design practices ought to make clear how this can be achieved without it becoming exploitative. I welcome the discussion here – it is this discourse that I feel is substantially missing.

You can read the entirety of The purpose of metrics in a game over on Psychochild’s Blog – check it out!

If game design is a craft, what becomes of it when game development is driven solely by financial metrics? Does any of the craft remain? Or are games reduced to mere commercial pit traps, luring in and monetising their unwitting victims?

A little over a decade ago, when my friend and colleague Richard Boon and I were writing 21st Century Game Design, I had predicted that this century in games was going to be characterised by a new focus upon understanding players, and that this would be attained by various models of player behaviour. I suggest (with the benefit of hindsight) that this general claim was correct, and that we have gone from an era where game design was dominated by dogmatic assumptions and self-satisfying design practices (although neither of these have gone away…) to one where understanding how players relate to games is an inescapable part of the videogame industry.

But we made one crucial error in that book. My assumption had been that modelling player behaviour entailed understanding how to satisfy play needs, which is to say, having a positive, inclusive, moral and practical relationship with players. But the dominant forms of player modelling right now have absolutely no need to understand how to satisfy players in any form, because the principal form of model we are using right now is analytic metrics – and these metrics are blind to any aspect of the mental states of the player whatsoever. If our image of game design in the 21st century was that the industry was going to be making money by creating games that deeply satisfied their players, what we are actually facing now is an industry that makes the majority of its money by simply analysing where the leaks are in their cashflow, and acting as digital predators to suck spare change out of players’ digital wallets.

It may be helpful to look at the key metrics at use today to verify what I’m claiming. Firstly, there are the measures of activity – Daily Active Users (DAUs), Sessions, Stickiness (DAU/MAU), Retention and its inverse, Churn. Then, the measures of monetisation – Conversion Rate (percentage of players making purchases), ARPDAU and ARPPU (Average Revenue Per Daily Active User, or Per Paying User). Also, game economy measures for Sources, Sinks, and the Flow Rate of in-game currencies, all geared towards engineering sufficient sparseness that players will be encouraged to pay money for advantages. And that’s what it’s all about: squeezing money out of players' impulses – although in analytics, there are no players, only users, just like the narcotics industry. As the company Game Analytics observe with the admirable unvarnished honesty that belongs to these thoroughly pragmatic commercial practices:

Successful free-to-play games create long-term relationships with users. Users that enjoy the experience enough are willing to pay to for a competitive advantage. A game needs to have strong retention to have time to build this relationship. (Emphasis added.)

One of the most coherent supporters of the free-to-play business model where such metrics dominate is Nicholas Lovell, author of The Curve and regular on the same speaking circuit as me. We first met at Develop Liverpool, many years back, and our paths still occasionally cross. He views the challenges of that side of the market as not so much about monetisation (he rankles at being called a ‘monetisation consultant’) as about retention, in accordance with the quote above. But I read very little from him about the craft of game design, and his recent talks have tended to be framed in terms of the keywords ‘Acquire, Retain, Monetise’, which sounds like a scaled down version of the Ferengi Rules of Acquisition. Nicholas continually insists our industry can self regulate itself away from abusive practices – but I still don’t see any sign of this, nor indeed do I detect much interest in doing so.

The focus on metrics over game design has brought the videogame industry closer to its less reputable but more profitable cousin ‘gaming’ – what's commonly known as gambling – and with it, we have a host of ethical questions about what we are doing, none of which can be merely presupposed. We urgently need a debate on monetisation practices to establish what ethical metrics consist of, but the industry does not want to have this talk. I offered a dynamite panel to GDC this year on this topic, but it was knocked out of contention instantly. The industry is afraid to have the conversation, but until we are ready to address questions about what metrics mean for game design as a craft, we have a serious unaddressed problem that affects the integrity of the games industry. Of course, in purely capitalistic terms there is no integrity, there is only money. But money is just another of our imaginary games – it just happens to be one that we all take very, very seriously, since we have lost our ability to feed ourselves without it.

One game designer who has taken a stand on the ethics of monetisation is former Free Realms creative lead Laralyn McWilliams, who quit a job out of disgust over the issues I’m highlighting here. In an interview back in 2014 entitled “The problem with ‘best practices’ in free-to-play”, Laralyn reports how designing for ‘friction’, which is to say, monetising player frustration, finally became something she couldn’t endorse:

…a designer came to me and said there was a spot where it got really rough; there weren't enough quests, and the grind was really terrible. He wanted to add five or ten quests to make it feel better…. But when I looked at our numbers that was the spot where we had our best monetisation. The awful feeling of that grind was getting people to spend money, so I had to say no to something that would make players happy because it would cut our revenue. At that point I said, ‘Nope,’ and I got out of social games.

Against the ruthless focus on the bottom line is the possibility, if nothing else, that game design can fulfil its calling as a craft, and that informed practitioners of that craft can satisfy the play needs of many different kinds of players. This does happen, even in the battleground of metrics, and developers that are willing to commit to doing so can build a loyal fanbase that supports them, and helps other players to find them. It’s a harder path, to be sure, because it means making commercial artworks that are worthwhile instead of just cranking the sausage machine of rehashed ideas. Nothing good comes without effort. But if we want to walk this path, it entails more than simply resisting the purely metrics-driven concept of commercial games.

Sadly, indie developers who have avoided going down the predatory monetisation path have tended to simply default to making what they like to play and then gambling upon finding an audience for it, which I view as a hugely risky way to pursue a career in game design. I’ve seen dozens (perhaps now hundreds) of developers fail doing this... it’s simply not a good enough plan to trust that – by chance – your play needs will align with enough players to magically make ends meet. As Rami Ismail of Vlambeer suggested to me when I accused him of giving this exact advice:

...I've told developers to make what they want to make - [but] never in that vacuum. My entire existence as a public figure exists because I was one of the very few prolific 2010-generation indies that was yelling about taking business seriously, engaging with publishers and marketing, and doing the work to make your game visible.

21st Century Game Design will be going out-of-print soon; its multinational publisher has withdrawn from publishing books about making games entirely, which in itself says something. Our first book’s core vision – that there are methods for game design, but there is no single, perfect method for game design – remains as true today as it ever did. Our deployment of that vision through a fusion of horizons between psychology of play and the history of videogames remains, I believe, an extremely fruitful way of understanding the craft of game design. Alas, the games industry didn’t choose this path. It choose instead an unholy schism between dogmatic indie design on the one hand, and pragmatic monetisation design on the other. Personally, I feel that the artworks we call games deserve more than this, but I appear to be in the minority. In a games industry divided between a stubborn individuality unable to reliably feed itself, and investment-glutted money farms, there seems little room left for cultivating the craft of satisfying players.

Agree? Disagree? I’d love to hear your comments! Have a blog? Any and all replies at other blogs will be promoted here to keep the conversation going – just let me know the link in the comments or on Twitter.

Before you ask, I’m not playing Pokémon GO, nor do I plan too. I’m a father, a writer, and a business owner – I don’t have time to play an MMO. But it’s interesting to me, since this is another example of a game where the fictional content is far and away the critical factor in its success. My old MUD crowd played Niantic’s previous game, Ingress, and had a lot of fun with it – but anyone who has ever enjoyed any aspect of Pokémon is playing GO, and that’s not just the power of branding – it’s the power of fictional worlds.

Here, in case you missed them, are the best articles so far about Pokémon GO:

• Firstly, do not miss Raph Koster’s piece, AR is an MMO. The title may not grab you, but this is Raph on top form, reiterating a point he and I have both made (and that he made first…) which is that the important thing about both MMOs and AAA console games is that they are worlds – and Niantic need to wise up fast to the consequences of a player community this large sharing one world.
• At the border of paranoia is Omari Akil’s Warning: Pokemon GO is a Death Sentence if you are a Black Man. Thanks to my old school chum, Simon Cox, for drawing my attention to this one.
• Omari’s fears are grounded in the horrific reality of race relations in the US, of course, so it’s perhaps worth reminding everyone that there are more prosaic risks entailed in playing a game on your phone while you walk around a city… Consider the 15-year old girl in Pennsylvania who got hit by a car while allegedly playing the game.
• As for Nintendo, the total value of their stock (their market capitalisation) has gained $11 billion (!) since the game was released…
• …but as Steve Schaefer at Forbes correctly points out, there’s no way the game can deliver income commensurate to that increase in stock value, so Nintendo ought to prepare themselves for the other side of that particular mountain.