Metacritic vs. Modern Warfare 3: The Psychology of Fanboys

There's certainly nothing wrong with disliking Modern Warfare 3. There's nothing wrong with disliking anything.

There's a vast difference, however, between constructive criticism and what's going on over at Metacritic right now.

Metacritic, if you're not aware, is a website that collates and averages review scores for a variety of entertainment products, including games. Over the years the site has become a hugely influential component of the video game industry. The stocks of video game publishers can rise and fall significantly based on Metacritic scores. Earlier this year investors were spooked by Homefront's 72 out of 100 average on Metacritic; THQ's stock sank by 21 per cent on the first day of the game's release, well before any sales data was available. Some companies have even been reported using Metacritic scores as indicators for additional rewards for developers, working a desired 'Metascore' into publishing deals.

Metacritic differs from sites like GameRankings, however, by featuring a user score. User scores are not factored into the critic Metascore, but they sit alongside it.

The user score system, however, is flawed. The user score, out of a possible ten points, for Modern Warfare 3 at the time of writing is 3.2 on Xbox 360, 2.8 on PS3 and 2.0 on PC. The critical averages (out of 100) for the game are 89, 88 and 81 respectively.

What's of note here isn't that Modern Warfare 3 has attracted negativity. Gamers are permitted to dislike any game you put in front of them, Modern Warfare 3 included. What's noteworthy is just how many gamers are so turned-off by it that they are compelled to give it zero out of ten.

Dr. Daniel King is a research associate at the University of Adelaide's School of Psychology, in South Australia. King is happy to help us examine exactly what's going on here. He took a few moments to read some of the community reviews of Modern Warfare 3 on Metacritic.

"So many zeros!" says King. He's not wrong.

"My skim read of the content of these reviews suggests to me that the number one complaint is the perceived lack of innovation and new content from the previous iteration of the series," he continues. "People aren't necessarily saying the game itself is a 'zero' as an overall package, but are expressing a very strong dislike of the series' overall direction as a yearly series that makes relatively small incremental improvements and changes. I'm a gamer, I know where they're coming from. You expect value for your gaming dollar, especially in Australia – these reviews suggest that there is at least a significant minority of players who feel that the Call of Duty franchise is no longer delivering along those long-held gaming values of originality, innovation, what-have-you."

Many Metacritic users have turned a ten-point scale into a two-point scale; "Loved it! 10!" and "Hated it! Zero!" There's no pressure on users to think about assigning numerical scores in context. Fairness, balance and credibility are shot out of the sky when those upset with one flaw in a title rate it zero out of ten. There's no perspective to these sorts of knee-jerk scores and there's very little analysis. Does a zero really help your fellow consumers? If Modern Warfare 3 is a zero, what is Superman 64? Worse? How? It's not objective criticism; it's a protest.


"I think it's pretty fair to say a lot of the user reviews are rants," says King. "Many user reviews lack obvious criteria and systematic reviewing. Many reviews are typically, 'I love this game' or, 'I hate this game and I hate the corporate greed of Activision'. They are framed in terms of the emotional experience of playing the game, be it in awe of the action and spectacle or feeling ripped off for buying the game despite it seeming similar to past games."

"The emotionally framed reviews are also generally dichotomous in their scoring, a zero or ten, because it's hard to put numbers to our feelings. It's not a finely-grained process for many people. We either feel angry or excited, or we don't. That's not saying there aren't some really good reviews on Metacritic, because there are, it's just that they get lost in the sea of dissenting voices.

"I also suspect not everyone wants objective reviews of video games. Emotionally attached people will gravitate to emotional arguments. It's harder for these people to relate to the more objective analysis of a video game, and actually feel turned off or bored by it."


King has actually recently noticed something similar in his current experiment into how video gaming affects sleep onset latency and sleep quality. As part of the study they have had their participants rate the video game they played along emotional indicators of 'exciting', 'boring', and 'fun', among others. They then rate the game with an overall score out of ten.

"It's not a major part of the study, we are mainly interested in the sleep aspect, but it's been interesting to see how breaking down and quantifying the emotional experience of playing the video game has granted insights into how people evaluate games," says King. "For example, some participants will finish playing the game and say, 'This game sucks'. They'll rant about some part of the game where they got stuck or thought the enemies were unfairly difficult."

"Then you give them the survey where they can systematically rate the game on factors like fun, excitement and boredom and they really have to stop and think about it. We see greater variation in scores – like 7 on fun, and 9 on frustrating – and the overall 'score' based on emotion, 'it sucks', doesn't reflect the more systematic evaluation. Having to think in more focussed way seems to give a more balanced overall view rather than give a knee-jerk, emotion-based judgement. Metacritic could introduce criteria for reviewers, but I expect that agreement on what criteria to use would be impossible by democratic vote and would take a lot of fun out of the community."

Metacritic tightened restrictions to cut down on 'review bombing' in August 2010 by requiring users to make an account (prior to this you just needed a unique email address) but the system is still open to abuse. There are more fundamental issues than simply trolling here, even though the latter is a key factor.