Login to access exclusive gaming content, win competition prizes
and post on our forums. Don't have an account? Create one now!
Why should you join?
Click here for full benefits!
GamesForumsCheatsOut Now
Call of Juarez 2: First trailer | Ten new Damnation screens | Free CoD: World at War map hits XBL | LotR: Conquest DLC released | Punch Out dated in huge Nintendo update | Lips gets Coldplay, improved scoring | Resistance PSP MP mode detailed, video | Worms Armageddon Decade confirmed | Ninja Blade out April 3 | Universal joins XBL Video Marketplace | Microsoft shares drop to 11-year low | Killzone content bound for LittleBigPlanet | Analyst: iPod Touch to kill off PSP | Prince of Persia DLC delayed | Xbox Live bans "offensive" lesbian | New Wii controller: "No plans" for Europe | LittleBigPlanet PSP won't get in the way of LBP PS3 | Perry backs up PSP2 claims | 360 tops Wii, PS3 in Japan weekly sales | MadWorld: New Trailer | Three new DSi colours for Japan | Activision reveals Bakugan game | Sega serves Stormrise trailer | Street Fighter IV too difficult - HD Remix man | CV reveals Aliens RPG is canned
All|PC|PlayStation|Xbox|Nintendo|Games on Demand
Search CVG
Computer And Video Games - The latest gaming news, reviews, previews & movies
CVG Home » News
PreviousKing Diamond for GH: Metallica Blu-ray sales to top 100M in '09  Next

Why Metacritic doesn't matter

Do review scores affect sales?
There is a prevailing theory that review scores affect sales. It is certainly true that there is a high correlation between aggregate review ratings, like those found on Metacritic.com, and sales; games rated over 90 percent tend to record higher game sales than their lower scoring peers. The question that was posed to the developer panel at DICE Summit 2009 was whether they designed games with reviewers in mind. The answer was an overwhelming no.

"The first person you have to design for is yourself," says Danny Bilson, creative director at THQ. He acknowledges that Metacritic is a powerful force. He's found that he can game Metacritic by improving what he calls "linear assets" that don't affect the actual code, which the team spends most of their time writing. By that he means offering up a better story, better voice acting, and better music. Reviewers have to write, says Bilson, so give them something to write about. By paying attention to such things, you can gain a five to ten point bump because in Bilson's eyes game reviewers care more about story than consumers.

The consensus among panelists was review scores in and of themselves don't matter. Fingers pointed to the success of games like Wii Fit that scored in the 60 percent range. Reviews of such games just aren't relevant to their audience, says Rich Hilleman, chief creative director at Electronic Arts. Social gamers are not the people who are going to be reading hardcore game magazines.

No, the issue isn't the impact of the score on sales, it is how your company uses them, says THQ's Bilson. They are either a touchstone or they are a weapon. Companies use Metacritic scores to project game sales. When teams pitch a game, they're asked to estimate the score. Why would a development team ever shoot for anything less than 100 percent, he asks. The problem is that these projected scores determine game budgets and marketing spends.

Metacritic scores are trailing indicators. Chris Taylor, chief executive of Gas Powered Games, doesn't need a Metacritic score. He knows if he put out a "shitty game." Developers get that information from retail, he says. If it sells well it was good, if it didn't it was bad.

The worst part about these reviews, says Mona Hamilton, SVP of marketing at Capcom, is they are suggestive--and their scores often depend on how drunk you got the media at a Street Fighter event.

Another problem with Metacritic scores, says Bilson, is they aren't weighted. Reviews from Time Magazine are treated the same as some kid's blog. A low score from a "couple weird sites" can significantly drag down average review scores. [Metacritic's website actually notes that scores are weighted. Thanks to readers for pointing this out.]

Hilleman suggests that game developers don't need Metacritic at all. With telemetric data, they can see what is wrong with all of their products--the answers are in the data. Community, he says, is the most important thing.

Besides, says Bilson, "How can you put a number on art?"

Article supplied by Edge-Online

computerandvideogames.com
// Interactive
Share this article:  
Digg.comFacebookGoogle BookmarksN4GGamerblips
del.icio.usRedditSlashdot.orgStumbleUpon
 
Posted by Black Mantis
Doesn't matter to me. I just use it as a quick link to online reviews so I can read the finer details.
Posted by nottsville
I couldn't give a toss about scores. I've had a great amount of fun from games that have only achieved a 6/10 score, and downright hated some games that got 9/10-10/10!

Although Suiv lives and dies by metacritic, so it matters to some people!
Posted by wildhook2
I lost the faith in scores when they gave Halo 3 a ten.
Posted by morgadp
A couple of years back, Edge released an issue where they didn't put scores on their reviews as an experiment (though they did place them as a footnote at the back of the magazine). It was really interesting reading the text of the review without the context of a final score and, I think, actually gave a more rounded picture of the game. It's a shame that they did not abolish review scores altogether, though I can imagine that they were under pressure.
What does interest me in this article is the blatant hypocrisy on display. These publishers and developers say that review scores do not matter, and yet they are the first ones to boast a high score on the box in order to boost sales. Publishers have a long and documented history of "buying" good review scores - have we forgotten the whole Kane and Lynch debacle? So for these people, who clearly treasure a high score, to then say that they don't matter, makes me a bit angry.
The attitude is also slightly worrying - they cite Wii Fit, with its average review scores but high sales. They say that a good story and high production values would increase the score but not affect sales. It seems almost as though they are saying "it's ok to make a lazy game as long as it sells."
In retaliation to the final comment, I would say that if they put their words in the article into practice then what they will be producing will not be art at all.
Posted by The_KFD_Case
Besides, says Bilson, "How can you put a number on art?"


Indeed, and how can you put a sales number on whether a game is truly good or bad? Doesn't seem to stop you and others though, eh Mr. Bilson? Gotta love hypocrisy.

I can appreciate that from a business perspective the company exists to make money and thus a "good" product is one that sells well. However, I like to think that a game with a good story and interaction and a well selling game are not mutually exclusive.

P.S. I'm in the same boat as you, Black Mantis, where it pertains to Metacritic. I find it to be a useful way to gain quick access to reviews of games I might not otherwise register on my "gamedar" (yes, made up word). :P
Posted by roland82
I wanna know how you can call '50 cent blood on the sand' and 'Bratz babies' art?
Posted by roland82
Sorry, hate to double post but we all know the best use for metacritic is to wind up fanboys when 'their' game gets low score.

I think most informed gamers make their own minds up either by trying the game or just taking a chance on the games they really want.
Posted by johnway
Metacritic is flawed in some ways. Some games get more review scores then others, some scores don't translate well into metacritic ones. Games reviewed out of 5 or on a single numerical value (as to decimals or percentages) will always provide more disappointing scores.

Some reviews i personally find come from sources i wouldn't call reliable or worthy. Others i find are naturally strict and will always give low scores.

But then, the only persons opinion and score that only matters is yourself. Who cares if everyone hated it. If you enjoyed it, its a success and a good game. Sure you might get a rough idea as to what everyones views are and you would be more interested in whats so bad about a game by reading lots and lots of reviews to get a bigger picture. But most of these reviews are still opinions and should be taken with an open mind.
Posted by peteuplink
I don't think reviews have as much of an impact on games sales as they used to in the mid 80's to early 90's.
Posted by lonewolf2002
You can keep your scores from reviewers as posted before you should know if you are going to like a game or not regardless of score. If you live and die by what score your game gets then I suggest getting a life.

As for metecritic I do occasionly visit there to check out the reviews but I pick the top review, one from around average and the bottom to get a feel from people who loved the game, indifferent and really didn't like it. Although this really wouldn't stop me getting a game I wanted (like all the 360 JRPG's for example).
Posted by TezChi
I lost faith in reviews a long time ago now, it just they never truly reflect a games quality anymore, but rather its mainstream appeal or amount of $$$ pumped into advertising etc (as mentioned in the article, review scores are suggestive).

I think as someone said above, that reviews are not taken as seriously as they were in the 90's, where i can distinctly remember agreeing with and reading reviews for info (and of course, the good old paper CVG mag! ^_^).

But these days reviews have become convulted, payed out or just plainly biased opinion peices aimed at a target audience.

THE only way for a serious gamer to find out about which games they will like these days is to do the following: Play Demos, watch trailers, read interviews, look at the company, developer and name and etc.

If I was to buy games that got the highest scores all the time, Id keep ending up with some pretty bad titles! :lol: :lol:
Posted by The_KFD_Case
That, and it's been a number of years since the 90's which may very well have seen you (and others - myself included,) develop and change in terms of how we perceive ourselves, what we value and appreciate along with what we don't care for. In other words: We are the same creature yet different from what we were back then and this is now reflected in our tastes (including how we view reviews). :)
Posted by pishers
scores dont really matter, its the text of the review. metacritic is handy to find a number of differing views because i find it hard to trust publications these days. however, i usually know if i am going to get a game before the reviews land unless it reviews particularly badly. i tend to go for games from devs i trust, in a genre i like with an interesting premise.
Posted by Railgun_Sniper
Galleries 'put a number on Art' every day.
Posted by Dajmin
Demos, that's the way forward.
Metacritic is a good way of judging something if you also read the review synopses there. I read the top few and the bottom few just to see what they think the best and worst features are. Sometimes they work, sometimes they don't (or I don't know). Sometimes the demo will be good despite bad scores or reviews.

A good demo will sell a game better than a bad review, so a developer has nobody to blame but themselves.
Posted by ParmaViolet
It's quite interesting how demos have had such an impact on what developers do these days. I read an article that discussed how they kept in mind what they would make available for the demo while they were creating the game. In some cases, this could mean a level specifically designed in a way to make it a viable option for release on Live or PSN.

Obviously, there are some excellent examples of how a demo can be - Crackdown is perfect - but, some games seem to suffer because of them. Dead Space seems to be one that people highlight as not a true representation of the full game.

I guess it's kind of 'hit & miss'....still, I think they're actually a great way of getting people interested in games that they wouldn't usually bother with.

In terms of Metacritic - I'm not big on review scores, as such....I'm far more into actually reading the review. Plus, when you look at their music section, isn't Brian Wilsons' 'Smile' the highest rated album?......I mean, c'mon - that in itself puts it all in perspective.

:)
Posted by _Marty_
They seem to be a double edged sword to be honest.
Halo Wars, for example, I wouldn't have looked twice at in a shop. However, I figure I'd give the demo a go (cos, why not?) and I absolutely loved it. It's now pre-ordered - and I've had confirmation of it shipping too. Woot.
Resi 5 however, a great many people were massively excited about this game, but since playing the demo, I reckon their pre-orders got slashed in half.

A good demo can do wonders for a game, a poor demo can utterly ruin it.
Posted by ParmaViolet
»

Absolutely - some are actually just awful.

I think the problem is what you do for the demo - how many times has a friend said 'it's a good game - a little slow to start with, but it gets a lot better'?...so, it's not necessarily the best idea to make the first level the demo, but you can hardly put a level up that contains spoilers.

:)
Posted by Crow555
This is precisely my problem with Metacritic and why I think Gamerankings is better. If some nobody with a chip on their shoulder can get their blog review added to metacritic and bring down a game's average score then, it's worthless.

Having said that, I take review scores with a pinch of salt (OPS2M's 10/10 for MGS2 was the last straw for me) and read reviews to confirm or remove any doubts I have about a game.
Posted by _Marty_
Agreed - but then that is ABSOLUTELY down to the game too. I believe Dead Space demo would have been MUCH better had it started from the beginning. The crash, being cut off, all hell breaking loose and you not even having a weapon yet. Such a better 'hook' for the demo.

It's a tough call, but one that should have a LOT of thought put into it as this, far more than reviews, can make or break a games sales.
Posted by Dajmin
Yes, I have to say that the Dead Space demo was utterly crap. It wasn't one of the games on my "want" list anyway (yes you can call me a heretic) but I had read enough reviews and stuff to get that.
The DS demo lost all horror pacing and just went for all-out swarms of death - not a great idea when you're still learning the controls!

But for the most part demos are a good idea, as long as they are a true representation of the actual game. I'm not sure I agree with the idea of actually designing a demo since it seems like you could just stick in all the best parts to try and sell it. But going "here is level 1, if you like it then you'll like the full game" is what it should all be about.
Posted by _Marty_
I like the theory, but how many games have you played that have had a slow start?
Take Shenmue for example, after the intro, walking round asking EVERYONE if they 'remembered the day it rained' and 'did you see a black car that almost ran over that cat' was hardly the pinacle of excitement.
That was just the tip of the iceberg though, and there was a lot more to it than that.

I'm with you on not making a demo, it should be a true representation of the game, for sure. However, significant thought should be put into what part of the game the demo highlights. Some games it would be enough to just have the tutorial or the first level. Others, there just isn't enough substance to them to do a demo justice.

Also, try Dead Space :)
Posted by nb_nmare2
Some of these developers are completely out of touch with reality. Let's leave the business side of things to the publishers, eh guys?

Take Chris Taylor's comment, for example. There are countless awesome games which had piss-poor sales (e.g. pretty much anything made by Looking Glass Studios, like Thief and System Shock), and probably even more sub-par or even terrible games that have sold by the bucketload (cough *Halo* cough).

My suggestion for making a good game: hire playtesters who not only look for bugs, but who are also willing and able to actually comment on how good/bad particular aspects of the game air. Preferrably these guys will be employed by a 3rd party, so they won't be worried about getting fired by a developer pissed off by someone telling him "sorry dude, your game blows".

This method would have worked wonders for Mr. Taylor's own Space Siege, methinks.
Posted by Dragonahcas
Another game I can think of with a slow start but a good game in general is The Witcher.
The start of the game is wrist slittingly slow, but the game gets a lot better as your progress.
Posted by bunneyo
cum on for christ sake, reviews have been flawed since the 60's!
jus look at garbage hollywood gives us, i swear watch the tv for an hour and you'll get some bollox film trailer which at the end will say...

exciting, intense, edge of your seat, 3 outa 4 stars, 4 outta 4 stars!

then you go and see it and come out at the end and think....wooah biggest pile of shiiiit i've seen in a long time! its all about selling product and making money, all reviewers get paid or given something to up or lower the score of a product! people stop being so blind about it
Posted by jon_hill987
I use MC, I look at the score but I then read at least one of the top reviews and at least one of the lowest reviews.

Gives me a good idea of the game.
Posted by vule
the scores are great for describeing quality of game
but taste is a personal thing for everyone for themselves
Read all 27 commentsPost a Comment
// Popular Now
News | Reviews | Previews | Features | Interviews | Cheats | Hardware | Forums | Competitions | Blogs
Top Games: Tomb Raider: Underworld | Metal Gear Solid 4 | Grand Theft Auto IV | Grand Theft Auto IV | Call of Duty 4: Modern Warfare | LittleBigPlanet
Burnout Paradise | Unreal Tournament III | Halo 3 | Xbox 360 Elite | Bioshock
Top Reviews: Silent Hill: Homecoming | Dead Rising: Chop Till You Drop | Dragon Quest V: Hand of the Heavenly Bride | Warhammer 40,000: Dawn of War II | Halo Wars | 50 Cent: Blood on the Sand
Sega Mega Drive Ultimate Collection | Race Pro | GTA IV: The Lost and Damned | Street Fighter IV | Bomberman II
Copyright 2006 - 2009 Future Publishing Limited,
Beauford Court, 30 Monmouth Street, Bath, UK BA1 2BW
England and Wales company registration number 2008885