Last week I bought WET because it looked like fun and I wanted to play the game. After playing half way through the game I found it didn't only look like fun, but it delivered on its promise. It was what I wanted Stranglehold to be and what Gungrave never delivered. Lot's of mindless, shooting fun. Sometimes that's just what I want in a game. But apparently, most critics did not share my view. When I first looked on metacritic the score was in the deadly sixties, but has since moved up into the safe haven of mediocrity found in the seventies. Not quite green banded goodness, but not bearing the red mark of humiliation. I wish metacritic didn't matter, but unfortunately, developers' livelihood is based on this hopelessly useless, conflicted, arbitrary measurement system even though more and more and more people are realizing marketing and word of mouth are more significant factors in the purchasing decision than a Metacritic score. If you really want to compare apples to apples in admittedly anecdotal but still compelling example, Rock Band 2 for the 360 scored a 92 to Guitar Hero World Tour's 85 last year. But Guitar Hero, with a 40% larger marketing spend, outsold Rock Band 2, the placebo, by a wide margin. Publishers still use these numbers as gating to signing developers and not only are they useless, they are shoddily calculated numbers based on arbitrarily assembled numbers.
Out of curiosity, I decided to see who was responsible for raising the score and what they had to say about the game. They were courageously disagreeing with gaming stalwarts like IGN at 66, Gametrailers at 63 and sponsor and influence free Giant Bomb at 60. It looked like it was Armchairempire.com. I say looked like because they were noted as giving the highest score the game:
But when I clicked through to the review, I saw they didn't:
Now, I am sure this is a careless error, but how dare you be so careless when developers' livelihoods are at stake. Didn't anyone at metacritic feel the need to confirm the numbers posted on the site, or is the move from 75 to 88, part of metacritic's "weighted average" calculation, described as:
The METASCORE is considered a weighted average because we assign more significance, or weight, to some critics and publications than we do to others, based on the overall stature and quality of those critics and publications. In addition, for music and movies, we also normalize the resulting scores (akin to "grading on a curve" in college), which prevents scores from clumping together.
Metacritic does acknowledge scores are misreported and suggests a solution:
Q: Hey, I AM Manohla Dargis, and you said I gave the movie an 80, when really I gave it a 90. What gives?
A: Now, if you are indeed the critic who wrote the review, and disagree with one of our scores, please let us know and we'll change it.
This does happen from time to time, and many of the critics included on this site (such as Ms. Dargis) do indeed check their reviews (as well as those of their colleagues) on metacritic.com.
Are you serious? You can't be bothered to confirm you are accurately transcribing numbers and it is up to the critics to fact check?
But shoddy journalism is not my only concern. It is the sites' holding itself out as objective when it is really a conflict laden subjective aggregation of a limited set of already subjective market data. Now, of course the fact that CBS owns of Metacritic and Les Moonves, President and CEO of CBS Corporation is on the board of Zenimax, parent company of WET's publisher wouldn't influence metacritic to hunt for some favorable reviews and maybe fudge some of the weighting or even a number, but a purportedly objective site should not be in a position where it must explain why not. When I was in law school, they taught us to avoid impropriety, but the appearance of impropriety. Avoidance is simple. You disclose. In this regard, I renew my suggestion from an earlier post and I offer a disclaimer:
We are affiliated with a studio, record company, television networks and game companies, in fact we are better connected in entertainment than CAA and WME combined. We can probably get Les Moonves on a conference call. To give you a better idea, here is a partial list of our family members:
- CBS Television
- CBS Records
- MTV Games
- Paramount Pictures
- Paramount Television
- Paramount Digital Entertainment
- Dreamworks Animation
and of course our distant cousin, Bethesda Softworks. We do our best to avoid influence from our parent and siblings, but the significant subjective component in our scores makes it kind of hard.
This has been a public service message. Thank you and goodnight.
Oh, and by the way, pick up WET. It really is a good game.