Editorial - What's Wrong With Video Game Reviews
An editorial by
Richard Brownell published on 12.1.2004
Boy, that's a negative title, is it not? Video games have been around for quite some time now--in many cases much longer than the people playing them. Video game reviewing has been around almost as long. Since video games became available for home use, people have wanted to know which ones to get. I'd like to name off some of the first publications to have these reviews, but I'm not quite that old nor that knowledgable. But I am old enough to know one thing: there is something seriously wrong with video game reviews. By something, I mean many things and they aren't all the fault of the reviewers. No, in many cases it is the mere concept itself.
But before I get into what is wrong with video game reviews, we should first try to find a definition of what a video game review is or at least what it is for. I know some of you are already typing in definition.com to see if you can 1up me on the definition, but try not to take me too literally there. This is purely conceptual. Thus I present to you my definition: a video game review is meant to help gamers purchase games they will enjoy. To go a bit more in-depth, gamers many times already know about a game before going into a review. But let's assume that they haven't already made their purchasing decision. A reviewer should be able to go beyond the screenshots, the trailers, the spec sheets, the interviews, and tell it like it is. A video game review should be able to show any gamer whether or not they will enjoy that game.
I think we can all agree that this is something a video game review should do. But there is either a fundamental lack of understanding of that concept or a complete disregard in most cases (notice I didn't automatically slap a "mainstream" here. This applies to everybody). Now, let's see what is wrong with reviews. I'm dividing this into subsections, which you may find later is hypocritical. Enjoy the hypocrisy!
scores
For as long as anybody can remember, a review, be it for a movie, game, or what have you, has had a score to go along with it. Sure, some places have eliminated the score. But now more than ever with sites like
GameRankings.com and GameTab.com, scores have been given more and more importance. Scores can certainly be good. They give a quick indication of how positive the review will be and how much the reviewer enjoyed the game. It also gives gamers a chance to compare what a reviewer thought of one game when compared to another they have played.
But scores aren't all fun and games. In fact, scores on the whole are a bad thing. Just think of what makes the difference between a score of a 3 and a 4, or a 5 and 5.5, or a 94% and 95%. Also, looking back at a review, a reviewer is unlikely to find many things in the text that she would change. But perhaps she would think that 6.7 should have been a 6.8.
One of the main problems with video game review scoring systems is having too broad a range. As stated above, you cannot put into words what makes the extra point here or there. It's easy enough to compare one game to another to get a more exact score, but that's still not perfect. One day Final Fantasy X is a 90%. Later, you enjoy Final Fantasy X-2 a bit more and give it a 91%. Oh, but FFVII was definitely better, so it's gotta be a 95%. But FFVI was so much better. Perhaps it should get a 99%. FFXII comes out and it's awesome; 98%. But you didn't really enjoy it as much as FFX, so it's a 89%. But hey, it was definitely a better quality game than FFVII. Oh, the confusion. When perhaps scoring them all 9-10 out of 10 or 4-5 out of 5 would have worked better. Then again, the most accurate score system is the 2-point thumbs up thumbs down system. But that just says whether you liked it or not. Barring people like Hot Ticket that plaud a movie then don't give it a hot ticket, you can get that across in a review.
But scores get
really convoluted when multiple scores are thrown into the mix. In general, these are divided up into gameplay, graphics, sound/music, overall, and perhaps some others here and there. Now, this is all well and good. Technically, it is easier to judge one part of a game. But if you have just convinced somebody to purchase
Grand Theft Auto: San Andreas, do they really care about your low sound rating because you didn't care for the voice acting? Or do they care that you took off graphics points because of low poly counts?
Unfortunately, all manner of scoring problems tend to come together in packages. Since before internet coverage existed, print mags were already dividing their scores into categories and using percent systems. It just looks nice to have so many scores there. But more scores doesn't equal more merit. What it really comes down to is the review text itself. But before I get email over this controversial part of reviewing, I will be the first to say that scores are important and here to stay. In some cases, more thought should be put into consolidating those score systems so that less thought can be put into the scores and more into the review, but the real issue behind scores is that readers need to learn to look past them. If GameReviewerX gives YourFavoriteGameY a score you don't like and you write him hatemail over it, you should know that you suck. No, seriously. Teh suck even. Calm down, learn to respect others' opinions, then use your anger to write your own take on the game. The internet is big. If you didn't like his review, chances are others didn't as well and you can make a club dedicated to your dislike of said review.
One last problem with scores is that many reviewers have completely different concepts of what is poor, fair, average, good, and great. Many places, GAF included, would tell you that a score of 5 is average. That doesn't necessarily mean that the average review scores of any reviewer will be 5, as most below average games don't ever make it to market (most, not all). The problem, however, occurs with places that use something like 6 or 7 as average, or even 7.5 in IGN's case. It looks great to PR people, but means plenty more titles will fall in the 7.5-10 range. However, the important thing here is mainly that the reader keep in mind what the reviewer intends (this is a good reason to you know, read the review). This scoring issue also means sites like GameRankings and GameTab must determine how to rank all these different scoring options together to make averages. These sites also prove the fact that most sites/mags rarely or never give scores below 4/10. Of course, sites like those are largely massive guilty pleasures for those who don't want to read every single review of a game, myself included. They don't change the fact that a reader should find a few game critics that they agree with and see what they think of games they want.
reviewing titles before or just after release
When Mr. PR Guy at CompanyXYZ is working with Mr. Marketing Guy on launching a game's marketing and PR campaigns, one topic that comes up is which print mag (or major web site) gets the exclusive (and much coveted) first review. This first review many times comes a month before the game's release. And it isn't the only one to come early. Most gaming publications review games just before their release. Going back to our definition above, it is imperative to have reviews early, so you can't fault them on that.
What is unfortunate is the circumstances. These are games, not movies. And back in the NES days this would be fine. But we live in a world of unfinished games, patches, upgrades, and online modes. PC games are the main culprit for unfinished games. Why complete bug-testing when there are so many different PC setups out there that half of buyers will have problems anyway? May as well just ship it with plenty of known bugs, resolve them, add in a new armor and a new bad guy, and call it a "Super Sweet Upgrade." The problem is all these bugs are present in the copies that are being reviewed. And though PC reviewers are used to this and don't harp on these bugs, this practice has actually begun slowly but surely in the console world. There have been games where the initial shipment is more buggy than any later shipments. Good practice? Perhaps not, but if that's what it takes to get the game released on time, that's what will happen.
Also on the console gaming front is Xbox Live. Much of Xbox Live exclusive content has been small bug fixes to games. In many cases, these are simply things that weren't found initially. And in other cases, it is the "Duh!" complex like Ninja Gaiden's controllable camera download, or Halo 2's HUD problem on TVs using the component cables and playing in 480p resolution that was recently fixed. But be warned, game industry. Do not take Xbox Live and services like it as an excuse to turn into the PC game industry. Console gamers and reviewers appreciate completed games.
online
Not mentioned in the last section is online modes being difficult to review before release. That's because this is a particularly bad problem for video game reviews. Online modes are quickly becoming a standard feature in games. That's great. However, there's only so much reviewing you can do without actually playing said modes. Yet this sometimes must happen with early releases of games. The regular players simply aren't on the servers yet. PC MMORPGs are the exception here, but in many cases, reviews simply go over what the online modes are or they do not focus on it. Many reviews for Halo 2 didn't say all that much about the Xbox Live modes. This is because a big part of reviewing modes like that is how players play there. SOCOM became full of cheaters exploiting the game, yet you won't see that in any early published reviews.
The main solution for this would seem to be having beta for online modes involving press and a small amount of players. Or at least have press and testers in the online modes. It's certainly a better option than having reviews reiterating a press release or not covering a game's online modes until several weeks into its release.
mmorpgs
MMORPGs are a tough issue. They are inherently different than most other genres. And thus, they deserve to be approached from a different angle than normal games. Because of the longevity involved in MMORPGs, it would be ideal for a reviewer to really spend a good amount of time with the game. Not necessarily a lot per day, but a little for several weeks. Unfortunately, this ideal would either involve reviewing a game based solely on beta or publishing the review after the game has come out. Unfortunately for reviewers and those wanting to know which MMORPGs to play, there really is no ideal.
The MMORPG community already has their solution in place though. Be a beta tester or pay attention to what the beta testers say. All the information gamers need to make their decision is available in that manner. It basically makes reviewing MMORPGs pointless and indeed, several MMORPGs and their expansions have not gotten much press coverage. The major ones like Everquest II and the upcoming World of Warcraft and Middle Earth Online games obviously get plenty of coverage, but lesser known MMORPGs do not get the same. A quick check of GameRankings.com showings some with only 5-10 reviews, much less than most game releases. So as much as it pains me as a game reviewer myself, reviewing MMORPGs simply isn't the best way to give them coverage.
game series: sequels, prequels, side stories
It is hard to say what is wrong with reviewing these types of games in particular. This is more of a topic for debate, though if a definitive conclusion was reached, then reviews that have something "wrong" with them in this case would be known. The debate has two distinct sides, though many reviewers try to find a healthy medium. The first side says that a sequel (prequel, etc.) should be judged in direct comparison to the original game. The second says that all games should be judged as individual works.
Unlike most topics in this editorial, there really is no clearcut answer here. Taking the first approach leads to two problems. Judging a sequel to a killer app like The Legend of Zelda: Ocarina of Time for N64 simply based on comparison can lead to a much lower score than the original. Also (though this would only really happen in theory, not in real life land), if say a new Army Men game were to come out and actually be good, it could [theoretically] get a much higher score than it deserves. These are both with strict adherence to approach one.
Approach two, though as fair as possible, has a completely different problem. As part of a series, it is just silly to ignore past installments. If a game doesn't get improved, it really does deserve a whack. So, again, this game series review problem isn't exactly clearcut. The main problem reviews are the ones taking the first approach, but the interesting thing is that the problem is much more prevelant in how readers rate games than how game critics do. On GAF and other places like GameRankings.com, sequels like Zelda: Wind Waker and Halo 2 get a number of [user voted[/b] scores of 1. Of course, this is just an offshoot of the type of sandbagging going on at IMDB.com where disgruntled Tolkien fans give Jackson's LotR movies low scores. You showed him!
novelty games
Novelty games include things like light gun games, dance games, or basically anything requiring a special peripheral to be attached to your console/pc. These unfortunate fan favorites always tend to get low scores. Phrases usually tossed around include "lack of innovation", "nothing new here", and "didn't I play this in 1991?" Reviewers would rather focus on the fact that there is no special new gaming innovation present in these titles except the first time a particular peripheral makes it onto the market.
EyeToy: Play got decent scores when it came out, but I'd be willing to bet the newly announced
EyeToy: Play 2 will get lower scores regardless of quality. What it comes down to is that novelty games are hit with the same problems as sequels do, regardless of whether or not they are actually sequels.
hype/buzz
For the sake of this argument, let's make a distinct difference between hype and buzz. In practice, they are generally used interchangeably. But simply for this discussion, hype shall be the excitement generated directly by marketing campaigns and buzz shall be the excitement generated by gamers. Each has very real effects on how games are reviewed.
Hype is fairly easy for companies to generate. It's definitely desirable. Put numerous screenshots out there. Slowly release information on gameplay. When the release gets closer, put out demos (perhaps at E3) and trailers. The people experiencing this marketing blitz are directly affectly by this for better or worse. Unfortunately, the people that are hit hardest by this are game reviewers. Reviewers work for the media outlets that all this information goes through before it gets to gamers. It can hard to be objective when Halo 2 has been in your brain for three years, but you hadn't heard of Ribbit King until you passed it (and probably didn't play it) at Bandai's booth this year. Not that those games have anything at all to do with each other...
Suffice it to say, hype basically poisons the review pool. Reviewers (or sometimes their bosses/editor in chiefs) can consciously or subconciously feel obligated to the people hyped about a game and the company doing the hyping to give it a good score. I think we can all remember seeing countless reviews of major titles that say something like "Overall, GameX didn't bring anything new to the table. It's got slow load times, the story isn't that great, and the music leaves something to be desired. But it is an excellent title with solid gameplay. 9.5/10"
But that isn't all that happens during a game's marketing campaign. What is even more desirable than a quality hyping is buzz. Again, buzz for this discussion is the excitement generated by gamers. This is most common for imports and niche titles due to them largely not having any hype-generating marketing machine. Disgaea and La Pucelle succeeded because of buzz. Games that publishers didn't expect to do so well, like Prince of Persia: The Sands of Time, for example, did very well due to buzz. It's no secret that publishers have begun doing covert PR campaigns through small fan sites, message boards, email newsletters, and the like. These get people talking, sometimes much more than releasing screens and trailers.
The real importance of buzz is that it doesn't seem to have the same effect as hype on video game reviews. A reviewer can't feel any impact from something they aren't involved in. If 50,000 people that like games and anime are really excited about the new Cowboy Bebop PS2 game and are busy telling all their friends they simply MUST go buy it, it really doesn't affect the reviewers here at GAF or at any other site. And despite the massive marketing campaigns of games like Halo 2 and Grand Theft Auto: San Andreas, we're still seeing a lot more practices (even in those games) of developer diaries and the above-mentioned buzz tactics by companies. This is good, and with all these high-calibur releases this year, I suspect many games are doing well based on the buzz generated by gamers themselves. Gamers like to have power and I think many sometimes forget just how much power word of mouth really is.
review length
One issue that gets brought up all the time is the length of the review. This is brought up by readers, other video game journalists, and even my own staff. And it comes up all too often. The general consensus seems to be that a longer review is a better review. Length must mean that the author knew more about the game. It must also mean that the reader will now get a better idea of how good a game is. Of course neither of those is true. Neither longer nor shorter is better for a review.
The problem with reviews being too long is that honestly, who really wants to read all that? And it seems the games people need the least amount of information on such as major releases like
Halo 2 get the longest reviews. It simply isn't necessary. You can easily venture into the realm of too much information. A review can certainly be too short as well. The obvious cases are some of the "reviews" that appear in magazines at 200-300 words. A few choice sentences are really not enough to give a good impression of how good a game is. They do come with a score...but see above for more information on that.
What I try to get across to my staff, though they and myself as well may not always follow this, is that a review should be as long as it needs to be. An action puzzle game can only be detailed in so many ways.
Tetris can be explained and reviewed in two paragraphs even though it provides endless hours of entertainment. A strategy RPG, on the other hand, may need a couple pages simply to describe how the game works. The more complicated the game, the more difficult it is to review. The "longer is better" crowd would simply try to cover everything, as part of this attitude is really just to prove that the author indeed knows everything about the game. It can be very difficult to say "enough" about a game in a review to get across the game concepts without boring your reader half to death. It's hard enough to write well to appeal to an audience ranging from 12-year-olds to however old people play video games now, but to hold all their interests is nigh impossible. It is for these reasons that I say being concise is very important. Of course, anybody who has read this far into this editorial, or any of Sam's reviews/editorials, probably cares little for conciseness.
review organization
Many magazines and web sites choose to divide their reviews into sections, which are usually something like gameplay, graphics, sound, and overall, occasionally having things like story, atmosphere, replayability, or packaging. This seems to be a great organizational technique at first glance, but it really doesn't help a review. What it really does is ease the workload of the reviewer. The reviewer can then write a paragraph on graphics and move on with no thoughts on keeping the review cohesive, writing transitions, or any other such intricacies. But, the sections don't provide anything for the readers. There aren't many readers who look at a review so that they can scroll down to the replayability section easily.
And what is tragic is that the ease of reviewing due to these sections, something that seems to force a reviewer to cover everything in a game, can actually hinder the review. Simply giving a paragraph or two to each section can cause a reviewer to miss out on what a game is really about. A review for a game like
Rez divided into sections can give a dull account of its gameplay, graphics, and sound without describing that each of those aspects works in harmony to produce an innovative and unique game.
After all that negativity, it may seem like I am out to get everybody else doing video game reviews. This is not so. I do review them myself, after all. As stated, many things, such as reviewing MMORPGs and reviewing games based on an early build, are not easily remedied and apply to everybody. Unfortunately, there are some things that will always be troublesome. But we as reviewers have a responsibility to our readers. It isn't to cover every single thing in a game or to give it the exact score down to the 10th percentile. It isn't for our reviews to live up to the hype or to prove the reviewer's ability as a gamer. It is to give a true account of our experience with the game in a concise and helpful manner to those that want to know about the games.
