Evaluating Video Game and Movie Grading: Two Differing Schools of Thought

My criticisms of the Empire Magazine's 50 Worst Movies Ever list made me think about a few things over the weekend, and I reread an earlier post I did about game reviewing, and I think I've come up with a few new thoughts on the matter.

Now I don't really trust singular reviews of movies or video games, but I have always appreciated sites like Metacritic or Rotten Tomatoes which provide an aggregate score based on a relatively large sample size, as that generally removes the effect of outliers and gives me a sense of how good a piece of media as judged by a wide variety of perspectives.

In looking at sites like those above, I've definitely noticed something interesting between the scores that movies get and the ones that video games receive.

With movies, even if the aggregate scores are low, there is still often something worth watching in the ones that have scores down into the lower end of the scale. They aren't fantastic efforts, but often times, there may be something redeeming in them that still makes them entertaining, and not in a cult sense. For the most part, they are judged based on story and character alone, and the technical aspects don't come up unless they are brilliant or noticeably awful.

If I am being honest, there are movies that I've watched on the Metacritic's 200 Lowest Scored movies list that I thought weren't bad. I mean, I have to admit that I've seen Biodome at least 10 times in my life and that I actually liked Half Baked, and I don't care that it has a bad overall score (and I will touch on that later).

Now contrast that with the scoring of games, and an entirely different pattern emerges.

Basically, if the title's aggregate score is below 60, well, then you are entering the territory of it being an unplayable game, something which either has really noticeable and crippling technical or design flaws, or lacks anything good or original in terms of gameplay, sound, graphics etc. Such scoring makes deciding which games are worth your time a lot easier than the comparable ratings for films because in most cases, a film with the same score is a much better product.

In fact, video game ratings have strong parallels to the academic system where there is a crisp demarcation between passing and failing, with a few outliers (in my opinion) here and there. You have your A through D graded games (100-60%) and then all those games that got an F (59% and lower). Some would argue that the standard should be 50%, but even in the low 60's, games are noticeably mediocre.

I took it upon myself to breakdown the ratings for the Playstation 2 games listed at Metacritic, because it is a relatively large sample size with over 1700 titles, and they are presented in such a way that it makes this analysis much easier.



It is a standard bell curve shifted over so that the majority of games listed fall within the "pass" side of things, which means that all other factors being equal, they are technically playable and conform to the standards of that console generation.

While video game scores may seem to be artificially high when compared with the scores given to other forms of media, every individual title has to overcome the additional hurdle that the technical side of things presents. Movies generally don't stop you from experiencing them if they have bad camera work or poor production values, which means that someone can make a movie that while horrific, it can be experienced in full by billions of people in theaters, with no problems.

That brings up another interesting point. I tried and failed to find a bell graph for movie ratings at Metacritic, even for a single year, but in the midst of that search, I did discover an interesting article on Gamasutra which introduced evidence that as game scores rise, so do revenues for individual titles at the higher end of the scale, and conversely, titles with poorer ratings tend to under perform greatly as well.

All things being equal, this is exactly the kind of thing you would expect, that quality would be rewarded. However, looking at the Metacritic/Rotten Tomatoes ratings for the movies that succeed and fail at the box office, an entirely different pattern is evident. We've all seen time and again, critically panned movies making enormous profits while critically praised movies are time and time again box office failures. The fact that this disparity even extends into the realm of DVD sales and rentals shows that it isn't an issue that involves people giving a bad movie a great opening weekend before the negative reviews come in and tank it. It seems that the majority of the movie going public just doesn't care if something received bad reviews or not, and I have to say, I am guilty of this too, so in essence, I am supporting bad cinema through my spending habits. But with games, I avoid poorly rated games like the plague. I've bought a few recently at the spur of the moment out of ignorance, but usually, I seek out quality work, and 95+ percent of my collection is in that range.

I think the scary thing about that Gamasutra article was the sales figures for Wii games. The fact that casual gamers are buying a lot of shovelware for the system (as shown through the better than average sales for games in the 50's and 60's scoring range) is a distressing trend, as it is telling developers that they don't have to try as hard to make quality products, and in the end, that is bad everyone who plays games and it does little to help interactive entertainment grow as an art form.

Tags: gamasutra, video games, playstation 2, playstation 3, xbox 360, half baked, biodome, pass/fail, ratings, score, metacritic, rotten tomatoes, empire magazine, 50 worst movies ever, shovelware, dvd sales, box office, quality sells, academic grading