The new forums will be named Coin Return (based on the most recent vote)! You can check on the status and timeline of the transition to the new forums here.
The Guiding Principles and New Rules document is now in effect.

Review Scores? What do you think?

LewiePLewieP Registered User regular
edited August 2008 in Games and Technology
I've been having a discussion with a bunch of guys from various games sites about review scores.


I am of the opinion that review scores are at best pointless, and at worst detrimental to the games industry as a whole. If the body of the review has not successfully communicated the reviewers opinion on the game, adding a number will not suddenly fix that, and if the body of the review has successfully communicated the reviewers opinion, then the number is redundant.

Review scores give big sites more influence over a large section of the games buying public. I also think metacritic is pretty bad.

They can contribute to the homogenization of games, and push them away from being an art form, or at the very least a creative endeavor, towards being a commodity.

They attempt to make something wholly subjective into something entirely objective.

That's just em though, and I am a bit of a hippy.

Here's some other people's opinions:

90% of the people who "read" reviews don't actually read the body of the review, they'll simply look at the score at the end and say oh well, Game Y is Better than Game X - you'll see this all over the internet when fanboys argue about Eurogamer's 8/10 for Halo for example.

You can try and get away from Review scores all you like, but at the end of the day i believe a large majority of people are just not going to read the actual body of the text and would look elsewhere.
Without review scores ? Impossible. It was created (even though it's old) by many people. We can't just change it now with our 20+million figure.

Review scores are the only thing they look at. And even then, if you read our comments, most are just pity flameboy wars. Doesn't really speak great about our audience but I have no control over it.

If it was for me, I'd ban all those idiots and keep the comments very limited to our reviews.
It'd be bad writing if we didn't provide a reader with what they were looking for. It'd be like walking into Tesco only to discover they'd given up selling bread and milk or had decided to hide their chocolate bars inside watermelons.

In simple, non-waffly, terms: it is a totally unsustainable idea. Sure in an ideal world it'd be great. If we're all to get on our high horses and complain that most people probably skip over our hard work to look at a number and express outrage about that and nothing else...

Yes it is perhaps a little depressing when I spend two hours writing a really well structured and critiquing review only for 90% of people to go '7/10. WTF?! This deserves an 8 you mother fucking cock head'. But surely that's what being a journalist is all about, in any way shape or form. Doing your best to inform people, it is up to them what they do with the information you provide. I feel that without a score you're doing people who can't be arsed to read 800 words of text a disservice and any other thinking is just appealing to your journalistic ego.

You're as small as a bug - no matter how important you become as a journalist, the reader (as a collective) is a big shoe waiting to crush you.
I've tried that on my site (reviews without a score), on and off. I found that my stats are better on the reviews with a number score compared to the reviews without. I suppose a bulk of my readers are just looking for a thumbs up or thumbs down. It's kind of disheartening.


What do you guys think?


Also, download this song. It's a 7/10.

LewieP on
«134

Posts

  • XagarathXagarath Registered User regular
    edited August 2008
    I think numbers can be helpful, well-used. When applied on a full scale, and used to reflect the reasoning in the review, they work as a summary that people can hold in their heads to remind them of the rest of it.
    Of course, only a minority do this at all well.

    Xagarath on
  • Desert_Eagle25Desert_Eagle25 Registered User regular
    edited August 2008
    Scores need to be broken down intwo two levels; aesthetic and gameplay. These single-score reviews are BS as it is.

    Desert_Eagle25 on
  • The_ScarabThe_Scarab Registered User regular
    edited August 2008
    I should really post up my analyzing the review score article.

    It ends up with fruit all over the place.

    The_Scarab on
  • AroducAroduc regular
    edited August 2008
    Rule #1 of the world. People don't like to read. You can write an entire treatise explaining every single flaw and innovation in the game, and nobody will read it.

    Aroduc on
  • StericaSterica Yes Registered User, Moderator mod
    edited August 2008
    Scores are too subjective to be of real value, and the problem is people put TOO much value in a game getting a 7 or a 10. At most, the reviewer should post if he or her recommends the reader check out the game in some manner. You're supposed to read the damn review, not just check out the number of purple monkeys they handed out.

    Sterica on
    YL9WnCY.png
  • FaceballMcDougalFaceballMcDougal Registered User regular
    edited August 2008
    This thread gets a 4.7/10

    I like scores for movies, because I don't like to read reviews and opinions and if there's no word of mouth on the latest X-Files movie I want to know, at a glance, if I should see it.

    I don't care about scores for games. Because I'm overly informed already.

    So you can imagine there are people out there that are like me for movies... but with games. There's no harm in reaching both audiences.

    FaceballMcDougal on
    xbl/psn/steam: jabbertrack
  • StericaSterica Yes Registered User, Moderator mod
    edited August 2008
    Aroduc wrote: »
    Rule #1 of the world. People don't like to read. You can write an entire treatise explaining every single flaw and innovation in the game, and nobody will read it.
    You don't have to be overly detailed in the review just because a number isn't present.

    You are correct, however, in that the popularity of scores stems from sloth.

    Sterica on
    YL9WnCY.png
  • LewiePLewieP Registered User regular
    edited August 2008
    Aroduc wrote: »
    Rule #1 of the world. People don't like to read. You can write an entire treatise explaining every single flaw and innovation in the game, and nobody will read it.

    TLDR

    LewieP on
  • AroducAroduc regular
    edited August 2008
    Rorus Raz wrote: »
    Aroduc wrote: »
    Rule #1 of the world. People don't like to read. You can write an entire treatise explaining every single flaw and innovation in the game, and nobody will read it.
    You don't have to be overly detailed in the review just because a number isn't present.

    You are correct, however, in that the popularity of scores stems from sloth.

    The worst of all worlds!

    They're actually at a pretty decent balance right now I think, in terms of amount of text. They could certainly use more brains and wit, but what couldn't?

    Aroduc on
  • SzechuanosaurusSzechuanosaurus Registered User, ClubPA regular
    edited August 2008
    I basically agree with your opinion, Lewie.

    And Aroduc is wrong. I enjoy reading reviews, assuming they are well written. Which a lot of them really aren't. I don't enjoy reading mindless, boring drivel.

    And trying to score elements of a game (such as aesthetics and gameplay) individually is an even worse idea, because it's an irrelevant abstraction of components of a game. Why stop at dividing the score up into aesthetics and gameplay? Why not also sound and script and story and coding and animation and length and number of unique equipable items and traversable landmass in kilometres and number of in-game trees?

    All that really matters is whether or not you're going to enjoy a game. A reviewer can't definitively tell you this with a score or a review. He certainly can't tell you it by simply listing all the features of a game. But he can describe to you his experience of playing the game and from that you might be able to get an idea of whether or not you'll enjoy it.

    Szechuanosaurus on
  • EvilBadmanEvilBadman DO NOT TRUST THIS MAN Registered User regular
    edited August 2008
    I am a fan of the BUY RENT or PASS scoring system, easy and no fuss.

    EvilBadman on
    FyreWulff wrote: »
    I should note that Badman is fucking awesome
    XBL- Evil Badman; Steam- EvilBadman; Twitter - EvilBadman
  • StericaSterica Yes Registered User, Moderator mod
    edited August 2008
    I wonder if the quality of reviews is brought down by the writer's knowledge that, for his hours of work, the only thing most people will give a shit about is a number. That would...kill my motivation a tad.

    Sterica on
    YL9WnCY.png
  • XagarathXagarath Registered User regular
    edited August 2008
    Mind you, I find review scores almost useless, personally. I can just see how some might feel otherwise.

    Xagarath on
  • ZackSchillingZackSchilling Registered User regular
    edited August 2008
    I like the rotten tomatoes approach to score aggregation. You get a whole lot of people, mostly big critics, and you ask them the question: "did you personally like this?". Then you present a percentage of the people that liked it. It's not some random number based on an arbitrary scale, you know exactly what it means. It tracks the quality of movies pretty well too. I'd like to see something similar for games.

    ZackSchilling on
    ghost-robot.jpg
  • XagarathXagarath Registered User regular
    edited August 2008
    I like the rotten tomatoes approach to score aggregation. You get a whole lot of people, mostly big critics, and you ask them the question: "did you personally like this?". Then you present a percentage of the people that liked it. It's not some random number based on an arbitrary scale, you know exactly what it means. It tracks the quality of movies pretty well too. I'd like to see something similar for games.

    It's not without problems, though.
    Mediocre stuff that nobody really disliked winds up with higher scores than divisive, daring arthouse-type stuff.

    Xagarath on
  • TelMarineTelMarine Registered User regular
    edited August 2008
    EvilBadman wrote: »
    I am a fan of the BUY RENT or PASS scoring system, easy and no fuss.

    I'd rather have this coupled with the summary at the end still for each specific part of the game (Gameplay, graphics, sound, etc.). I find scores to be flawed in the fact that different games have different expecations with it, so 9/10 for a game that was unknown and tried out real well versus an 8/10 for a franchise that has lofty expectations.

    TelMarine on
    3ds: 4983-4935-4575
  • AxenAxen My avatar is Excalibur. Yes, the sword.Registered User regular
    edited August 2008
    I am apparently one of the few that actually does read the review. I find it much more helpful to me than a score. I do use scores to determine which game reviews I should read. Normally 7.0 and up I will take a look at seeing how usually anything 6.0 or below is generally regarded as crap. There have been a few exceptions to the rule, but that's where G&T comes in.

    If they need to have number scores I would prefer "out of 5". I think its the best gauge. None of this 8.7, 8.8, 8.9 crap.

    Axen on
    A Capellan's favorite sheath for any blade is your back.
  • SzechuanosaurusSzechuanosaurus Registered User, ClubPA regular
    edited August 2008
    I like the rotten tomatoes approach to score aggregation. You get a whole lot of people, mostly big critics, and you ask them the question: "did you personally like this?". Then you present a percentage of the people that liked it. It's not some random number based on an arbitrary scale, you know exactly what it means. It tracks the quality of movies pretty well too. I'd like to see something similar for games.

    Even that isn't a great measure of whether you will like it, unless you know each of those people and what their tastes are like.

    Seriously, you can't simplify opinion and personal taste and experience. You need to discuss it and express it. We spent thousands of years developing a language for complex communication. People need to fucking use it.

    Szechuanosaurus on
  • LewiePLewieP Registered User regular
    edited August 2008
    I also think it's pretty funny how Zero Punctuation fits into this.

    It basically took the entire internet by storm, and it is the most subjective type of reviews out there, and has no score.

    I guess if you bring enough funny, and are delivered in lazy-friendly short video format you can buck the trend.

    LewieP on
  • ZackSchillingZackSchilling Registered User regular
    edited August 2008
    I agree that the mediocre get inflated percentages in a RT system, but it's counteracted by how much movie critics hate by-the-numbers mediocre movies. The line only gets blurry between "mainstream good" and "quirky excellent".

    Also, it's really only good as a baseline to satisfy people who want a single number. Of course reading one complete review tells you more than that aggregate number.

    ZackSchilling on
    ghost-robot.jpg
  • EvilBadmanEvilBadman DO NOT TRUST THIS MAN Registered User regular
    edited August 2008
    I like the rotten tomatoes approach to score aggregation. You get a whole lot of people, mostly big critics, and you ask them the question: "did you personally like this?". Then you present a percentage of the people that liked it. It's not some random number based on an arbitrary scale, you know exactly what it means. It tracks the quality of movies pretty well too. I'd like to see something similar for games.

    So, the Gaming portion of Rotten Tomatoes? Or, Metacritic?

    EDIT: I coulda swore RT had a gaming area.

    EvilBadman on
    FyreWulff wrote: »
    I should note that Badman is fucking awesome
    XBL- Evil Badman; Steam- EvilBadman; Twitter - EvilBadman
  • AroducAroduc regular
    edited August 2008
    I basically agree with your opinion, Lewie.

    And Aroduc is wrong. I enjoy reading reviews, assuming they are well written. Which a lot of them really aren't. I don't enjoy reading mindless, boring drivel.

    This may be a surprise, but you are not the world. I apologize for the hyperbole of saying "everybody" when I meant around 90% of the population give or take a percentage point. Please find it in your heart to forgive me. Feel free to look up... oh... I don't know... any of the few hundred studies on usability should you feel the need to argue this further. I suggest starting with Jakob Nielsen.

    Aroduc on
  • ZackSchillingZackSchilling Registered User regular
    edited August 2008
    Game reviews themselves would need to change before you could apply the rotten tomatoes model to them. You can try but it's not going to be nearly as meaningful.

    ZackSchilling on
    ghost-robot.jpg
  • XagarathXagarath Registered User regular
    edited August 2008
    EvilBadman wrote: »
    I like the rotten tomatoes approach to score aggregation. You get a whole lot of people, mostly big critics, and you ask them the question: "did you personally like this?". Then you present a percentage of the people that liked it. It's not some random number based on an arbitrary scale, you know exactly what it means. It tracks the quality of movies pretty well too. I'd like to see something similar for games.

    So, the Gaming portion of Rotten Tomatoes? Or, Metacritic?

    EDIT: I coulda swore RT had a gaming area.

    Metacritic has a different system.

    Xagarath on
  • Santa ClaustrophobiaSanta Claustrophobia Ho Ho Ho Disconnecting from Xbox LIVERegistered User regular
    edited August 2008
    The big disaster with many reviews is a tendency of the author to skip or gloss over flaws and hand wave them away with 'but they were just minor, niggling flaws'. At the opposite end of this, is the inevitable accusations that the author is biased for or against something when they put detail into discussing any problems they had with the game (see: Zero Punctuation).

    As a consumer, it's much easier to determine my final interest in a game based on reviews from a source I'm familiar with. As in, what a reviewer likes and dislikes more than any final number.

    Santa Claustrophobia on
    You're muckin' with a G!

    Do not engage the Watermelons.
  • admanbadmanb unionize your workplace Seattle, WARegistered User regular
    edited August 2008
    Rorus Raz wrote: »
    I wonder if the quality of reviews is brought down by the writer's knowledge that, for his hours of work, the only thing most people will give a shit about is a number. That would...kill my motivation a tad.

    I sure hope that's their excuse, the alternative is that this is actually some guy's idea of a "critique." It reads like a very long-winded back-of-the-box description. I don't think the reviewer even knows how to review a fighting game. I'm not saying I do, but at least I'm not getting paid for it.

    admanb on
  • MonstyMonsty Registered User regular
    edited August 2008
    I never look at reviews...for anything, really. The thing is, I have a suggestion for the perfect system for reviewing movies, games, whatever--but I haven't seen anyone use it yet. It would basically be one-line "reviews" like so:

    You should buy(or rent) this game if you like __BLANK__
    You should watch this movie if you like __BLANK__

    Blank could be genres, other games or movies, or "DON'T GET THIS" for things that are truly reprehensible. I mean, even for genres of entertainment you don't like, it's usually pretty easy to guess what type of people would like it.

    And surely they could add paragraphs of description for fluff. But it shouldn't be necessary. I dunno. I guess I don't have anything to add.

    Monsty on
  • Santa ClaustrophobiaSanta Claustrophobia Ho Ho Ho Disconnecting from Xbox LIVERegistered User regular
    edited August 2008
    admanb wrote: »
    Rorus Raz wrote: »
    I wonder if the quality of reviews is brought down by the writer's knowledge that, for his hours of work, the only thing most people will give a shit about is a number. That would...kill my motivation a tad.

    I sure hope that's their excuse, the alternative is that this is actually some guy's idea of a "critique." It reads like a very long-winded back-of-the-box description. I don't think the reviewer even knows how to review a fighting game. I'm not saying I do, but at least I'm not getting paid for it.

    My eyes began to glaze over. Did he even mention how the game played at all? All I saw was how pretty it looks, how awesome the Star Wars characters are, and when I woke up, I was covered in butter and my left hand hurt...

    Santa Claustrophobia on
    You're muckin' with a G!

    Do not engage the Watermelons.
  • ZackSchillingZackSchilling Registered User regular
    edited August 2008
    Reviews should focus more on what a game is trying to achieve and how well it works. I want more substantive reviews that put games in context.

    Current reviews are too focused on providing buying advice. I think that's a mistake.

    ZackSchilling on
    ghost-robot.jpg
  • admanbadmanb unionize your workplace Seattle, WARegistered User regular
    edited August 2008
    My eyes began to glaze over. Did he even mention how the game played at all? All I saw was how pretty it looks, how awesome the Star Wars characters are, and when I woke up, I was covered in butter and my left hand hurt...

    Hahaha.

    No. No he didn't.

    admanb on
  • JCRooksJCRooks Registered User regular
    edited August 2008
    I agree with most of LewieP's points. The problem, as has been pointed out by many, is that the audience sucks. Many gamers are, sadly, just not mature enough to want to read, try to be non-biased, be open to different opinions, etc. Much of it has to do with the demographics. Although gamers are certainly getting older, you can't ignore that many are still uneducated and young. That's not necessarily a bad thing, just the-way-it-is. (I certainly would have balked at not seeing review scores when I was a young fanboyish lad)

    And this is by no means limited to just games either. I'm sure the same has been said of countless other media, from movies, TV shows, books, theater shows, etc. Certainly the demographics of each are different, and thus the problem of people "not reading" varies in severity, but it's all there in some way shape or form.

    I think the solution is pretty simple. As the industry gets larger and the demographics become broader, there will eventually be enough folks that want well-written reviews and editorials (even ones without scores) and a magazine/website will be able to provide this without going under. In the meantime, it's just not financially responsible. Many magazines and sites have tried this, and have gone back because they lost too many readers. Maybe in a few years another will try it again and succeed? Arguably, EDGE is the closest to having a really dedicated and older audience. Sites like 1UP and IGN? Not so much.

    JCRooks on
    Xbox LIVE, Steam, Twitter, etc. ...
    Gamertag: Rooks
    - Don't add me, I'm at/near the friend limit :)

    Steam: JC_Rooks

    Twitter: http://twitter.com/JiunweiC

    I work on this: http://www.xbox.com
  • mrsnackroadmrsnackroad Registered User regular
    edited August 2008
    Why does a review have to be so ELABORATE, though?

    Say whats good, say what's bad, say if its worth my $60.

    That's all you have to hit, really. It is way oversimplified, sure, but I don't think those basic points are being hit well enough to adapt and improve the review system. I'm going to be WAY more discerning about a video game than I will be about buying a video from the $5 bin.

    mrsnackroad on
  • The_ScarabThe_Scarab Registered User regular
    edited August 2008
    Why does a review have to be so ELABORATE, though?

    Say whats good, say what's bad, say if its worth my $60.

    That's all you have to hit, really. It is way oversimplified, sure, but I don't think those basic points are being hit well enough to adapt and improve the review system. I'm going to be WAY more discerning about a video game than I will be about buying a video from the $5 bin.

    In an average gaming year about a thousand games are released that are worth your $60.

    So if the reviews are just 'buy it' then you either have to buy them all or do a pot luck on which one to get. This is why reviews are specific, informed and precise.

    The_Scarab on
  • Toxin01Toxin01 Registered User regular
    edited August 2008
    Here's the thing.

    Most people don't consider Games an Art Form, I don't either. To most it's just entertainment, and it's much easier for the average person to pick up a Gameinformer and scan the review and see what "Super mega t-rex riding jet li adventure kids 7" got out of 10 then to read a 6 page long review detaling the game and all it's flaws and high points.

    Toxin01 on
    Aiden Baail: Level 1 Swordmage: 19 AC 14 Fort 15 Ref 13 Will (Curse Of The Black Pearls)
    GM: Rusty Chains (DH Ongoing)
  • WillethWilleth Registered User regular
    edited August 2008
    I quite like the xkcd fruit review scale that's been used on SavyGamer before - if you interpret the original graph as difficulty and enjoyment instead of its original intention then it works very well for games.

    Willeth on
    @vgreminders - Don't miss out on timed events in gaming!
    @gamefacts - Totally and utterly true gaming facts on the regular!
  • Vincent GraysonVincent Grayson Frederick, MDRegistered User regular
    edited August 2008
    I don't think a "score" (especially a blank out of blank score) is necessary, but that a general conclusion like "Buy It" "Rent It" "Skip It" is a favorable middle ground.

    The people who don't like reading still get decent advice, and it removes the stigma of Game X getting a 9.4 and Game Y getting a 9.3, therefore Game X is better.

    When it comes down to it, I'm rarely interested in whether GTA4 is "better" than Halo 3 or MGS4, or Okami, or whatever, but whether this person (or group of people) who has played the game thinks it's worth the price of entry.

    Vincent Grayson on
  • mrsnackroadmrsnackroad Registered User regular
    edited August 2008
    The_Scarab wrote: »
    Why does a review have to be so ELABORATE, though?

    Say whats good, say what's bad, say if its worth my $60.

    That's all you have to hit, really. It is way oversimplified, sure, but I don't think those basic points are being hit well enough to adapt and improve the review system. I'm going to be WAY more discerning about a video game than I will be about buying a video from the $5 bin.

    In an average gaming year about a thousand games are released that are worth your $60.

    So if the reviews are just 'buy it' then you either have to buy them all or do a pot luck on which one to get. This is why reviews are specific, informed and precise.

    This is true, yes - but, the specific, informed and precise review still has to say whats good and whats bad. Be in depth on BOTH points; and if the game is such a mess, then say so - don't skew an entire review against a game, then try and redeem the product with a paragraph tacked on at the end saying "It looks pretty" or whatever buzzwords are related towards saying why the game is so awesome.

    mrsnackroad on
  • gobassgogobassgo Registered User regular
    edited August 2008
    I think 1Up's A-F grading system works pretty well.

    gobassgo on
  • Santa ClaustrophobiaSanta Claustrophobia Ho Ho Ho Disconnecting from Xbox LIVERegistered User regular
    edited August 2008
    Toxin01 wrote: »
    Here's the thing.

    Most people don't consider Games an Art Form, I don't either. To most it's just entertainment, and it's much easier for the average person to pick up a Gameinformer and scan the review and see what "Super mega t-rex riding jet li adventure kids 7" got out of 10 then to read a 6 page long review detaling the game and all it's flaws and high points.

    Here's the natural progression of this thought: If all reviews must be boiled down to a simple numeric value, then what's the point of printing the magazine in the first place?

    Here's the next step full of hyperbole: They could just as easily list the 35 or so reviewed games in a one-sheet pamphlet and distribute it to passersby. The numbers could be obtained by rolling dice by the building's janitor. There could be no accountability at all with the review process by at least comparing the rating to the written word.

    (Hasn't there been more than one review where the text didn't seem to match the rating given?)

    Santa Claustrophobia on
    You're muckin' with a G!

    Do not engage the Watermelons.
  • JCRooksJCRooks Registered User regular
    edited August 2008
    I don't think a "score" (especially a blank out of blank score) is necessary, but that a general conclusion like "Buy It" "Rent It" "Skip It" is a favorable middle ground.

    The people who don't like reading still get decent advice, and it removes the stigma of Game X getting a 9.4 and Game Y getting a 9.3, therefore Game X is better.

    When it comes down to it, I'm rarely interested in whether GTA4 is "better" than Halo 3 or MGS4, or Okami, or whatever, but whether this person (or group of people) who has played the game thinks it's worth the price of entry.

    You know, a BUY/RENT/SKIP is essentially a score with a range of 3. Maybe some people like that conciseness, and it may work if you absolutely trust a particular reviewer's opinion. But I don't really care for that. I want to know more about the game, the experience, how it compares to others in the genre, how it stands in the game industry overall, what the reviewer's biases are, their opinions, etc. There's nothing wrong with lengthy reviews in my eyes, but then again, I also consider reading a hobby. :)

    Games are inherently subjective. Even putting something like BUY/RENT/SKIP can be seen as making it objective, and to many fanboy gamers, it is. Frankly, I don't care much for that nonesense. I don't take it personal if a critic happens not to like something that I enjoy deeply ... but I would like to know more about the reasoning behind it. Not because I want to fanboyishly "call him/her out" or anything, but because I'm curious and am open to hearing other people's opinions.

    JCRooks on
    Xbox LIVE, Steam, Twitter, etc. ...
    Gamertag: Rooks
    - Don't add me, I'm at/near the friend limit :)

    Steam: JC_Rooks

    Twitter: http://twitter.com/JiunweiC

    I work on this: http://www.xbox.com
Sign In or Register to comment.