It would have been interesting to run a 10-year poll, including all games since the poll's inception. It would also make it really tough to narrow it down to just 10-20 games though.
The massive change in voting depending on PA or Twitter sourced feedback is REALLY interesting (my day job is a Research Director specialising in online surveys) and demonstrates brilliantly that social media is an incredibly dangerous distribution method for non-invitational (anonymous) surveys due to the way viral distribution self-selects it's respondent profile through shared interest.
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter. It's possible to infer ranking from ratings (although you end up getting 'ties'), but impossible to infer ratings from rankings.
Instead of combining ALL nominations for the 'best game ever' poll, why not use the Top 20 (or 30 or whatever) from the results for each year as at least a baseline. Surely if it finished 75th in 2006 it won't sniff the top 20 for Best Ever.
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
<links>
That's interesting, but I disagree with the assumption that calling a square a square instead of a rectangle somehow degrades all squares and rectangles. "Game" doesn't have to be synonymous with "interactive (in the computer science sense of the words) entertainment", it's OK to more narrowly define certain terms and perhaps adopted the broader term ("interactive entertainment") instead of the more narrow one ("video game").
Does reading a book aloud fundamentally change it from being a book?
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
What is a game? Professor M. Mouse of Texas, America claims that the word game denotes
"the historical process by which the term game has been characterised and understood".
Easy for you to say, Professor!!
Those of us with a more down-home approach to codifying the various aspects of a
nebulous and unbearable human condition prefer to go by a simpler definition, thus.
A game is some combination of the following indivisable elements:
- skeleton
- red key
- score thing
- magic door
If you see something that looks like a videogame but isn't, you should notify the Police.
Instead of combining ALL nominations for the 'best game ever' poll, why not use the Top 20 (or 30 or whatever) from the results for each year as at least a baseline. Surely if it finished 75th in 2006 it won't sniff the top 20 for Best Ever.
The only reason I suggested that is because some games have become vindicated by history. Alpha Protocol, for example, regardless of @Tube inevitably disagreeing.
+2
Options
RandomHajileNot actually a SnatcherThe New KremlinRegistered Userregular
The massive change in voting depending on PA or Twitter sourced feedback is REALLY interesting (my day job is a Research Director specialising in online surveys) and demonstrates brilliantly that social media is an incredibly dangerous distribution method for non-invitational (anonymous) surveys due to the way viral distribution self-selects it's respondent profile through shared interest.
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter.
I've thought a lot about how you could get around this, and this is probably one of the better solutions. I'm sure there are papers written on the topic, but a decade out of college has eroded my will to read more deeply into this sort of thing. There's one idea rattling around in my mind after discussing (of all things) a fantasy football auction draft system, which would give a person a pool of points that they could apply to any game, but then it comes with the problem of how to fairly distribute points. Do you give more to people who have played more games (5 points per game) or an equal number of points to everyone (100 points no matter how many games you play)? Do you set a maximum number of points allowed to one game or just let people skew the results because they really liked that one game and nothing else?
The massive change in voting depending on PA or Twitter sourced feedback is REALLY interesting (my day job is a Research Director specialising in online surveys) and demonstrates brilliantly that social media is an incredibly dangerous distribution method for non-invitational (anonymous) surveys due to the way viral distribution self-selects it's respondent profile through shared interest.
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter.
I've thought a lot about how you could get around this, and this is probably one of the better solutions. I'm sure there are papers written on the topic, but a decade out of college has eroded my will to read more deeply into this sort of thing. There's one idea rattling around in my mind after discussing (of all things) a fantasy football auction draft system, which would give a person a pool of points that they could apply to any game, but then it comes with the problem of how to fairly distribute points. Do you give more to people who have played more games (5 points per game) or an equal number of points to everyone (100 points no matter how many games you play)? Do you set a maximum number of points allowed to one game or just let people skew the results because they really liked that one game and nothing else?
Yeah, that system seems like it would be really dependent on people avoiding hyperbole in their ratings.
I wonder if something that just allowed people to rate games that they played on a scale of 1-10 would be a decent way of going about it. You could even correct for people who have extreme response tendencies (1s or 10s only) with nothing in between using a multinomial logistic regression.
Instead of combining ALL nominations for the 'best game ever' poll, why not use the Top 20 (or 30 or whatever) from the results for each year as at least a baseline. Surely if it finished 75th in 2006 it won't sniff the top 20 for Best Ever.
The only reason I suggested that is because some games have become vindicated by history. Alpha Protocol, for example, regardless of @Tube inevitably disagreeing.
That's why I figured Top 20-30 instead of top 5-10, and only calling it a baseline. Exceptions like Alpha Protocol are probably very few in number, and we'd need to manually nominate everything that predates the polls as well. Culling the low finishers is a quick way to narrow down the last few years though.
0
Options
HardtargetThere Are Four LightsVancouverRegistered Userregular
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
Tycho you need to stop doing this when somebody challenges your opinion. Jdark is free to think what he wants and if you want to debate him on it actually debate him on it instead of linking to videos and links of people making arguments for you that may or may not hold water. You've been doing this a ton lately for any sort of post that you even slightly disagree with and it completely stops the discussion and breaks the conversation.
edit - with that said in this occasion I had not noticed Jdark actually did respond to this heh
There's one idea rattling around in my mind after discussing (of all things) a fantasy football auction draft system, which would give a person a pool of points that they could apply to any game, but then it comes with the problem of how to fairly distribute points. Do you give more to people who have played more games (5 points per game) or an equal number of points to everyone (100 points no matter how many games you play)? Do you set a maximum number of points allowed to one game or just let people skew the results because they really liked that one game and nothing else?
That's pretty much the system they're using on RPGCodex at the moment (one pool of points regardless of games played) and... it's not great.
Generally when I use a ranking system I'll have a 'points max' system where there is 10 points for the top ranked response and 0 for the lowest ranked response, and each other ranking is allocated points based on the number of items ranked by that individual (so if they rank 11 games then top rank is 10pts, second rank is 9pts, down to eleventh rank which gets 0 pts... but if they rank 6 games then top rank is 10, second rank is 8pts etc etc). It's a bit of a pain to calculate, and it 'assumes' that the bottom rank is somehow a bad game (even if it's just the worst game that person has played, but is actually a decent game) so it's faaaar from perfect.
Personally I like ratings. Most people understand a 0-10 scale, or 0-100 scale (easier to subsequently rank if you want to) and despite regional variance (some cultures are less likely to hand out 10's than others) it usually works out OK. The counter argument to ratings, other than individual dispensation to rate higher or lower, is that you end up getting something that might as well be Metacritic (but without visible scoring you hopefully avoid the rampant fanboyism and extreme high/low scoring to 'correct' others opinions).
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
I was referring to Depression Quest, not Gone Home.
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
Tycho you need to stop doing this when somebody challenges your opinion. Jdark is free to think what he wants and if you want to debate him on it actually debate him on it instead of linking to videos and links of people making arguments for you that may or may not hold water. You've been doing this a ton lately for any sort of post that you even slightly disagree with and it completely stops the discussion and breaks the conversation.
edit - with that said in this occasion I had not noticed Jdark actually did respond to this heh
This isn't really the thread for a whole debate on the topic. When the same question came up in a previous thread someone PMed me to continue the conversation. @jdarksun and anyone else who wants to have the "what is game" conversation can PM me and I am more than happy to have that talk. Someone could even make a D&D thread although I'm less sanguine about the prospects of that. All I will say here is that the idea of calling something "not a game" in the GOTY thread seems to me to directly lead to the question "should it even be on the voting list, then?" and anyone who wants to exclude Gone Home from the voting is on pretty shaky territory. Anyone who doesn't want to exclude Gone Home but still prefers not to call it a game is at that point not even making sense to me (if it's on the GOTY list, that makes it a G, right?), but again, I'm happy to talk it over in PMs. The reason I provide the links instead of writing something is because in the past I actually have begun to write up long responses only to find I was effectively parroting Errant Signal and the other sources. Rather than duplicate their arguments without attribution I now simply link to the most effective presentations of the same arguments I myself would make.
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
I was referring to Depression Quest, not Gone Home.
Depression Quest sounds super interesting. I only just checked it a minute ago.
The massive change in voting depending on PA or Twitter sourced feedback is REALLY interesting (my day job is a Research Director specialising in online surveys) and demonstrates brilliantly that social media is an incredibly dangerous distribution method for non-invitational (anonymous) surveys due to the way viral distribution self-selects it's respondent profile through shared interest.
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter. It's possible to infer ranking from ratings (although you end up getting 'ties'), but impossible to infer ratings from rankings.
Is social media a more dangerous distribution method than pinning the poll to the top of the PA forums? Surely PA forum members are self-selected based on shared interest as much as (more so than?) mcc's twitter followers and whoever reads the retweets.
Meh. People complained when Walking Dead won (it won, right?) last year because it wasn't a 'game'. But I argue that it's more of a 'game' than Gone Home (at least, from what I can tell). However people quite enjoyed it, and it made the list so I'm not going to even go as far as saying that it bothers me that it won.
The list, in the end, shows that there are a TON of good non-AAA games out there. That's pretty awesome if you think about that.
The notion that for someone to rank something genuinely personal, intellectually stimulating, emotionally touching, and unique in the landscape of games above a game where you stab 500 people in the face with a Skyhook and literally go to Racist Disneyland requires the person making the ranking to not have played many games is a pretty toxic one.
That's certainly one opinion.
Another opinion is that Gone Home is not a game in any traditional sense of the word, and is instead closer to a audio drama, book, or movie.
The fact that the two groups rated Gone Home radically differently - to the point where Gone Home is not even in the PA Top 10 - shows pretty clearly that it targets a different audience.
Heck, there's only a few points of overlap between Twitter's Top 10 and PA's Top 10 : The Stanley Parable, Pokémon X and Y, and Saints Row IV.
That hints pretty strongly towards some sort of dramatic demographic difference.
Also, Bioshock Infinite's ending spoke pretty goddamn deeply to me as a father, so. You know. Opinions.
I was referring to Depression Quest, not Gone Home.
Depression Quest sounds super interesting. I only just checked it a minute ago.
It would have been interesting to run a 10-year poll, including all games since the poll's inception. It would also make it really tough to narrow it down to just 10-20 games though.
All I will say here is that the idea of calling something "not a game" in the GOTY thread seems to me to directly lead to the question "should it even be on the voting list, then?" and anyone who wants to exclude Gone Home from the voting is on pretty shaky territory.
Why? What is fundamentally dangerous about categorizing interactive entertainment? There are a million different literary awards, subcategorizing different types of literature into things like audiobooks and plays and podcasts, not to mention novels and novellas and short stories and flash.
Anyone who doesn't want to exclude Gone Home but still prefers not to call it a game is at that point not even making sense to me (if it's on the GOTY list, that makes it a G, right?)...
I think including stuff like Gone Home into a larger category of stuff like "Interactive Entertainment" and having an "Interactive Entertainment of the Year" is totally reasonable. Or if we want to co-opt "game" into "interactive entertainment", then I think there needs to be a new term that means "game with a fail state" (which is how I currently interpret the term "game").
Happy to talk about this in a different thread or via PM though @TychoCelchuuu!
@mcc, would it be possible to post the alternative ranking methods for PA only? I'm interested in what PA thinks, moreso than the twitter followers.
OK, a minute ago, when I said I'd post that tonight? It was easier to generate than I thought, I'll just post it now
Alternate rankings, known Penny Arcade voters only:
BORDA RANKING (STANDARD)
1. Bioshock Infinite (360, PC, PS3) (2259)
2. Tomb Raider 2013 (360, PC, PS3) (1799)
3. The Last Of Us (PS3) (1544)
4. Saints Row IV (360, PC, PS3) (1387)
5. The Legend of Zelda: A Link Between Worlds (3DS) (1218)
6. Grand Theft Auto V (360, PS3) (1211)
7. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (1187)
8. Fire Emblem: Awakening (3DS) (1187)
9. Pokémon X and Y (3DS) (1102)
10. Papers, Please (PC) (873)
11. Brothers: A Tale of Two Sons (360*, PC, PS3*) (834)
12. Rogue Legacy (PC) (794)
13. Gone Home (PC) (748)
14. The Stanley Parable (PC) (729)
15. StarCraft II: Heart of the Swarm (PC) (668)
16. Super Mario 3D World (WiiU) (650)
17. Gunpoint (PC) (619)
18. Metal Gear Rising: Revengeance (360, PS3) (604)
19. Ni No Kuni (PS3) (598)
20. Guacamelee! (PC, PS3*, Vita*) (579)
FIRST PAST THE POST (FIRST PLACE VOTES ONLY)
1. The Last Of Us (PS3) (39)
2. Bioshock Infinite (360, PC, PS3) (23)
3. Saints Row IV (360, PC, PS3) (21)
4. The Legend of Zelda: A Link Between Worlds (3DS) (19)
5. Fire Emblem: Awakening (3DS) (16)
6. Tomb Raider 2013 (360, PC, PS3) (13)
7. Pokémon X and Y (3DS) (13)
8. Grand Theft Auto V (360, PS3) (12)
9. Dota 2 (PC) (10)
10. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (9)
11. Gone Home (PC) (9)
12. StarCraft II: Heart of the Swarm (PC) (9)
13. Metal Gear Rising: Revengeance (360, PS3) (8)
14. Etrian Odyssey IV: Legends of the Titan (3DS) (6)
15. Monster Hunter 3 Ultimate (3DS, WiiU) (5)
16. Brothers: A Tale of Two Sons (360*, PC, PS3*) (4)
17. Papers, Please (PC) (4)
18. Shin Megami Tensei IV (3DS) (4)
19. The Stanley Parable (PC) (4)
20. Super Mario 3D World (WiiU) (4)
APPROVAL (RANKINGS IGNORED, ONLY LIST INCLUSION COUNTED)
1. Bioshock Infinite (360, PC, PS3) (148)
2. Tomb Raider 2013 (360, PC, PS3) (113)
3. The Last Of Us (PS3) (90)
4. Saints Row IV (360, PC, PS3) (87)
5. Grand Theft Auto V (360, PS3) (80)
6. Fire Emblem: Awakening (3DS) (77)
7. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (77)
8. The Legend of Zelda: A Link Between Worlds (3DS) (73)
9. Papers, Please (PC) (71)
10. Pokémon X and Y (3DS) (69)
11. Rogue Legacy (PC) (60)
12. Brothers: A Tale of Two Sons (360*, PC, PS3*) (59)
13. The Stanley Parable (PC) (52)
14. Gone Home (PC) (51)
15. Guacamelee! (PC, PS3*, Vita*) (51)
16. Gunpoint (PC) (47)
17. StarCraft II: Heart of the Swarm (PC) (45)
18. Shadowrun Returns (Android, PC, iOS) (43)
19. Ni No Kuni (PS3) (42)
20. Far Cry 3: Blood Dragon (360*, PC, PS3*) (42)
INSTANT RUNOFF VOTE
1. Bioshock Infinite (360, PC, PS3) (148)
2. Tomb Raider 2013 (360, PC, PS3) (113)
3. The Last Of Us (PS3) (90)
4. Saints Row IV (360, PC, PS3) (87)
5. The Legend of Zelda: A Link Between Worlds (3DS) (73)
6. Fire Emblem: Awakening (3DS) (77)
7. Grand Theft Auto V (360, PS3) (80)
8. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (77)
9. Pokémon X and Y (3DS) (69)
10. Papers, Please (PC) (71)
11. Brothers: A Tale of Two Sons (360*, PC, PS3*) (59)
12. Gone Home (PC) (51)
13. Rogue Legacy (PC) (60)
14. The Stanley Parable (PC) (52)
15. Gunpoint (PC) (47)
16. StarCraft II: Heart of the Swarm (PC) (45)
17. Ni No Kuni (PS3) (42)
18. Super Mario 3D World (WiiU) (40)
19. Shadowrun Returns (Android, PC, iOS) (43)
20. Metal Gear Rising: Revengeance (360, PS3) (40)
All I will say here is that the idea of calling something "not a game" in the GOTY thread seems to me to directly lead to the question "should it even be on the voting list, then?" and anyone who wants to exclude Gone Home from the voting is on pretty shaky territory.
Why? What is fundamentally dangerous about categorizing interactive entertainment? There are a million different literary awards, subcategorizing different types of literature into things like audiobooks and plays and podcasts, not to mention novels and novellas and short stories and flash.
Not that I'm interested in arguing this since I think its a bunch of stupid. But the danger in categorizing too much is you limit the creativity allowed in a medium.
Also...all that stuff you mentioned would still fall under the very big umbrella of literature.
Just like a lot of stuff can be a game.
And honestly, trying so hard to categorize things is a waste of time. And I believe that categories are only good for giving people a decent idea of what they're in for before they consume a product and even that isn't set in stone.
I mean that isn't to say I'm on anyone's side, because I honestly find Tycho's indie game hipster posturing to be quite annoying. I just think that in an industry that's still developing trying to pigeonhole it is pretty counter-intuitive.
+3
Options
RandomHajileNot actually a SnatcherThe New KremlinRegistered Userregular
The massive change in voting depending on PA or Twitter sourced feedback is REALLY interesting (my day job is a Research Director specialising in online surveys) and demonstrates brilliantly that social media is an incredibly dangerous distribution method for non-invitational (anonymous) surveys due to the way viral distribution self-selects it's respondent profile through shared interest.
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter.
I've thought a lot about how you could get around this, and this is probably one of the better solutions. I'm sure there are papers written on the topic, but a decade out of college has eroded my will to read more deeply into this sort of thing. There's one idea rattling around in my mind after discussing (of all things) a fantasy football auction draft system, which would give a person a pool of points that they could apply to any game, but then it comes with the problem of how to fairly distribute points. Do you give more to people who have played more games (5 points per game) or an equal number of points to everyone (100 points no matter how many games you play)? Do you set a maximum number of points allowed to one game or just let people skew the results because they really liked that one game and nothing else?
Yeah, that system seems like it would be really dependent on people avoiding hyperbole in their ratings.
I wonder if something that just allowed people to rate games that they played on a scale of 1-10 would be a decent way of going about it. You could even correct for people who have extreme response tendencies (1s or 10s only) with nothing in between using a multinomial logistic regression.
That's actually not a bad idea. I mean, it still has its problems as noted, but I had a hard time with basically everything after The Last of Us on my list. To be able to just independently rate each one would have made me feel a bit better about the whole list (though I'm pretty happy with how my list ended up).
DragkoniasThat Guy Who Does StuffYou Know, There. Registered Userregular
I'm really not sure why people bring up trying to redo the system every year.
Its fine the way it is and I don't think we've had many issues with it. Not to mention its just a poll that doesn't really mean anything to begin with, so why try to add needless complexity?
In the most recent game I created, you are trapped in a tiny liferaft, bobbing up and down on an angrily frothing ocean drawn in red wireframe. You can look around from a first person perspective but you can not move or affect your environment in any way. Forty seconds after the game begins, totally regardless of anything you do, the waves finally begin to calm, a little, just enough for you to see you are steadily drifting toward a giant whirlpool hole in the sea, which your little boat tumbles into, probably just as you are beginning to notice it coming toward you, and your vision blurs out and you die. The game quits.
I don't find the question of what is or is not a "game" very interesting
The massive change in voting depending on PA or Twitter sourced feedback is REALLY interesting (my day job is a Research Director specialising in online surveys) and demonstrates brilliantly that social media is an incredibly dangerous distribution method for non-invitational (anonymous) surveys due to the way viral distribution self-selects it's respondent profile through shared interest.
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter.
I've thought a lot about how you could get around this, and this is probably one of the better solutions. I'm sure there are papers written on the topic, but a decade out of college has eroded my will to read more deeply into this sort of thing. There's one idea rattling around in my mind after discussing (of all things) a fantasy football auction draft system, which would give a person a pool of points that they could apply to any game, but then it comes with the problem of how to fairly distribute points. Do you give more to people who have played more games (5 points per game) or an equal number of points to everyone (100 points no matter how many games you play)? Do you set a maximum number of points allowed to one game or just let people skew the results because they really liked that one game and nothing else?
Yeah, that system seems like it would be really dependent on people avoiding hyperbole in their ratings.
I wonder if something that just allowed people to rate games that they played on a scale of 1-10 would be a decent way of going about it. You could even correct for people who have extreme response tendencies (1s or 10s only) with nothing in between using a multinomial logistic regression.
That's actually not a bad idea. I mean, it still has its problems as noted, but I had a hard time with basically everything after The Last of Us on my list. To be able to just independently rate each one would have made me feel a bit better about the whole list (though I'm pretty happy with how my list ended up).
It also eliminates a lot of that "oh geeeeez, I can't decide between these two!" factor. If more than one game deserves a top score, in your opinion, you can rate them that way, but you would also have to be aware that doing this too much would water down your overall impact.
I mean that isn't to say I'm on anyone's side, because I honestly find Tycho's indie game hipster posturing to be quite annoying. I just think that in an industry that's still developing trying to pigeonhole it is pretty counter-intuitive.
Take the "But is it art?" bullshit elsewhere. For the sake of this thread, anything on the list is a game and I don't need @Jdarksun and @TychoCelchuuu giving me a Siskel & Ebert routine.
I'm really not sure why people bring up trying to redo the system every year.
Its fine the way it is and I don't think we've had many issues with it. Not to mention its just a poll that doesn't really mean anything to begin with, so why try to add needless complexity?
Some of us like discussing how to improve polling data through the use of statistical analysis. I certainly don't expect any discussed changes to be implemented nor would I be upset if they aren't, but if you don't want to talk about it with us then maybe just... don't?
Posts
Thanks once again @mcc for hosting this poll and doing the analysis. I'd be intrigued to know how much different the results might have been if games were rated rather than ranked. I really like ranking systems but it's sometimes hard to find a fair and consistent way of weighting results when there is so much variance in the number of games selected/played by each voter. It's possible to infer ranking from ratings (although you end up getting 'ties'), but impossible to infer ratings from rankings.
Does reading a book aloud fundamentally change it from being a book?
Penny Arcade Rockstar Social Club / This is why I despise cyclists
Don't forget thecatamites.
The only reason I suggested that is because some games have become vindicated by history. Alpha Protocol, for example, regardless of @Tube inevitably disagreeing.
This is a clickable link to my Steam Profile.
Yeah, that system seems like it would be really dependent on people avoiding hyperbole in their ratings.
I wonder if something that just allowed people to rate games that they played on a scale of 1-10 would be a decent way of going about it. You could even correct for people who have extreme response tendencies (1s or 10s only) with nothing in between using a multinomial logistic regression.
That's why I figured Top 20-30 instead of top 5-10, and only calling it a baseline. Exceptions like Alpha Protocol are probably very few in number, and we'd need to manually nominate everything that predates the polls as well. Culling the low finishers is a quick way to narrow down the last few years though.
edit - with that said in this occasion I had not noticed Jdark actually did respond to this heh
That's pretty much the system they're using on RPGCodex at the moment (one pool of points regardless of games played) and... it's not great.
Generally when I use a ranking system I'll have a 'points max' system where there is 10 points for the top ranked response and 0 for the lowest ranked response, and each other ranking is allocated points based on the number of items ranked by that individual (so if they rank 11 games then top rank is 10pts, second rank is 9pts, down to eleventh rank which gets 0 pts... but if they rank 6 games then top rank is 10, second rank is 8pts etc etc). It's a bit of a pain to calculate, and it 'assumes' that the bottom rank is somehow a bad game (even if it's just the worst game that person has played, but is actually a decent game) so it's faaaar from perfect.
Personally I like ratings. Most people understand a 0-10 scale, or 0-100 scale (easier to subsequently rank if you want to) and despite regional variance (some cultures are less likely to hand out 10's than others) it usually works out OK. The counter argument to ratings, other than individual dispensation to rate higher or lower, is that you end up getting something that might as well be Metacritic (but without visible scoring you hopefully avoid the rampant fanboyism and extreme high/low scoring to 'correct' others opinions).
Edit: Same with Depression Quest or other stuff that people voted for.
Penny Arcade Rockstar Social Club / This is why I despise cyclists
The list, in the end, shows that there are a TON of good non-AAA games out there. That's pretty awesome if you think about that.
I didn't say what the S stood for
Well, don't leave us hanging!
Obviously S stands for "Story".
Incidentally I don't understand why COD is supposed to be an FPS, or even a game really, all people do is stand around and click on each other
Penny Arcade Rockstar Social Club / This is why I despise cyclists
You should be aware of the "Game of the Decade" poll that some of the G&T members did using my script. (I hosted and generated results, but didn't make the noms list or anything. There's probably a thread about it.)
What interests me about a redo Best Game Ever poll is seeing how preferences change over time, at least within this one community.
I think including stuff like Gone Home into a larger category of stuff like "Interactive Entertainment" and having an "Interactive Entertainment of the Year" is totally reasonable. Or if we want to co-opt "game" into "interactive entertainment", then I think there needs to be a new term that means "game with a fail state" (which is how I currently interpret the term "game").
Happy to talk about this in a different thread or via PM though @TychoCelchuuu!
Penny Arcade Rockstar Social Club / This is why I despise cyclists
Seems those of us who love it, really love it, while it lacks broader appeal...which I think falls perfectly in line with what I expected.
Alternate rankings, known Penny Arcade voters only:
BORDA RANKING (STANDARD)
1. Bioshock Infinite (360, PC, PS3) (2259)
2. Tomb Raider 2013 (360, PC, PS3) (1799)
3. The Last Of Us (PS3) (1544)
4. Saints Row IV (360, PC, PS3) (1387)
5. The Legend of Zelda: A Link Between Worlds (3DS) (1218)
6. Grand Theft Auto V (360, PS3) (1211)
7. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (1187)
8. Fire Emblem: Awakening (3DS) (1187)
9. Pokémon X and Y (3DS) (1102)
10. Papers, Please (PC) (873)
11. Brothers: A Tale of Two Sons (360*, PC, PS3*) (834)
12. Rogue Legacy (PC) (794)
13. Gone Home (PC) (748)
14. The Stanley Parable (PC) (729)
15. StarCraft II: Heart of the Swarm (PC) (668)
16. Super Mario 3D World (WiiU) (650)
17. Gunpoint (PC) (619)
18. Metal Gear Rising: Revengeance (360, PS3) (604)
19. Ni No Kuni (PS3) (598)
20. Guacamelee! (PC, PS3*, Vita*) (579)
FIRST PAST THE POST (FIRST PLACE VOTES ONLY)
1. The Last Of Us (PS3) (39)
2. Bioshock Infinite (360, PC, PS3) (23)
3. Saints Row IV (360, PC, PS3) (21)
4. The Legend of Zelda: A Link Between Worlds (3DS) (19)
5. Fire Emblem: Awakening (3DS) (16)
6. Tomb Raider 2013 (360, PC, PS3) (13)
7. Pokémon X and Y (3DS) (13)
8. Grand Theft Auto V (360, PS3) (12)
9. Dota 2 (PC) (10)
10. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (9)
11. Gone Home (PC) (9)
12. StarCraft II: Heart of the Swarm (PC) (9)
13. Metal Gear Rising: Revengeance (360, PS3) (8)
14. Etrian Odyssey IV: Legends of the Titan (3DS) (6)
15. Monster Hunter 3 Ultimate (3DS, WiiU) (5)
16. Brothers: A Tale of Two Sons (360*, PC, PS3*) (4)
17. Papers, Please (PC) (4)
18. Shin Megami Tensei IV (3DS) (4)
19. The Stanley Parable (PC) (4)
20. Super Mario 3D World (WiiU) (4)
APPROVAL (RANKINGS IGNORED, ONLY LIST INCLUSION COUNTED)
1. Bioshock Infinite (360, PC, PS3) (148)
2. Tomb Raider 2013 (360, PC, PS3) (113)
3. The Last Of Us (PS3) (90)
4. Saints Row IV (360, PC, PS3) (87)
5. Grand Theft Auto V (360, PS3) (80)
6. Fire Emblem: Awakening (3DS) (77)
7. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (77)
8. The Legend of Zelda: A Link Between Worlds (3DS) (73)
9. Papers, Please (PC) (71)
10. Pokémon X and Y (3DS) (69)
11. Rogue Legacy (PC) (60)
12. Brothers: A Tale of Two Sons (360*, PC, PS3*) (59)
13. The Stanley Parable (PC) (52)
14. Gone Home (PC) (51)
15. Guacamelee! (PC, PS3*, Vita*) (51)
16. Gunpoint (PC) (47)
17. StarCraft II: Heart of the Swarm (PC) (45)
18. Shadowrun Returns (Android, PC, iOS) (43)
19. Ni No Kuni (PS3) (42)
20. Far Cry 3: Blood Dragon (360*, PC, PS3*) (42)
INSTANT RUNOFF VOTE
1. Bioshock Infinite (360, PC, PS3) (148)
2. Tomb Raider 2013 (360, PC, PS3) (113)
3. The Last Of Us (PS3) (90)
4. Saints Row IV (360, PC, PS3) (87)
5. The Legend of Zelda: A Link Between Worlds (3DS) (73)
6. Fire Emblem: Awakening (3DS) (77)
7. Grand Theft Auto V (360, PS3) (80)
8. Assassin's Creed IV: Black Flag (360, PC, PS3, PS4, WiiU, XB1) (77)
9. Pokémon X and Y (3DS) (69)
10. Papers, Please (PC) (71)
11. Brothers: A Tale of Two Sons (360*, PC, PS3*) (59)
12. Gone Home (PC) (51)
13. Rogue Legacy (PC) (60)
14. The Stanley Parable (PC) (52)
15. Gunpoint (PC) (47)
16. StarCraft II: Heart of the Swarm (PC) (45)
17. Ni No Kuni (PS3) (42)
18. Super Mario 3D World (WiiU) (40)
19. Shadowrun Returns (Android, PC, iOS) (43)
20. Metal Gear Rising: Revengeance (360, PS3) (40)
Penny Arcade Rockstar Social Club / This is why I despise cyclists
Last year saw a more frothy rage over TWD winning. Jdark and Tycho are having a pretty civil debate IMO.
Not that I'm interested in arguing this since I think its a bunch of stupid. But the danger in categorizing too much is you limit the creativity allowed in a medium.
Also...all that stuff you mentioned would still fall under the very big umbrella of literature.
Just like a lot of stuff can be a game.
And honestly, trying so hard to categorize things is a waste of time. And I believe that categories are only good for giving people a decent idea of what they're in for before they consume a product and even that isn't set in stone.
I mean that isn't to say I'm on anyone's side, because I honestly find Tycho's indie game hipster posturing to be quite annoying. I just think that in an industry that's still developing trying to pigeonhole it is pretty counter-intuitive.
This is a clickable link to my Steam Profile.
Its fine the way it is and I don't think we've had many issues with it. Not to mention its just a poll that doesn't really mean anything to begin with, so why try to add needless complexity?
I don't find the question of what is or is not a "game" very interesting
It also eliminates a lot of that "oh geeeeez, I can't decide between these two!" factor. If more than one game deserves a top score, in your opinion, you can rate them that way, but you would also have to be aware that doing this too much would water down your overall impact.
Take the "But is it art?" bullshit elsewhere. For the sake of this thread, anything on the list is a game and I don't need @Jdarksun and @TychoCelchuuu giving me a Siskel & Ebert routine.
Some of us like discussing how to improve polling data through the use of statistical analysis. I certainly don't expect any discussed changes to be implemented nor would I be upset if they aren't, but if you don't want to talk about it with us then maybe just... don't?