Reviews - your input needed!

Information from and to the site administrators.

Moderator: Alastair

Post Reply
Message
Author
User avatar
Gunness
Site Admin
Posts: 1939
Joined: Tue Dec 07, 2004 7:04 pm
Location: Copenhagen, Denmark
Contact:

Reviews - your input needed!

#1 Post by Gunness »

The new site is approaching a state where it can go live, which makes me endlessly happy.

One thing that still needs some work is the reviews section, which I've been working on with kind assistance from Alastair. So I'd very much like your input!

The current review model allows ratings in the following categories:
- Vocabulary/parser: how well does the game understand you?
- Writing: originality and atmosphere. Any graphics are mentioned here, too - which makes the category name a bit misleading, but it was the best we could come up with :)
- Puzzles: quality, originality and logic
- Fairness/cruelty*
- Overall

Any opinions on the above?

Our major problem is the fairness/cruelty rating. I think it's an excellent idea, I'm just uncertain of its exact content.
My first thought was Andrew Plotkin's

Cruelty scale:
- Merciful : Cannot get stuck.
- Polite : Can get stuck or die, but it’s immediately obvious that you’re stuck or dead.
- Tough : Can get stuck, but it’s immediately obvious that you’re about to do something irrevocable.
- Nasty : Can get stuck, but when you do something irrevocable, it’s clear.
- Cruel : Can get stuck by doing something which isn’t obviously irrevocable (even after the act).

Good points: It's nice and clean, and it's used on other sites so it'll be easy to understand for a lot of people
Bad points: The scale was created for modern games. An overwhelming lot of oldies will be "cruel", and very few will be "merciful" or "polite"

Alastair's alternative was
Winnable/Unwinnable:

- Always winnable: you can never get into a situation where you cannot complete the game, and you cannot lose
- Unwinnable after obvious warnings: e.g. drinking a labelled bottle of poison; an unsafe bridge is clearly marked as such; deliberately smashing a priceless vase you are meant to return
- Unwinnable after little or no warning: e.g., sudden death without warning; a priceless vase breaks when you put it down
- Unwinnable with no indication: e.g., failing to carry out an action in a location that cannot be reached later on; pulling a lever which you later learn smashed that priceless vase

Frustration:
- Little or no frustration: e.g., few, if any, random elements; objects stay where you put them; ability to carry a large number of objects
- Frequent frustration: e.g., elements that needed to completing the game being random; random/unmappable mazes; objects often moved to non-obvious locations when dropped; very limited carrying capacity.

Which of these work the best?
Mr Creosote
Posts: 1121
Joined: Tue Sep 22, 2009 9:23 am
Contact:

Re: Reviews - your input needed!

#2 Post by Mr Creosote »

Are "winnable" and "frustration" distinct proposals or are those two facets of the same fairness model?
User avatar
Gunness
Site Admin
Posts: 1939
Joined: Tue Dec 07, 2004 7:04 pm
Location: Copenhagen, Denmark
Contact:

Re: Reviews - your input needed!

#3 Post by Gunness »

They are distinct and can be used separately. He thought about combining them but in my opinion it over-complicates matters. Since Frustration only has two values, it probably makes the most sense to just include this info in the main review.
Alastair
Posts: 1151
Joined: Fri Nov 11, 2005 12:21 am

Re: Reviews - your input needed!

#4 Post by Alastair »

Jacob,
you forgot to mention that a problem with "Winnable/Unwinnable" (apart from the ugly terms - does anyone have better suggestions?) is that unless you have completed an adventure it can difficult to know if a game is "always winnable."

Note that the "Frustration" scale can be expanded, I reduced the size so that combined with "Winnable/Unwinnable" there would be a small number of ratings, in this case eight. If "Winnable/Unwinnable" and "Frustration" are separated then that limit no longer applies.
Mark
Posts: 90
Joined: Sun Feb 27, 2005 7:04 pm
Location: Germany

Re: Reviews - your input needed!

#5 Post by Mark »

I think the cruelty scale isn't all that bad. May be "invented" for new games, but as far as I remember the games I played, it's applicable enough to all of them. Personally, I would go for the cruelty and the frustration ratings. I'm sure you will provide a freetext box for any additional information (like "random monsters") anyway.
Mr Creosote
Posts: 1121
Joined: Tue Sep 22, 2009 9:23 am
Contact:

Re: Reviews - your input needed!

#6 Post by Mr Creosote »

I thought the main reason against using the cruelty scale was that it might not be suited too well for classic games. Though does the winnable/unwinnable scale solve this? As I see it, "unwinnable with no indication" is "cruel", "unwinnable after little or no warning" is "nasty", "unwinnable after obvious warnings" is "tough" and "always winnable" is "merciful". Meaning the only step it leaves out is "polite" (which would fall under "unwinnable after obvious warnings"). Though maybe I'm misinterpreting the winnable scale?

If my assumption is correct, though, and there is little difference between the two, I'd say go for the cruelty scale, simply because it's, well, established.

I don't see the requirement of having finished a game to assess it on any of the scales as much of a problem. If I understood it correctly, it will be entered by a reviewer whom I'd expect to have finished the game he's reviewing anyway.

By the way, I think it's good that we're talking about 'verbal' scales instead of numeric ones here. Assessing fairness on a numeric scale would leave too much room for interpretation (what does "5" mean?).

Last, but not least, you already told me that, Jacob, but I think it's a crucial point, so I'd like to repeat it: If the overall rating is (as planned) not an automatic (mathematical) result of the other ratings, but can still be given freely by the reviewer (e.g. the reviewer can decide to give a game the best overall rating in spite of a "bad" fairness rating), it's not necessarily a disadvantage for old games. It's just honest to admit that many classic games were "cruel".
Alastair
Posts: 1151
Joined: Fri Nov 11, 2005 12:21 am

Re: Reviews - your input needed!

#7 Post by Alastair »

Re. Cruelty and Frustration
When I first thought of what was originally called "Fairness" I was thinking more in terms of frustration rather than winnable/unwinnable, in fact I was not even aware of Andrew Plotkin's scale. I have no problems with games where you can die or get into situations where you cannot complete the game. I have problems with games where death occurs without warning, especially if this happens repeatedly throughout the game, or because you typed in something the parser did not understand and you are not given an opportunity to type in something else, especially frustrating if the word the parser does not understand is a synonym of a word that is accepted as 'right'[1]; where you have to traverse the same 'few hundred' locations several times over because the author thought that that was legitimate method to increase the length of the adventure; where mapping of mazes becomes impossible because objects disappear when you drop them; and where important parts of the game are random in occurrence so you spend inordinate amounts of time waiting or wandering around before you can progress.

There are many more examples of frustration that can be given, and no doubt everyone can add to the list. It is just my opinion that frustration that is not due to the puzzles in the adventure is more likely to determine if you will enjoy the game, rather than whether-or-not you can die or get into other situations where you cannot complete the game.


[1] Not to be confused with a limited parser which comes under "Vocabulary/parser," the frustration outlined here is that you have one opportunity to type the correct response and if you type in something the game does not understand then that one opportunity is gone.
Alastair
Posts: 1151
Joined: Fri Nov 11, 2005 12:21 am

Re: Reviews - your input needed!

#8 Post by Alastair »

Gunness wrote:- Writing: originality and atmosphere. Any graphics are mentioned here, too - which makes the category name a bit misleading, but it was the best we could come up with :)
Weren't we going to call it "Atmosphere"?
User avatar
Gunness
Site Admin
Posts: 1939
Joined: Tue Dec 07, 2004 7:04 pm
Location: Copenhagen, Denmark
Contact:

Re: Reviews - your input needed!

#9 Post by Gunness »

Actually, I forgot what we ended up with. There are pros and cons to both "atmosphere" and "writing", as neither label really covers everything it should. I'm open to both of them.
User avatar
Gunness
Site Admin
Posts: 1939
Joined: Tue Dec 07, 2004 7:04 pm
Location: Copenhagen, Denmark
Contact:

Re: Reviews - your input needed!

#10 Post by Gunness »

Mr Creosote wrote:Last, but not least, you already told me that, Jacob, but I think it's a crucial point, so I'd like to repeat it: If the overall rating is (as planned) not an automatic (mathematical) result of the other ratings, but can still be given freely by the reviewer (e.g. the reviewer can decide to give a game the best overall rating in spite of a "bad" fairness rating), it's not necessarily a disadvantage for old games. It's just honest to admit that many classic games were "cruel".
Indeed they were, often overwhelmingly so :) And yes, the overall rating is completely as dictated by the writer.

As for fairness vs. cruelty, that rating should be supported by whatever's written in the main body of the review, so at the end of the day I think either will work. It's just a matter of deciding which is better.
User avatar
Gunness
Site Admin
Posts: 1939
Joined: Tue Dec 07, 2004 7:04 pm
Location: Copenhagen, Denmark
Contact:

Re: Reviews - your input needed!

#11 Post by Gunness »

Ok, we'd better move on with this so the site can get up and running :)

As has been pointed out, the winnable scale has a number of similarities with the cruelty scale, and as the latter is fairly well known, I think we'll just stick with it.

Thanks for the input, everybody.

Edit: The "writing" category becomes "atmosphere" instead, as suggested by Alastair. I think it makes sense.
Post Reply