Multiple scores

[Spriggan]

Why the hell can't you just ask people before making such a change?  Seriously how hard would it be to ask  someone "do you think your score is adequately reflected in your review?" and then point them to the review criteria.  If you do that then people will LEARN the system instead of relying on you to do it for them.

[Skar]

True, but those that really want to do this are willing to improve and work with the people at the site.  What I don't think we need are those that come in and say our methods are stupid and we should use theirs or not care enough to put in any effort.

There have been several people that have submitted VG reviews to us but weren't up to par, they would have like 200 words for a 5-6 clock review and I'd contact them and explain what I liked about the review and what they should add to make them better.  Basically following Fell's guidelines and then giving us a review that was average length.  None of them write for us anymore probably because they realized they had to put some effort into this and we are serious about having good reviews.

No one expects someone's first few reviews to be perfect or even good, be we should expect a reviewer to challenge themselves to do the best job they can.

Dude.  Are you really saying that the Gibbs Brothers, those who pointed out the problem with our review scoring system, don't write good reviews or that they haven't put much effort into it?

I disagree.  I think they write great reviews, have put a lot of effort into not only writing them but getting them to us in a timely fashion, and they have sent a couple of good articles above and beyond as well.  Not only that but they've only been able to take advantage of press screenings once or twice.  It's not like they're getting loads of free stuff from us to offset the work they're doing.

And on top of that I think they've got a point.  And I've made my understanding of that point pretty clear so are you talking about me?

[The Holy Saint, Grand High Poobah, Master of Monkeys, Ehlers]

Why the hell can't you just ask people before making such a change?  Seriously how hard would it be to ask  someone "do you think your score is adequately reflected in your review?" and then point them to the review criteria.  If you do that then people will LEARN the system instead of relying on you to do it for them.

You've done it. You KNOW how difficult it is to do at times. I've waiting *weeks* for people to get back. People aren't interested in doign a second draft. That's fine, I'll make the second draft if they don't get to it. That's my JOB. I should be allowed to do it. I'm not saying it should be done on every review, but when it comes to the content of hte article, the clock score is a little thing in terms of creative control. You're so anal that you would quit a site over that small change? I seriously cannot see what the problem with it being understood that the editor has the power to make a necessary change in an rticle when he can't get hold of the author in a timely fashion. Seriously. I can talk to them after the fact if I have to make the change. But I seriously think that since this sort of control is technically already in the editor's power, it  is not unreasonable for him to be allowed to exercise it.

[Mad Dr Jeffe]

I am of the opinion that any wholesale change to our rating system would ideally involve dropping numbers altogether and substituting adjectives: Poor, Fair, Good, Excellent, Perfect, etc. (though of course we'd have more than that). That would make it obvious what we mean, without the need to quantify artistic expression.

Now this is something I can get behind. Frankly I'm skeptical about the merits of 5 being more intuitive than 6 ... no offense meant E but without any real data other than you saying 5 is more intuitive, its a bit of a hard sell. Now if you do indeed have a little data supporting it then I'd love to see it. I mean the whole point that all of us are missing is that we all love the site we've had a hand in creating. Because guess what, as writers and editors we all have an emotional stake in TWG. I think we can agree that the site is special to us all and that we want to see it get better and not worse. We're treating several issues like they are just one issue and that seems to be a big hangup right now. The first issue is that some people misunderstand the six clock system. I have to admit that it seems pretty easy to figure out from my point of view. I mean it is just a stepping stone to the review. However some people are of the opinion that its hard to fathom. Ok, I'll buy it, but changing overnight or displaying two seperate review styles isnt going to make it any easier for our readers to get it. The way I've seen it the 6 clock system has been a site trademark for a while mainly because its unique. There are star systems out there (for 1-4 or 1-5) stoplight systems, thumbs up or down systems and so on. I've always seen our 6 system (with half clocks) as a this site goes up to eleven kind of review. Its a bit rebellious and pretentious at the same time... pretty much like us. To me just abandoning it for another system thats frankly just as arbitrary betrays the spirit of the site. Im trying to decide if thats a good or bad thing though...
It could be very good
or it could be very Comicbook Guy.

Im leaning toward the opinion that changing our system from 6 to 5 clocks just looks arbitrary and lame. Having said that, I would definately fall behind a more descriptive and less arbitrary system ala Fell's adjuctive mania system. It avoids pointless bickering over stars as adjectives are more precise and even friendly.

Our second problem though is more insidious.
For a review site, we rarely give bad reviews. Oh sure we give some, but the bulk of our reviews come off as fannish.
This is because as Stacy says we tend to review stuff we like and only pan stuff when it surprises us with how bad it is. Im not entirely sure how to combat that. Some of it is caused by the Game Company sending us material... which is something that has concerned me a bit. I think we should have to disclose donated or review material status. If Wizards gives us a book, we should say it, if we bought it ourselves we should say it. I think that this might make us think about the product a little more when we review it (and it will definately make the readers a little more critical). I feel like I can talk about it because I feel a little conflicted when I review books I bought myself instead of say a book given to me to review from Atlas. I mean if I love Castle Falkenstein (and I do) how objective can I be? I honestly dont know.
As reviewers we dont seem to take criticism well. This has been seen most recently in Movies but it affects all aspects of our forum system. We need to be a little more relaxed and a little less dogmatic when it comes to discussion... yes even me

[Mad Dr Jeffe]

Our third problem right now is stagnation.

Yes Stagnation.

We used to be one of the best Webcomic review sites out there, untill people stopped writing about them. I know some people on the site read more webcomics than almost everyone out there and I dont know why that happend.

In the RPG front, we get fewer and fewer products to review. Im not sure if thats the fault of the editores responsible for getting them, or if game companies have changed their policies. It needs to be fixed if it can.

I think it needs to be said that movies is our most active section.
Kudos to Skar and the Gibbs brothers for doing that work for us. It must be very taxing to  watch all those movies
Seriously though they do a great job and I want them to know that this debate about scores isnt a shot at them. We like their reviews a lot.

I think we could go to a more organized release schedule though if we gave assignments to willing writers. Especially ones who have delivered in the past.

We definately should think about stockpiling some reviews for slow weeks in a kind of dead file. (Especially webcomic reviews). I think our regulars could be trusted to dash out x amount of words on a deadline and not only that I think they might really like the challange.

Also maybe we should put a minimum word length on reviews...

Just a thought.

[Lightning Eater]

I agree with e that a six-clock review system isn't as easy to understand, because lots of people are used to systems on 10 because of fact that the marks in school and most competitions are on 10 and reviews on five can easily be converted.

But I also agree with Mad King WencesJeffe that the six clock system is basically a symbol of the site, and I definately agree that it shouldn't be taken away.

My personal opinion is that the clocks should be kept, except with adjectives like Fellfrosch had suggested

I am of the opinion that any wholesale change to our rating system would ideally involve dropping numbers altogether and substituting adjectives: Poor, Fair, Good, Excellent, Perfect, etc. (though of course we'd have more than that). That would make it obvious what we mean, without the need to quantify artistic expression.

This way you wouldn't have to spend the time changing the code of the site or editing old reviews, and with descriptions of the score people who find the present system complicated would be able to grasp the six-clock system after only reading a few reviews, from seeing the corresponding adjectives to the number of clocks.

It also solves the problem mentioned by e earlier, of companies no longer helping the site because of misuderstandings about their thing getting a 3 1/2 or 4, since it'll stop people from assuming 4 or so is a bad score.

Also, it accomplishes what the recent double rating system was trying to accomplish without all the recent systems drawbacks.

[Skar]

I think we should keep the clocks.

In the end we have to deal with the fact that we don't exist in a vaccuum.  No matter the subject matter, other review engines hand out full marks, whether it be 5 out of 5, 4 out of 4, or 10 out of 10 to films/games/what-have-you, that we, according to the current definition, would not give 6 clocks.  

Out there in the world full marks does not usually equal world-shattering, genre-forging perfection.  It equals a darn fine piece with minor problems, if any.  Now, don't get me wrong, conformity for conformity's sake is not my cup-of-tea.  If we want to hold to the standard and keep our 6 out of 6 system with a blank clock or more hanging off the end of nearly every review, and force the world to decipher our code, I'll play along forever.  I think it may be a bit of a reach though.  

Perhaps we should bow to the wisdom of an outside perspective (the Gibbs brothers are fairly new to the site but long in the tooth when it comes to the movie review world) and admit that there may be a problem with having an empty clock on every review.

My solution, 5 clocks shown, empty or full, on all reviews with the extra half or full clock tacked on for the truly extraordinary pieces, the 11s as it were, keeps the 6 clock system (nearly intact) doesn't really require a change to old reviews since the standard would essentially be the same just the visual representation would change slightly from here on out, and suddenly, voila, run-of-the-mill perfect scores would look perfect and at the same time be in line with the intuitive standard set by most of the other review engines out there.

I'm not advocating getting rid of the 6th clock altogether, just giving it the visual significance it deserves as a special ultra nifty case.

What doesn't make sense about that?  I really want to know.

[The Jade Knight]

My vote:  We keep the 6 clocks as they stand, but we also add adjectives to describe (sum up) what the clock rating stands for.

I'm pretty sure that that would solve most of our problems, surprisingly enough (except, of course, issues like stagnation or some score inflation).

[Spriggan]

Dude.  Are you really saying that the Gibbs Brothers, those who pointed out the problem with our review scoring system, don't write good reviews or that they haven't put much effort into it?

Nope, not at all.  Their first review, catwoman, wasn't all that good but it was their first one and they've really improved.

Just to get this out of the way so everyone understands this, there has only been one person that has written for this site that was bad, ie horrible writer, didn't take it serious and other things and he DOSEN'T write for us anymore.  It was quite a while ago, and I have no problems with anyone currently writing for us.

I am annoyed at the Gibbs wanting to add their own system on top of ours, they didn't point out a problem they instead just said we like our method better we're going to use that instead.  They should have started a thread like this if they didn't care for our system and discussed it first with all of us before doing the multiple score system.

[Spriggan]

You bring up some good points, and I wanted to address some of them.

Our third problem right now is stagnation.

Yes Stagnation.

We used to be one of the best Webcomic review sites out there, until people stopped writing about them. I know some people on the site read more webcomics than almost everyone out there and I don't know why that happened.

They got busy, ie real jobs that took time.  We're a volunteer site, Fell doesn't pay us so things only get done with people have the time.  EUOL, SE and Tage all have better things to do then reading through hundreds of comics just to write one review.  I agree we should have more, 42 has tried to get a few done for us which is great, but they do take a lot of time.

In the RPG front, we get fewer and fewer products to review. Im not sure if thats the fault of the editors responsible for getting them, or if game companies have changed their policies. It needs to be fixed if it can.

It's not our Editors, it's the companies have changed their policy.  I know Fell tries to get stuff as often as he can but some, like wizards, have changed how they send out review materials and we just aren't big enough to get stuff anymore.  With other companies it's just because they're small, they've got few people working on this and when the higher a new PR person things get lost in the change or misplaced.  That happened several times this year just like it did last year, the convention seasons are notorious for this.

I think we could go to a more organized release schedule though if we gave assignments to willing writers. Especially ones who have delivered in the past.

We've tried this before, and honestly lots of me and SE's current argument about the score changes would be easier to handle if we had such a schedule and people actually turned things in several days before the actual publish date.  But, as I stated in the first reply of this post, we're volunteer and all our Editors realize that it's a lot to ask people to stick to a schedule when we've all got lives outside this site.  Heck with myself I've got 3 jobs (including TWG) and school I'm dealing with and since my third job dosen't pay me money I have to relegate it to the bottom of the priority list.  It's the same way with lots of people, jobs, school, family, they all take more time and priority over TWG.

We definitely should think about stockpiling some reviews for slow weeks in a kind of dead file. (Especially webcomic reviews). I think our regulars could be trusted to dash out x amount of words on a deadline and not only that I think they might really like the challenge.

We've tried to do this too, but it always turns out one or two people writing 4+ articles a week and then they get burned out and don't write anything for a few weeks.  Seriously with only about 6 regular writers that's a lot to ask people when many already write 1-2 things a week (any many of us haven't had time this winter to even do that).  It's a great idea, but again it's back to the volunteer problem.

Also maybe we should put a minimum word length on reviews...

I've always thought this too, but have never pushed for it since I know that people do what they can.  My personal criteria is 800 words.

[Spriggan]

You've done it. You KNOW how difficult it is to do at times. I've waiting *weeks* for people to get back.

I understand this, and you're right I've done it.  But that doesn't mean we still shouldn't, and you know what if you contact someone and a week later still no reply then I have no problem with you changing the score.  I still think there should be some effort on the part of the editors to inform people about how to write better articles.  And honestly, if you ever ask me, jeffe, or the bro Gibbs I know all three of us would reply very quickly so would any of the regular reviewers.  And those are the ones that it would be best to try and help follow the guidelines.

[The Holy Saint, Grand High Poobah, Master of Monkeys, Ehlers]

I'm more than happy to try first. I just want it known that I reserve the right to change if someone doesn't get off their lazy butt and get back to me.

We think in terms of base-10 numbering, Jeffe. It's a major reason why Arabic numbering is so much easier to use than, say, Roman numbering. Percentages are difficult to figure from a base-6 system. Base 5 makes more sense because of it's relation to 10, and, when you do half-clocks, it's base-10 anyway.

In my opinion, sticking with 6 clocks just because it's different is sort of like wearing that offensive T-shirt simply because your mom doesn't want you to. It is, in Jeffe's words about changing, is arbitrary and lame. I have thought of the clocks as a trademark, but not specifically the 6 clock system. And if it doesn't work, well, that's all the more reason to change your identity.


Look, I'm the associate editor. It's my job to get articles. That means frankly, I couldn't give a whit what your emotional investment is when you aren't contributing. Right now, the Gibbs account for about 30-40% of our content over the last two months or so. That's at least 3 times as much as ANY OTHER CONTRIBUTER. Thus, I'm willing to give their impression that the system doesn't work 3 times as much consideration as anyone else's. I want to make it as easy as possible for new people to jump in and get started with us. If we made a lot of money and weren't frantically searching for content every morning then I'd be a bit more demanding and make them learn us. But since we get so little content right now, I bend to THEIR whims.

If the rest of you were giving me that much, I'd be fine with keeping it even thought I *know* it's an awkward system. As it stands, I can't believe we're even considering not fixing it from the ground up.

[Spriggan]

Well I have to say I'm glad the final decision isn't up to you SE since I find it offensive that you'd put more weight on the opinions of the "flavor of the month" then people that have been with the site as long as you or longer.

And keeping the 6 clock system is nothing like the wearing an offensive tee-shirt just to annoy people, that's insulting too (there's nothing offensive about our system), it's like wearing a stylized tee-shirt to show off your personality.

[The Holy Saint, Grand High Poobah, Master of Monkeys, Ehlers]

Only the Gibbs don't show any sign of slowing down unless *we* hinder them.

Look, I know why people don't contribute as much. It's for the same reasons that *I* don't contribute as much. There's too much other stuff going on: work, school, family, whatever. But if we drive them away, we'll have nothing to run.

[Spriggan]

I know you understand, heck we all know how busy this year has been for you.  I don't see how asking them to keep to our scoring system is hindering.

This will be my last post on the topic, I'm leaving for Christmass break and frankly just don't think there's anything else that can be added to the fray, by me at least, I'll just wait for Fell's decision and keep my trap shut.