Modifications to Mortal Reviewers

Xenthos

Nearly every designer has run into this; you submit a design, it gets rejected for some fix, you make the fix and resubmit it...

... and then it just sits there forever, because it requires a certain threshold of "yes" votes and a reviewer cannot vote on it again.

This setup is very frustrating from a designer's standpoint, because if you get it right the first time it will breeze through, but any error will get your submission stuck in limbo for (potentially) weeks.

A suggestion I read on Facebook from Nymerya would adjust the system so that designs can be improved by cooperation between the reviewers and designers instead of sending them to the Void.

When a reviewer rejects a design, DO NOT prevent them from voting on it again. This way the designer and reviewer can discuss things in comments until the issue(s) are resolved.

Pay the reviewers at the end of the process instead (once it is accepted). I imagine the fear was that reviewers would just blanket accept things in order to get the reward, but my understanding is that the reward is minuscule and any reviewer doing that would need to be replaced anyways (it should be pretty obvious).

Another simple step would be to just remove all votes on the design when it is rejected, so as to encourage reviewing changes when it is resubmitted.

Lerad

That or recruit more slaves, I mean Ephermals to do reviewing duty.

Eritheyl

"Pay the reviewers at the end of the process instead (once it is accepted). I imagine the fear was that reviewers would just blanket accept things in order to get the reward, but my understanding is that the reward is minuscule and any reviewer doing that would need to be replaced anyways (it should be pretty obvious)."

I'm not entirely sure what you're asking for, here. Reviewers are already not paid until the threshold of designs are accepted/rejected. Do you mean make rejections count for nothing in terms of reward for the reviewers' work put forward?

Lavinya

I would suggest simply having all comments remain visible to all reviewers, whether it's still in the queue or back for a second look. That should hopefully remove the big complaint of being told 'change x to y' then it being resubmitted and rejected again for 'change y to x'.

All comments can be cleared by the trademaster even after a design has be approved. (This is already an awesome thing)

Also, sometimes a design will be rejected multiple times, and not because the reviewers got it wrong. Sometimes they are resubmitted with different typos/errors, sometimes they don't have certain errors corrected, etc. Making these not count towards payment (because it is not a payment per design reviewed, but per multiple designs successfully rejected/approved) you're providing even less incentive to do the work. The pay is peanuts as is.

Xenthos

Lavinya said:

I would suggest simply having all comments remain visible to all reviewers, whether it's still in the queue or back for a second look. That should hopefully remove the big complaint of being told 'change x to y' then it being resubmitted and rejected again for 'change y to x'.

All comments can be cleared by the trademaster even after a design has be approved. (This is already an awesome thing)

Also, sometimes a design will be rejected multiple times, and not because the reviewers got it wrong. Sometimes they are resubmitted with different typos/errors, sometimes they don't have certain errors corrected, etc. Making these not count towards payment (because it is not a payment per design reviewed, but per multiple designs successfully rejected/approved) you're providing even less incentive to do the work. The pay is peanuts as is.

You would get paid at the end (once it's accepted), not based on # of times reviewed (currently you can only review once, so you get the pay whether you accept or reject).

That is the intent, at least.  I don't really care if a design gets rejected multiple times for valid reasons.  What I care about is when the design is rejected multiple times and then just sits in limbo forever afterward because there are not enough active reviewers to let it finish the process.

Lavinya

This can also be adjusted with my proposal.

At present, it takes a minimum of two reviews for a design to go through (sometimes needing a third). If you have one vote to reject and one to approve, it sits, waiting for more reviews. One more vote isn't enough to counter the other. So you can potentially see a design with 5 reviews already on it, two for one, 3 for another (and yes, it does happen), and since no one can see any of the comments or votes on it, we have to guess at why it's sitting there. Am I missing something I should reject it for? Is someone rejecting for something they shouldn't be? Then when it finally goes through, and if it comes back, there's a lot more reviewers who can't look at it again. And THAT is why they can sit forever. There are a lot more than two reviewers, but with the present system, a good chunk can get caught up on one review. And what if it happens the second time? Often it will sit until a divine comes and puts the final say on it.

Make it so we can see all the comments, and it's less likely to happen. Oh, I did miss that spelling mistake, I shouldn't approve it. Or the reviewers can discuss if something is too picky, or shouldn't be rejected, because we can see the comments. Instead of not knowing until it's decided anyway, and going 'oh, oops, well too late to change it now'.

Eritheyl

I imagine the fear with seeing all comments is "Oh, well they're not going to actually read the design, just copy the rejections/approvals present." To which I have to say shame, shame, shame. I feel like our reviewers know better, and wouldn't twist this around for the sake of, as you said, peanuts.

+1 @Lavinya.

Idrazil

I hope at least one of my three pending designs will be approved before deadline for Beauty, being quite a few days already.