11

We introduced canned custom off topic close reasons for common reasons to ease community moderation.

These are not meant to replace individual close reasons, and they should not be the only reasons we have on reviewing.

Increasingly we observe people to choose such a canned reason for questions where this reason is only marginally correct. This alone would not matter much, as we need 5 close votes to put a question on hold.

But

Sadly we do have enough robot reviewers that blindly follow the first vote. This leads to closing questions with a wrong reason. It does not help anybody to do so, even less if is accompanied with closing without comment (that occurs often too).

This is a network wide issue which led to the introduction of review audits to be able to ban such robo-reviewers.

We are a small site, still in beta, hence probably won't need such audits but moderating robot-reviews is not easy. We may not detect robot reviews easily and we do not want to ban the wrong people from reviewing.

Any suggestions on how we should deal with wrong close reasons and robot reviews are welcome.

1
  • 1
    Just now I wondered if when watching wrongly motivated VtCs roll in whether "flag for mod" might be appropriate?
    – LаngLаngС
    May 16, 2018 at 22:48

3 Answers 3

Reset to default
2

2 ideas:

a) Select, let's say, 5 wrongly closed question by hand. Look, who closed them. If a voter appears at least 3 times, PN them, and ask for improved performance.

Depending on how big the problem is, and how many sloppy closers there are, adjust the thresholds or repeat in longer or shorter intervals.

b) Let an algorithm protocol, how often a person follows the reason given so far. If there is never a divergence from the uniformly given reason, act as in a).

1

In case we continue to see many such Robo reviews with canned close reasons that do not fit to the questions we will need to inactivate such frequently abused canned close reasons.

Because we do not want to do this we consider to take other measures to avoid sloppy reviews. This also includes an earlier issue of review bans.

6
  • 1
    Wouldn’t it be better to review-ban the offending reviewers?
    – Wrzlprmft Mod
    Jun 27, 2018 at 8:57
  • Why do you assume, that these are always "robo reviews" which "blindly follow the first review"? According to SE and SO this are "community driven sites". Now five members with a certain rep vote to close a question. You have a different opinion if a close vote fits or not. But istn't it imaginable, that these five members didn't erred and have the opinion that the chosen close vote is correct?
    – IQV
    Jun 27, 2018 at 9:38
  • A threat of review-ban from the mods means for me, that I do not review furthermore. That to "communitiy driven", then SE can change to classical moderated Q&A-site.
    – IQV
    Jun 27, 2018 at 9:41
  • @IQV We sadly have too many inconsistent votes here. We have to do someting against that. If your vote is correct then there is nothing to worry for you.
    – Takkat
    Jun 27, 2018 at 10:03
  • 1
    "If your vote is correct"? Who determines that? Apparently the first in a row of those reviews makes a 'wrong' choice. Is this exempt? How are genuine errors following that detected? – Is the time used for bringing up the reviews and choosing a VtC reason somewhere measured? I guess that would be the most machine readable data to gauge the sincereness of the reviewer?
    – LаngLаngС
    Jun 27, 2018 at 16:22
  • @LangLangC I do understand your concern but please do not worry too much. It should be an absolute exception. The whole point of community moderation is that we need 5 community members who agree on a vote. If only 3 of them blindly follow the first wrong vote the whole democratic community process is in danger. Only rare and clear-cut cases are affected by a potential ban. This is nothing uncommon, the whole SE network suffers from that and it is the main reason for review audits on many sites. Take it as a service of us moderators for you to avoid robot auditing.
    – Takkat
    Jun 27, 2018 at 17:12
0

Force users to comment on their vote:

Leave the voting system as it is, but put the “Other” text field on the screen that first appears when clicking the close button and make it mandatory (in addition to the choice between one of the other options).

This would hopefully result in a more reflected voting behaviour.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .