Improving moderation process

The idea to improve mod queue came up as a part of discussion in topic #11667. While there are certainly other issues contributing to both regular members and approvers dissatisfaction, the time-consuming nature of approval procedures and lack of feedback on rejected posts are among the reasons as well. Moderation queue is an old part of Danbooru, and it should probably undergo some renovations to make moderation process faster, easier and more satisfying to uploaders and approvers.

First of all, for all the users who never saw it, that's what the moderation queue actually looks like: http://imgur.com/3AEmZnU
Posts are ordered by their ID, so posts uploaded earlier are shown first, and older posts that are flagged always end up at top of the queue. Each approver has the possibility to hide the post from their queue, so that it doesn't show up for them anymore, unless they request it with "Show hidden posts" link. There's a counter for every post showing how many other approvers have hidden the post.
This is what "status:pending" box looks like for Janitor+: http://imgur.com/LhzNdOw

I'll summarize the ideas from topic #11667 in the second post. All Janitors+ are welcome to share their ideas and opinions whether the enhancements are necessary at all. Other users are welcome to contribute as well, but please keep it reasonable. Again, this topic is mod queue improvement discussion, it's definitely not for calling out names or getting angry at the system.

So, getting to the ideas.

I feel that there's lack of communication among approvers, and between approvers and uploader. Currently, approvers only see "Hidden" counter in the queue, which doesn't even mean much, because approvers doesn't necessary hide only 'bad' posts, but may rather avoid certain themes based on personal tastes. Uploaders get zero feedback from approval process until the post is deleted from the queue, often resulting in questions whether it's been simply overlooked by the staff.

So, one idea is to add ability to not just "hide", but outright "reject" or "disapprove" the post if the approver is opposed to it passing the queue. Disapproval counter would either be added to or replace Hidden counter. That way, disapproval should gain more weight, ideally urging other approvers to pay more attention before approving a post that others oppose to. That might soften or end arguments between approvers, which currently happen post-approval on posts of questionable quality. The same counters could also be displayed to uploader on the post page, actually showing that some approvers thought "it's not my thing", and some others actually rejected it as bad.

To complement the first idea, it's possible to introduce [dis]approval comment system, explaining the reason or disapproval. Such comments would either be chosen from a list predefined options, or written manually, or the combination of two. The comments would be visible to other approvers and the uploader; whether to show the comment author and whether to show the comments to other users is debatable. Hemoglobin has also suggested binding keyboard shortcuts to keys ~ through 0 to certain predefined comments - though I'm not sure how's that going to work in the queue with multiple images.

NWF Renim has suggested score-based auto-approval and auto-flagging/deletion system to ease moderators' work, but that idea requires polishing if it's going to be implemented, as pointed out by other users.

Log also mentioned that not all approvers use the queue; some use status:pending searches instead to find things they like and approve these. This should be kept in mind when adding features. Also, that fact is an indirect signal that queue page needs other improvements to become more useful.

Additionally, Hillside Moose suggested changing the approval system to require two approvers per post (see forum #105355). I'm mentioning that idea here, but it seems to be a major change which is bound to cause debate, so perhaps it should be discussed somewhere else. Albert flat stated this isn't happening.

That's the gist of things for now; perhaps I've missed something, though.

Type-kun said:

Additionally, Hillside Moose suggested changing the approval system to require two approvers per post (see forum #105355). I'm mentioning that idea here, but it seems to be a major change which is bound to cause debate, so perhaps it should be discussed somewhere else. Albert flat stated this isn't happening.

Where does he say that? At best he made a wishy-washy statement about how it would make it more obvious there's a two-tier score system, one for mods and one for everyone else. Which is the point, because what people upvote is so arbitrary it can't be used as a quality metric until you get around maybe 10+ votes.

It's not a "major change" either. Sure, some UI improvements to add two names to the sidebar and maybe some visual aid signaling if a post is initially approved or not would take some work, but it's just adding one more Janitor to approve a post.

Currently the balance between approving a post and rejecting a post is extremely skewed towards the former. You can mash your face into the keyboard and approve the whole mod queue, while rejecting a post requires waiting for three days and hoping some troll doesn't accept it anyway. Flagging it is even worse, as you have a limit to how many flags per day and you're subjected to harassment from the butthurt brigade, hopefully passing without it getting reapproved by the same troll again.

AT MINIMUM, I would want to see this change: Ban Janitors from reapproving their previous approvals. I remember when everyone gave Hazuki shit for reapproving that Beedrill scribble, and since then there was an unwritten rule to not reapprove your own approvals. It's clear that's not working anymore, so it needs to be a hard-coded rule.

Hillside_Moose said:
AT MINIMUM, I would want to see this change: Ban Janitors from reapproving their previous approvals.

I'm quite positive it's already there for a year or two.

Type-kun said:

Hemoglobin has also suggested binding keyboard shortcuts to keys ~ through 0 to certain predefined comments - though I'm not sure how's that going to work in the queue with multiple images.

forum #105363, for those curious.

In the queue or thumbnail views, those keys could be mode switches. There would, of course, then need to be a key binding to switch back to normal mode.
Also, some regional keyboard variants don't have a ~ key, so that probably needs to be on - instead.

Hillside_Moose said:

AT MINIMUM, I would want to see this change: Ban Janitors from reapproving their previous approvals. I remember when everyone gave Hazuki shit for reapproving that Beedrill scribble, and since then there was an unwritten rule to not reapprove your own approvals. It's clear that's not working anymore, so it needs to be a hard-coded rule.

Janitors are already banned from reapproving a post if they're the currently listed approver.

If their name got replaced by a second approver the first one does gain back the ability to reapprove the post. But that requires at least 2 janitors and 3 separate deletions of the same post. Not exactly a common occurrence, and when it does happen an admin could always step in and manually status lock the post to prevent anyone at all from deleting/approving/flagging/appealing that post.

Wypatroszony said:

I'm quite positive it's already there for a year or two.

Ah, I see. I thought this might be the case, but the years have blended everything together. How much of an effect did it have on the overall quality of the site, you think?

Toks said:

If their name got replaced by a second approver the first one does gain back the ability to reapprove the post. But that requires at least 2 janitors and 3 separate deletions of the same post. Not exactly a common occurrence, and when it does happen an admin could always step in and manually status lock the post to prevent anyone at all from deleting/approving/flagging/appealing that post.

What are the chances of that happening? jxh2154 might have if he was still here, but Albert seems disinterested in managing the site outside of code and the weekly alias/implication sweep.

Hillside_Moose said:

What are the chances of that happening? jxh2154 might have if he was still here, but Albert seems disinterested in managing the site outside of code and the weekly alias/implication sweep.

Doubt he'd notice it happening himself but you can always pm him.

Hillside_Moose said:

Ah, I see. I thought this might be the case, but the years have blended everything together. How much of an effect did it have on the overall quality of the site, you think?

Managed to kill off a few more flagged posts than it would have otherwise, I'm certain of that. Not a significant impact (because it wasn't a very common occurrence to my knowledge), but certainly took a big worry off the shoulders when it was applied.

It was almost entirely haduki reapproving my little pony posts before that so it changed very little.

Type-kun said:

Additionally, Hillside Moose suggested changing the approval system to require two approvers per post (see forum #105355). I'm mentioning that idea here, but it seems to be a major change which is bound to cause debate, so perhaps it should be discussed somewhere else. Albert flat stated this isn't happening.

I had suggested the same idea in a PM to albert before he replied to the thread. When albert replied and said "Requiring more mods to approve a post just makes it that much less likely that a post will get approved." I had kind of taken that as meaning the idea was a no go and a response to my PM, but I may have simply been misinterpreting him.

Double teaming to push re-approvals is something that likely doesn't occur at a level to be a real issue, if it does happen it's more likely to be coincidental than intentional.

As for balance between approvals and rejections, to be honest I think the system is skewed toward rejections, it's just that in practice those utilizing the system to remove content are few in number. There are a lot more people who can flag a post than there are those who can approve or even reapprove a post.

Type-kun said:

So, one idea is to add ability to not just "hide", but outright "reject" or "disapprove" the post if the approver is opposed to it passing the queue. Disapproval counter would either be added to or replace Hidden counter. That way, disapproval should gain more weight, ideally urging other approvers to pay more attention before approving a post that others oppose to. That might soften or end arguments between approvers, which currently happen post-approval on posts of questionable quality. The same counters could also be displayed to uploader on the post page, actually showing that some approvers thought "it's not my thing", and some others actually rejected it as bad.

I strongly support this. I never knew that one janitor can hide (without a word of explanation) any post from queue - this is terrible design, as it in fact allows one janitor to impose his decision to all others (who are using queue).

NWF_Renim said:

As for balance between approvals and rejections, to be honest I think the system is skewed toward rejections, it's just that in practice those utilizing the system to remove content are few in number. There are a lot more people who can flag a post than there are those who can approve or even reapprove a post.

It's skewed toward rejections but only if it's posted by normal users. Flagging is dirty and unrewarding work, numerical advantage of the ones who can do it means nothing as most people come here to watch some pictures and that's all.

richie said:

I strongly support this. I never knew that one janitor can hide (without a word of explanation) any post from queue - this is terrible design, as it in fact allows one janitor to impose his decision to all others (who are using queue).

They only can hide the post from their own queue, not from all janitors. Unless the post gets approved or outright deleted by one, but the latter almost never happens.

Wypatroszony said:

They only can hide the post from their own queue, not from all janitors. Unless the post gets approved or outright deleted by one, but the latter almost never happens.

Ah, I misunderstood then. But nevertheless - other janitors shouldn't be aware of which posts are hidden by others as it can possibly negatively influence their decision. Introducing "reject/disapprove" decision (with explanation why) seems to be a good idea though.

richie said:

other janitors shouldn't be aware of which posts are hidden by others as it can possibly negatively influence their decision. Introducing "reject/disapprove" decision (with explanation why) seems to be a good idea though.

But listing the reasons why a post was rejected/disapproved also has the effect of letting other janitors know that the post was rejected/disapproved by other janitors. You can't have the latter without the former.

Toks said:

But listing the reasons why a post was rejected/disapproved also has the effect of letting other janitors know that the post was rejected/disapproved by other janitors. You can't have the latter without the former.

As Type-kun said - you can "hide" post for various, let's say subjective reasons and because of that such hiding shouldn't be visible to others.
As for "rejected" ones there should be some kind of strong meritoric reason so if it's really valid one then I see no problem with letting others know it.

EDIT: to be clear - I'm for both kind of action being available for janitor. Except "hide" would be strictly a private one, while "reject" not.

It would be really helpful to have any indication at all if posts were just ignored because of obscure copyrights and such, or if there's actually a problem with the quality of the art. With my scant few uploads, it's not hard for me to look at an image that fell off the queue and realize it probably wasn't good enough. Knowing if the other moderators are rejecting images instead of just skipping over them would definitely help with the perceived problem of 'last chance' 3-day approvals by a certain janitor, though!

But it has to be in a way that doesn't burden the janitors. While I like the idea of keybinds, there might be other options? I can only guess that the moderation queue displays much like an image search, with thumbnails in a grid. On pixiv's mobile site, you can click a star on the bottom right of an image thumbnail to bookmark it. Having a similar feature with two buttons for hide and reject would be more helpful than just hiding posts.

I like the idea of adding a reject button that requires a reason. But this could be abused. A janitor could just exclusively use reject or approve and then approval will be entirely dependent on when a post gets uploaded and when a janitor sees it.

But if a reject were implemented, an auto-delete after 3 days would almost always be because of mediocre quality.

I think hiding or rejecting should also count as a down vote. You're expressing disapproval, and the score should reflect that. The only problem with this is the score would be dependent on when (and if) a post gets approved, since after approval it wouldn't be subject to further down voting. Maybe delay the actual approval until after the three days? This would clog up the queue further but it would ensure any mediocre post that gets approved would start out with a low score.

Going along with that, perhaps the queue should be sorted by score. Or make it an option.

If you only care about score you can just search score:>=x but A) just because it's been upvoted doesn't mean it's a quality post I've seen plenty of poorly-drawn porn with a high score and b) just because it hasn't been upvoted doesn't mean there's something wrong with it.

Maybe some links under the search bar for some common or user-definable searches? Have them saved client-side if they're user definable so we don't have to worry about wasting server space/adding tables or whatever.

albert said:

I think hiding or rejecting should also count as a down vote. You're expressing disapproval, and the score should reflect that.

Not a bad idea for rejections, depending on the reason (e.g. bad anatomy), but perhaps hiding (or a "Not My Thing" rejection reason) shouldn't count as a downvote. As Type-kun said above, some approvers hide posts based on themes they personally avoid, sometimes for subjective reasons that shouldn't be counted against the post.