Improving moderation process

Borrator said:

Users anxious for feedback. I remember that was a huge problem in my early days, not knowing if there's some problem with a post I don't see or just nobody really cares about it.

This is resolved by showing feedback after negative result of the 3 day moderation process. Being a simple user I can relate to this, and seeing then how my post was moderated (or not) is more than enough.

hemoglobin said:

There is no way to hide this information from janitors. Once a post is older than an hour or so, janitors would be able to tell what the crowd is doing anyway, just from the fact that it hasn't already been approved.

This is simply not true. I've already seen a posts not being touched by janitors after an hour or two. Even after three days you wouldn't know if the post is being massively hidden by janitors or is it perhaps result of lower activity of the mod team.
In other words - even after three days of the post of the queue you wouldn't know if there is 20 or maybe only 2 moderators who have hidden it.

MikeTheV said:

Uh, I can tell you that zero moderators reviewed both of those posts because they were uploaded by a contributor and they have no approver.

richie said:

This is simply not true. I've already seen a posts not being touched by janitors after an hour or two. Even after three days you wouldn't know if the post is being massively hidden by janitors or is it perhaps result of lower activity of the mod team.
In other words - even after three days of the post of the queue you wouldn't know if there is 20 or maybe only 2 moderators who have hidden it.

He's exaggerating but he's right. Once a post has been in the queue for a day I can guarantee you it's been seen by 6 people who all hid it, no matter the post.

Bad examples. Must've mixed them up with something else. :X That, or they weren't contributors at the same of posting.

MikeTheV said:

That, or they weren't contributors at the same of posting.

There's no "Approver:" listed in the sidebar for those images, so the uploaders must have been a contributor at the time they uploaded it.

richie said:

This is resolved by showing feedback after negative result of the 3 day moderation process. Being a simple user I can relate to this, and seeing then how my post was moderated (or not) is more than enough.

We're going to have to agree to disagree on this whole thing. Knowing what approvers think is not good enough is helping users right now. You might not agree, but waiting 3 days to find out the image you uploaded was deleted for poor quality is not as effective in improving upload quality, as letting users know more quickly after that it isn't going to be approved.

It halts attentive new uploaders from uploading poor images immediately after. They're forced to accept that someone else didn't think what they chose was good enough and it makes them consider raising their standards. It also separates them from genuinely bad uploaders who will ignore such data, and gives janitors a good tool in consider who will be a future good contributor, builder or approver themselves.

Some users ignore scores, and they pay no heed to deletions, but those sensitive to the thoughts of others, seeing that the janitors in real time don't feel their image is good enough doesn't hurt them, it actually tells them the truth faster, and respects their time. I'd much rather know 20 minutes after I uploaded a poor picture that it's not going to get approved, than agonize over whether it will be up in the air for 3 whole days and hope someone will grant mercy upon it.

I'll say this: I agree with your statements earlier about contributors. We really need to be pickier with new ones, and I think being able to tell who is and isn't paying attention to the opinions of others is a good litmus for promoting new ones. If anything, I feel that supports this new set of procedures, because any good member that takes note of what the mods are saying to them quickly, will be better uploaders and eventually better overall site boons, and someone ignoring the 9+ poor quality marks on their repeatedly lousy uploads would not care in 3 minutes, hours or days, rendering the timing issue moot for them.

MikeTheV said:
Anyway, 2 days - I'm gonna take a guess and say....at least 7 moderators before someone decided to approve it. And given how beautiful it is, it was pretty much guaranteed anyway.

Not every mod goes through the whole queue every day. So long as they do once every three days, or at least log in to check on pending posts, they'll be seen and approved if it was a post that mod would have approved anyway. Your Ryouko image for example would have been an insta-approve for me. Being an older series and an older art style may have made it not appeal to others, and that's ok.

MikeTheV said:
It seems a bit unfair to go "Well, *I* personally don't like it, so I ain't approvin'! Myah!", doesn't it?

Which comes to this point, which is actually exactly what we are guided to do. If we don't personally like a post we aren't *supposed* to approve it. The goal is to have a wide enough variety of moderators such that, while keeping quality in mind have varied enough interests to judge the posts they are most attuned to judge filtered on their own interests.

I can understand that you feel discouraged when a post that should (and does) ultimately get in sits in the queue for a couple days with 7 "not-interested"s. Honestly though the only new bit to that format that has been here for almost as long as the mod queue has been is that it's now viewable by non mod-team people. It was pretty much *always* the case that a post would show how many people had hidden it from the queue. There is no obligation to hide things, but it doesn't make sense to keep them around if you know you're not going to approve them. That sort of passive feed back to the mod team has pretty much always been in place. Honestly like I said above it's useful if and when you are on the fence. If something is obviously good, I'll approve it without checking that score out.

Well, alrighty then. Obviously, there are more pros than cons to this so I'll take the bench with this one.

I got nothing. All I'll say is that I've already seen a good amount of quality images linger in the queue for far longer than they should have. So I really hope there's an serious effort to select approvers with a wide variety of tastes.

Log said:

He's exaggerating but he's right. Once a post has been in the queue for a day I can guarantee you it's been seen by 6 people who all hid it, no matter the post.

And I can guarantee you that once a post been in queue for a day you wouldn't know it's only 6 people or 12.

buehbueh said:

It halts attentive new uploaders from uploading poor images immediately after. They're forced to accept that someone else didn't think what they chose was good enough and it makes them consider raising their standards. It also separates them from genuinely bad uploaders who will ignore such data, and gives janitors a good tool in consider who will be a future good contributor, builder or approver themselves.

Some users ignore scores, and they pay no heed to deletions, but those sensitive to the thoughts of others, seeing that the janitors in real time don't feel their image is good enough doesn't hurt them, it actually tells them the truth faster, and respects their time.

What a sweet fairy tale. And what a pity it have so little common with reality. Personally I loved the part about users who ignore their deletions but are so, oh, sensible that seeing one or two mod who hide their post in real time to drastically review their behaviour... which in fact means NOTHING - as the very second later may come another mod and approve their picture.
Seriously, if your skin is so thin that you're hurt and worried seeing one or two mods hiding your posts then the sooner you forget about this site, the better for you.

I'd much rather know 20 minutes after I uploaded a poor picture that it's not going to get approved, than agonize over whether it will be up in the air for 3 whole days and hope someone will grant mercy upon it.

...which is of course impossible, and that's exactly why we're giving whole 3 days time to every mod to review it. And that's ok - as long as they are reviewing it independently of course...

I'll say this: I agree with your statements earlier about contributors. We really need to be pickier with new ones, and I think being able to tell who is and isn't paying attention to the opinions of others is a good litmus for promoting new ones.

You do? That's interesting because I thought I stated very explicitely at forum #105658 that we should get rid with the insane priviledge of accepting only your own posts, and replacing it with more wide priviledge of accepting others'.
The talk about being picker and stricter with chosing new contributors is - pardon my french - a typical blablablah which leads to absolutely nothing, neither is something new at this site. It's not that we didn't have moderators and admins earlier, yet even in times of such power mods as jxh2154 or hazuki I could count the numbers of contrib+ demotions at my one hand and almost all after some serious stormshit. The reason why is that is pretty obvious: keeping such watch is very unrewarding, mundane work. You must be ready to draw unpleasant consequences from people who are not the nameless noobs but active community veterans, which often leads to some serious drama and/or ragequit. And this won't be changed only by sets of new old empty promises.

Shinjidude said:
It was pretty much *always* the case that a post would show how many people had hidden it from the queue.

And it was *always* a bad idea.

Honestly like I said above it's useful if and when you are on the fence.

What a moment.
Did you just admit, that when you're on the fence you absolutely conciously take into consideration the number of "HIDDEN" done by other mods?!

Hey, Wypatroszony - what did you say about treating mods like sheep?

richie said:
What a sweet fairy tale. And what a pity it have so little common with reality. Personally I loved the part about users who ignore their deletions but are so, oh, sensible that seeing one or two mod who hide their post in real time to drastically review their behaviour... which in fact means NOTHING - as the very second later may come another mod and approve their picture.

I've explicitly said that it's something I'll disagree with you on. If you're going to pick apart my post without noting that I qualified it by stating specifically that some uploaders will always ignore that bar, that still doesn't make anything you said factually correct or fair.

Seriously, if your skin is so thin that you're hurt and worried seeing one or two mods hiding your posts then the sooner you forget about this site, the better for you.

Some people's skins ARE thin. That's awesome! Do you know what that means?

It means they'll bother to watch for when something is wrong with their upload and worry about it! I'd rather have thin-skinned uploaders who will pay attention to a "poor quality" mark on their image immediately after upload, and then stop uploading from the same poor artist. That's an AWESOME feature to me, and it seems most mods agree. Having thick-skinned people who pay no attention to the site staff at all is absolutely infuriating, and having a passive mechanism by which a dozen of us can say "no, this is a bad upload, sorry," and mark it poor quality works.

We're better off having uploader X post an upload at 5PM, then see its marked twice for poor quality at 7PM, and thus think about what they're going to do if they upload again from the same poor artist at 8PM, or lurk a bit, see some images from a nicer artist, and upload something better at 9. Or they can choose to upload the same crappy artists work and then ignore it when 3 days later they notice their deleted uploads number is up by 1, or 25. When I saw my Hanairo uploads get deleted, I didn't say fuck it and give up, and I didn't load just more crap, I learned how to look up better images! When my 27 score Bellows picture got marked as a tumblr sample and deleted and replaced, instead of giving up, I raised my bar! I was one of those "bad uploaders" who this feature could have helped, and I'm glad its there!

...which is of course impossible, and that's exactly why we're giving whole 3 days time to every mod to review it. And that's ok - as long as they are reviewing it independently of course...

The majority of the staff have indicated that they don't really let the opinions of others sway them. But, I'll declare right now: I DO let the opinions of the other mods influence me, ON EVERY image I approve. If an image is marked as a poor quality, I'LL THINK VERY HARD ab out whether I want to risk approving it, only to see it get flagged for deletion. Just as I learned to hold images I find to high scrutiny, I've learned to do the same for others.

I DON'T CARE about having an "independent opinion", with no consideration of the way others see quality, I see no reason not to support the 40+ other eyes on this site that may have more wisdom in approvals than I do, and moderate myself like a decent janitor. Isn't that what we're all expected to do? Having an active mechanism to do that only makes that easier, and it is making it easier NOW for ME as a janitor. If I care what the other mods think, that doesn't make me a sheep, it means I'm not an arrogant jerk to the other mods. The process is subjective! It always was! And being able to have a subjective marker means I can weigh that several people before me thought about it and said nah, and then think about it more myself.

If 12 think that an upload is not interesting, say some obscure copyright, that's fine, but if I can see with my own eyes that it fits the criteria for this site, I'll approve it. If it's marked as a violation or a poor quality upload by other people who gave it their own consideration, I'll look for what made them feel that way in an image, and reject accordingly. If It's only marked poor quality once and otherwise doesn't seem bad, well-faved and obviously made with good line, color and form, I'll approve, period. That's as independent as the process actually needed to be, and that's how most here who have approval power do it. Some don't, but that doesn't mean the rest are sitting on the matter with no thought either. A good mod cares what others think, and a hard number helps.

If you think I'm a sheep for giving a shit what other people think about an upload, well fine, I'm happy to be a sheep whose standards fit the quality of the site! Everyone here is a volunteer who does this cause we want great images! Wypatroszony, Shinjidude and Log don't get paid! Almost no one here does! The only person who is paid to do this is albert and that's cause this site is his baby. Everyone else invests time in this cause they care too, and want a better imageboard, thats why its hard to demote people, because we have to be realistic and fair when people make mistakes. Beyond that, the new change DOES help improve things for future uploaders and any new approvers.

When you go on acting like other uploaders, approvers and mods are in the wrong with this, and then complain that the new real-time mechanism for allowing approvers to watch each other is a bad idea, you're contradicting yourself. Do you want us to hold each other to a higher standard NOW? Or do you want us to be free to approve as we please? Or wait 3 whole days while an uploader posts stuff we honestly don't care about to say "Sorry, you're bad at this, post something else."

As an uploader, I WANT to know NOW if something I posted is bad and the mods think its no good!
As an approver, I WANT to know NOW what the mods think!

I've been on both sides of that fence, and it's a great feature to have a hard number on how many have seen it, hate it, think its poor or a violation. It empowers new users and old users alike to know what each other is thinking when considering what is and isn't good. We've trying to change this site over and over to fix this issue. We've discussed all this, over and over, and yes, it has taken too long to see visible progress. But you know what? Your attitude is terribly misguided, and myopic to the needs of this site. I will use this feature whether or not you like it, the way I see fit. It's helping me and others, who do a role that could have used it a long time ago, and it's helping uploaders who could have used it long ago as well. I'm happy with this site improvement, and it seems everyone else who does this daily is. If you're going to insist on improving quality control, we're going to use the mechanisms that improve that quality control.

richie said:
And I can guarantee you that once a post been in queue for a day you wouldn't know it's only 6 people or 12.

Sure he would, sure (almost) anyone using the queue would. By experience.

richie said:
Hey, Wypatroszony - what did you say about treating mods like sheep?

I asked you to stop treating every single one like that? I thought it was clear enough where I first posted it. It's troubling you need it called out because one (out of fifty) suddenly triggers your 'sense'. I'll leave it to you to decide what that makes you look like.

richie said:

What a moment.
Did you just admit, that when you're on the fence you absolutely conciously take into consideration the number of "HIDDEN" done by other mods?!

Hey, Wypatroszony - what did you say about treating mods like sheep?

How about you re-read the bit where I say that if I know I like a post and know it's high quality, I approve it without even consulting the counts?

For a borderline case, I think it's foolish *not* to see what others thought. Also as buehbueh says, for a borderline case, being able to see other's opinions is *helpful*. It can be the difference between "I think this is sketchy, and everyone else says it's rule-breaking or poor quality, so I definitely shouldn't approve this" and "I sort of like this, and no one else has any strong opinions on it, so I should approve it to make sure it gets through." Regardless of what you argue, I don't agree that bias in that sort of case is inherently bad. We are *supposed* to agree in broad strokes to the level of quality we want to see on the site. Reinforcing each other to reach that level of quality is a good thing. Mindlessly approving posts everyone else disagrees should be here is a bad thing, and should lead to flags.

Please note that richie hasn't proposed to hide the poor quality and rule breaking notes from other moderators, only the "didn't like enough" notes.

Also, I agree with richie about the contributor user level. Approving others' posts should come before bypassing the approval process for one's own posts. Most current contributors should be demoted to builder and given approve permission instead.

hemoglobin said:

Please note that richie hasn't proposed to hide the poor quality and rule breaking notes from other moderators, only the "didn't like enough" notes.

Also, I agree with richie about the contributor user level. Approving others' posts should come before bypassing the approval process for one's own posts. Most current contributors should be demoted to builder and given approve permission instead.

Yeah, no. This is just going to lead to a SHITLOAD of drama and a mass exodus of users.

Log said:

Yeah, no. This is just going to lead to a SHITLOAD of drama and a mass exodus of users.

Definitely. So it'll need to be done in 2009. I'll let you know when I get done with my time machine.

hemoglobin said:

Please note that richie hasn't proposed to hide the poor quality and rule breaking notes from other moderators, only the "didn't like enough" notes.

Also, I agree with richie about the contributor user level. Approving others' posts should come before bypassing the approval process for one's own posts. Most current contributors should be demoted to builder and given approve permission instead.

His problem is in moralizing at those who don't agree with him. Being able to know if one or four people mark an image for poor quality is useful to approvers, as is the no interest counter; and we should influence each other, it should not be an independent process. I wouldn't have flipped out if not for the condescension, not just at myself, but at the mods too.

And its not quite related to the topic of this thread, but removing most contributors right to upload without moderation isn't in my mind the proper solution. I have some alternative strategies, some inspired by others, some all my own, fit into the topic of improving the mod queue. Now, they're only suggestions, and I can't attest that anyone will like them if used, but here:

1) Before doing anything to the contributor rank: We need a contributor report like the one for janitors and regular members, so we can look at the ratios of how many deletions, +3s/+6s, upload averages, etc will give us better tools to evaluate this. There's no excuse in having a debate on contributor quality without better records on them. My personal extra suggestion is to also add some contributor specific things that scrutinizes how many tags are on their images when uploaded, how many times they've uploaded duplicates, bad anatomy, broken images or banned content, and all should be available as a permanent counter with a separate part mentioning the number of flagged images they've posted in the past 30-90 days.

2) Have a moderation lottery, where contributors see one out of of ten-twenty of their uploads automatically go to moderation queue anyway. This could be in place when quality is perceived to drop, and could be something albert or another specially selected admin turns on and off as necessary, if and when they feel quality is dropping. This could guarantee that contributors might be forced to consider the possibility of needing to think about mod approval, even on images that don't get put into that lottery.

3) Any contributor that has an image flagged, will temporarily have their uploads subject to the moderation queue during the flagging period. Once the flagging period is over they can upload as normal, but the uploads they uploaded during the flagging period will be in queue till approved or deleted. This will halt responsive uploaders who might wait out the period, or convince them to be pickier, if even only for a short time.

4) All contributors lose their status automatically if their ratio of deletions hits 3-5 percent, and must upload good pictures subject to the queue and to moderator approval till their ratio is restored whereupon only then will they be able to upload freely. This will essentially guarantee that all uploaders will seek aggressively to upload as best a selection of pictures as they can if they're borderline, and will halt borderline uploaders who fear losing their status from acting like they're above moderation.

5) Severe and repeated violations mean that status isn't restored unless a moderator steps in and resets it on their behalf. Say three violations. Make this something a commonality, and something that a contributor should absolutely avoid, and if they've proven egregious, then the mods can decide not to bestow a restoration on their status. This is a last resort weapon we can keep for the worst ones.

Every time any of these kick in, you can label it contributor(off) status, a status that will always put them in subject to the queue. Using some of these strategies to keep the contributors on their feet won't stop them from uploading great stuff if they're already doing so, but it keeps borderline ones in their place if someone decides they're going to ignore the Danbooru standard. This status could still have the power of a builder so they're still able to keep working with the power they had before.

All of this is something that can be hard coded, so the site automatically does it. Now I know not everyone will agree with some of these, they're only my opinion and I cannot guarantee any of them will work, but its more constructive in improving how we moderate images to begin with, and its likely our solution is in any one of a number of things neither I nor any of us have mentioned outright yet. As for finding out who is and isn't a good contributor, let's keep that discussion to the thread its meant for.

To the mods and other janitors, what do you think? Is there anything here we can use? It might mean lengthening the queue with extra stuff to approve, so I'm mixed on that. Maybe we can also modify one of these.

A probation period for contributors would be useful. A flagged post+bad quality disapproval would trigger it. During a probation period all their uploads would be subject to the mod queue. Each successive probation would have stiffer penalties. If this happens four times in a month demote them.

Hate to break into the current discussion, but could we get a split for the "poor quality" selection when moderating? There are times I'd like to use "poor quality" but the response is too ambiguous and can be misleading. Currently if I select "poor quality" it doesn't distinguish if that is poor image quality or poor artistic quality. If someone isn't getting approved due to poor image quality that is saying that it might have had a chance if the image wasn't so heavily artifacted and that images that are artistically like it are acceptable. On the other hand if it wasn't getting approved due to poor artistic quality, well then we're kind of saying don't upload stuff that is artistically like this again.

I feel like poor quality needs a breakdown entirely but I don't know how you would do it from a backend/ui perspective. Something like click the link and another series of choices folds out with jpeg artifacts, like a handful of common artistic flaws (bad face/hands/proportions,) and just generally bad or something. Not a dropdown menu, though, the last thing we need is an ever-expanding list where it's impossible to find the actual reason you're looking for.

Why not just integrate personal approver comments into the system?