Donmai

Improving moderation process

Posted under Bugs & Features

albert said:

A probation period for contributors would be useful. A flagged post+bad quality disapproval would trigger it. During a probation period all their uploads would be subject to the mod queue. Each successive probation would have stiffer penalties. If this happens four times in a month demote them.

A month of probation for a single flag/delete? Of any of the user's uploads, no matter the post's age? That's rather harsh, unless you had other additional conditions in mind...

We could try something like 3-7 days for immediate infractions, with full restoration to contributor level automatically upon exit from the probation period.

Maybe the infractions could also accrue with a residual period before they disappear, so if you have 3 demerits or more within a 3 month period, then you lose contributor status till at least one of the points disappears off record.

So say, contributor Y makes an upload in the beginning of April that gets flagged, then two more are flagged in May and June, then they must wait till they're into July that the first demerit is off their record and they're no longer subject to the queue.

One could also grant an immunity for having flagged uploads more than 3 months old, so if a few posts in past history are flagged, it doesn't affect the status of the contributor in the immediate, and doesn't subject them to the queue then. It's whether the uploads a person makes in the short term are lousy and flag worthy that it happens.

And on the other subject, we could try having a "Weak Artistic Merit" button that's separate from "Poor quality", no lists or drop-down menus.

I think he's saying a post from a contributer would need flagged, it'd have to be hidden for "poor quality" while in the mod queue (by some absolute threshold / relative proportion of hide reasons?) before the probation gets triggered.

I do agre with you though,that there should be a statute of limitations beyond which a flag wouldn't trigger it. This is especially true for people who posted things before the mod queue was instituted and before upload rules went into effect. How many times do ancient posts of even Albert himself's end up in the queue, flagged because they no longer match the criteria we use today, even though they weren't against the rules at the time.

hemoglobin said:

Please note that richie hasn't proposed to hide the poor quality and rule breaking notes from other moderators, only the "didn't like enough" notes.

Thank you good sir. Allow me to dedicate your post to all of these who apparently never read what I write, but are first to patronize me and assign intentions I've never expressed.

In this whole discussion I've made one fundamental mistake. I wanted to explain how showing number of hidden posts could subconsiously influence moderation process.

As it turned out, I could try explaining to the end of the world and a day longer, because apparently (some) moderators are thinking that there is absolutely nothing wrong to influence and/or to be influenced by other mods' purely personal tastes. And not only subconciously but also by the design.

I'll leave this today with no further comment, though I reserve my right to express sardonic laugh every next time I hear albert (or someone else) mentioning about need to expand base of approvers and their tastes.

Updated

I'm catching up with the thread, but I want to touch on the current points.

First off, though: Let's get off the blanket statements in these quality control threads. This includes trying to indirectly call people "sheep" or trying to paint an opposite side with awkward, flimsy analogies to unrelated events outside of the site. There's better ways to discuss here than attempting to downplay someone's (hopefully well-stated) points in either way. This isn't about morality, but presentation. Please start communicating your points better, unless you want to paint a target on your own back.

Disapproval Counts
I see no problem with making this info available to other approvers. Now that there's multiple forms of disapproval, it's more useful than before since that can help with making a decision, and I expect recently added approvers are making use of that info. IF an approver chooses to base decisions on that. If not, they're allowed to. In either case, both should still understand that giving posts the okay attaches their name to it, and this doesn't make any disagreed-upon (maybe not necessarily "mindless") approvals any less subject to flags.

Contributor Probation
Something like this needs to be handled on a case-by-case basis. If we're going to include the option of putting Contributors on some kind of probation, it needs to be for a larger amount of flagged uploads, not just one or two.

Use of "Poor Quality"
Both NWF Renim and Log are right about how "Poor Quality" should apply. There's image problems like corruption and overt jpeg_artifacts, and then there's bad art quality. Some expansion for that particular disapproval would be okay, and I wouldn't mind if it was split into two disapprovals to cover both aspects. But wording it right is key.

re: poor quality

just tossing a suggestion, how about "poor digital aspects" for technical shortcomings. i would think the word digital would related better for artifacts, compressions, file sizes, and the likes. and maybe "poor artistic taste" or "poor artistic selection" for bad anatomy, grotesque subjects, and the likes. art is very subjective, having "artistic" in the phrase hopefully delivers what it really meant.

we can also use substandard, mediocre or questionable instead of poor.

Every contributor+ in case of any doubts about the quality of their post has an option of uploading the post through the queue.
In other words, if he's using his privilege of avoiding the queue then he's automaticaly declaring he's absolutely sure about the quality.
That's why if such post is not only flagged (which occurs fairly rarely), but also deleted then such contributor+ has just received a vote of no-confidence from whole moderator team.
How serious consequences would this mean is matter of lesser importance, what important is that they should be applied automatically.
Unless you want to introduce another rule which will become a dead letter, of course.
Some retroactive restrictions may apply, though.

Ghostrigger's wording is nice, "Poor Artistic Selection" is clear and concise. We could try working the poor art option as "Poor artistic choice" as well, keep it short and simple, and use, "Poor Digital Quality" for situations with more serious flaws along the lines of image degradation and digital noise issues.

On the contributor +/- thing, if having an upload be flagged plus marked poor quality occur, a note should be entered in Mod Actions or Feedback that the system itself has kicked in and forced the contributor on the queue, and labeled them in need of review. It doesn't have to be a negative, just a neutral feedback note the system makes. Maybe after an arbitrary set of times this has happened in a short time can give evidence to the moderators to enforce a probation, rather than an auto-probation that kicks off, so for that we need to note the exceptions:

Having the statute of limitations will prevent older images from unfairly affecting otherwise good contributors. This should also include an exception for artist cease n' desists, corruptions due to failed uploads when the system decides to screw things up, and immunity for when other users cause harassment (such as during some users mistagging images with banned artist tags).

So from that, lets say, Contributor X gets 4 failures due to poor quality in a set time of a month, then the matter is alerted to the staff, and a probation option is given, but only after a user review has occurred. Maybe putting user reviews into part of the moderation queue could be useful to this, no?

At the top we can have problem users appear in a list and their incidents show up, like they do in the dashboard, and the moderators can vote whether that contributor should be sent into probation, with a comment list of all admins/moderators/janitors marking for or against with a chance to explain their reasoning. Or we can spin that off to its own "Contributor Review" page and keep it out of the queue.

I agree with buehbueh's idea of alerting the staff for demotion instead of making it automatic, but I think that the probation period can easily be automatic without a real problem. What I think is a nice way to take age into account would be having the probation period start from the time the image was uploaded. For example, if it is a week since upload, if it gets flagged and delted immediately, it's 4 days, if it's flagged 2 days later it's only 2 days, and if it stays 4 or more days without a flag you cannot get a probation for it

Shinjidude said:

Honestly like I said above it's useful if and when you are on the fence.

I am not making any value judgement about whether or not the score should be displayed for moderators. However, I would like to point out that this is exactly what the scientific study I cited earlier was looking at, and they demonstrated empirically that having a score available causes the results to be more random and less influenced by the intrinsic quality of the item.

Whether reducing randomness in the results is worth hiding the score is a question for albert and the approvers, but you need to understand the tradeoff being made.

That post specifically was me, and what you're talking about more broadly is almost certainly me as well. I flag them poor quality because I think that's most appropriate for what I think is mediocre art. Not a perfect description of my feelings, but generally better than "no interest".

If every mod's going to use it the way they want there's no point in even having the other choices in the queue. Poor quality needs to be basically "this doesn't conform to the site's quality standards" not "this is below my own personal standards."

albert said:

People are going to have differing standards. One disapproval like that shouldn't be taken as gospel. Truth is achieved through consensus.

If that's the case then what for all this fuss with "poor quality" and "hidden" differentiation?
In my book "poor quality" rejections should be reserved for posts you'd insta-flag if only they've been already accepted (with poor quality reason).

As it is now, your precious thin-skinned uploaders seeing their uploads being massively marked as poor quality will stop uploading indeed. Anything, that is.

richie said:

If that's the case then what for all this fuss with "poor quality" and "hidden" differentiation?
In my book "poor quality" rejections should be reserved for posts you'd insta-flag if only they've been already accepted (with poor quality reason).

As it is now, your precious thin-skinned uploaders seeing their uploads being massively marked as poor quality will stop uploading indeed. Anything, that is.

Dude, cool it already. The only reason I brought it up was because it looked like a discrepancy that needed bringing up. And for reference, it's not a massive thing, it's something that only happens on a few posts that I've seen here and there.

MFZbdude said:

Dude, cool it already. The only reason I brought it up was because it looked like a discrepancy that needed bringing up. And for reference, it's not a massive thing, it's something that only happens on a few posts that I've seen here and there.

Just entered status:deleted search and many deleted posts had at least one "poor quality" reason.
And now you can tell me if they were really bad, or if only one of mods decided to emphasis his decision of finding something mediocre.

I think the long and short of it is I'm not gonna approve these posts because of their quality, so clicking anything else just feels like blowing smoke up folks asses. However if this is really sticking in people's craws that much I'm fine with flipping over to No Interest on some stuff.

richie said:

many deleted posts had at least one "poor quality" reason.

That's a good thing in itself. Having a lot of posts deleted just for lack of interest would be rather sad

ShadowbladeEdge said:
I'm fine with flipping over to No Interest on some stuff.

Please do. I think keeping literal and precise on "poor quality" is a lot more important than on "no interest" so if you gotta go for an inaccurate reason, "no interest" is better

1 3 4 5 6 7 8 9 10 11 12