Posted under General
Has it struck anyone that the current post approval system seems really inefficient somehow? For every low quality post that gets unapproved there seems to be a dozen good quality ones that just seems to fall through the cracks. Wouldn't it be more efficient to set the default mode to approved then send the army of regulators to remove the poor quality pictures? Or at the very least increase the approval period. Just a suggestion.
Updated by cleartailcat
After 3 days, every active mod and janitor has made their decision one way or another. Previous discussions like this have made that clear.
So waiting longer won't help in any way. More janitors/mods will help only if one of them makes a different decision, but we already have 33 janitors and 10 mods, so that seems rather unlikely. They're picked and trained for having more or less the same threshold on quality, after all.
Having to appeal some borderline posts is a pain, but a necessary evil of the moderation system. No system is perfect, and Danbooru prefers to err on the side of higher quality, rejecting a few decent quality posts that may be appealed, rather than accepting a few too many low-quality posts that have no place here as would be the case if accept-by-default were to be implemented.
No, it is necessary [to have an appeal thread], because we have an system that includes false positives (images that are good but are not approved). No matter how efficient our pruning method, under the current system we'll always have ones that will slip through the cracks. (for example, if we get 1000 images uploaded a day, and we are 99% accurate in deciding which images need to stay, we would still miss 10 good images a day). So we need an appeal thread under the current system.
The suggestion of having images approved by default is much more problematic though, because instead of losing good images, we would keep bad images. One major thing that sets Danbooru apart from almost the entire rest of internet is the quality control, where the casual user can pretty much be sure that everything on the site is up to some kind of quality, even if that is a somewhat subjective thing, obviously. But this would be a real issue and potentially ruin the cachet that this site has.
The await approval policy only really makes sense if the number of poor quality images being submitted is more than the number of good images being submitted. The thing is that as of present, the number of good posts being uploaded is several times more than the number of bad post being uploaded. Trailing through the status deleted archive, on average there's maybe one genuinely bad post every two pages that isn't a duplicate or an uploading error. This isn't erring on the side of caution. This is running so deep into the caution side you end up on another continent.
It seems to me that it did take alot less effort to send the mods after the few bad posts that get submitted rather than make them review and approve EVERY post that is uploaded, ever. Especially since the deletion appeal thread should be alot more useful in alerting mods to bad posts than it's present role. All the system really does now is generate alot of unnecessary hassle.
Unless you really think 90% of all images uploaded to danbooru is made of shit.
Updated
piespy said:
After 3 days, every active mod and janitor has made their decision one way or another. Previous discussions like this have made that clear.
I must admit, I've always wondered how this can possibly be the case, given that posts to the deletion appeal thread are reapproved >50% of the time. And a quick scan of status:deleted does show some "Unapproved in three days" pictures which are rather high quality. Maybe it's time to add a few more janitors again?
That said:
Megamet said:
It seems to me that it did take alot less effort to send the mods after the few bad posts that get submitted rather than make them review and approve EVERY post that is uploaded, ever.
This is true, but the factor that you're not considering is that under that system, somebody would have to do the 'sending after'. There's a sort of apathy effect that occurs when a bad picture is already on Danbooru that makes them somewhat unwilling to take the steps necessary to get it removed. (This is true even under the current system. I think the general consensus is that we've got a disproportionately high number of bad Touhou pictures, but even with the Unapprove button floating right over there, few of them ever get flagged.) Also, an approve-by-default system would require, in place of the deletion appeal thread, a 'deletion request' thread, and I don't even want to think about what a hotbed of drama and hurt feelings such a thread would become.
Basically, our system is geared towards maximizing precision, possibly at the expense of recall, and that's how it was intended to be.
Updated
Megamet said:
The await approval policy only really makes sense if the number of poor quality images being submitted is more than the number of good images being submitted.
This is (in our current evaluation system, and taking into account that there is no accounting for taste and that this is inherently a subjective call) the case. So for us, the current system simply makes sense from an efficiency standpoint.
Personally, when I go through an approval queue, I would say I approve maybe 10% of the queue, at most. And I consider myself to be relatively lenient in approving things.
glasnost said:
I must admit, I've always wondered how this can possibly be the case, given that posts to the deletion appeal thread are reapproved >50% of the time. And a quick scan of status:deleted does show some "Unapproved in three days" pictures which are rather high quality. Maybe it's time to add a few more janitors again?
This is because of the false positive thing that I was talking about before. It's a little counter-intuitive, but if there are 100 good images uploaded a day, and we have a 99% accuracy of determining them, we still miss 1 image a day. If all of those get appealed, and there's a 50% chance we re-approve them, that increases our accuracy to 99.5%. Well worth it, but also almost unavoidable that we need this extra layer.
I am sort of a statistics nerd...
EDIT: though I certainly wouldn't be against adding some Janitors. The moderation list currently shows 10 people hiding images that are on the verge of being deleted. Even given that some Moderators/Janitors don't use the mod queue or hide feature (I would suspect this is a minority of people), I think more people could not hurt. Even in the "about to be deleted" category I usually pick out quite a few that are absolutely Danbooru material.
Updated
In a way, an approve-by-default system would actually be harder on uploaders than our current system. As it is, only one mod needs to like a post for it to be approved, even if the rest of the mod team doesn't like it. An approve-by-default system would require mods to actively delete posts they thought were bad, which means the reverse would be true: a post would be liable to be deleted if any one mod disliked it, even if other mods may have liked it. The current system is set up precisely to avoid having mods actively deleting posts. I can just imagine the chaos that would ensue if we had all 40 people on the mod team going around deleting every post they didn't like.
Yes, we do need to increase coverage of the mod queue again. It's cyclical. Coverage becomes lacking, we increase coverage, things go well for a while, then the approvers sort of fade away and we have to get a new batch.
I don't, however, support changing the fundamental approach. I'd much rather have some good posts get missed than have every terrible image posted get automatically approved and have to be weeded out by hand. It would just turn everyone into a contributor, and people have raised many concerns in the past about some contributors with at times less than stellar judgment. Imagine giving every user out there the same powers. No way.
Evazion explains succinctly while an "approve by default" system would be chaotic at best.
I admit I don't check the queue systematically anymore, but as an admin I do have a lot of other things to deal with when I come to danbooru, so hopefully my lack of queue coverage is understandable.
I do check almost everything that goes through the appeal thread, though, and reapprove quite a lot of it. That's actually my barometer for how the approvers are performing: the more appeals I accept, the more I feel like the moderation team is falling behind.
So yes, at this time I fully support getting more janitors.
Megamet said: The await approval policy only really makes sense if the number of poor quality images being submitted is more than the number of good images being submitted.
It doesn't have to be more, it only needs to be a significant percentage. Surely you'd agree that 45% of uploads being bad would be unacceptable; even 20%, if you ask me. I know you can't see the mod queue, but bad images really are a significant percentage.
I would be interested in knowing what percentage of Member-level uploads eventually get accepted but there's no way to easily calculate that.
Sure, the main post index lists "only" 50k deleted posts (assuming that's accurate) and the front page lists 630k, but consider that, at the moment:
- Top 40 uploaders account for more than 50% of all uploads, despite accounting for only ~0.01% of registered members and ~0.7% of members with an upload.
- Top 100 account for 433k posts. We're over 2/3rds now.
- Top 200 (still almost all contrib+) takes us to 490k, over 3/4ths... you get the point.
Member-level uploads subject to the queue do not constitute a huge percentage of content here (it's still a huge number, mind you, which is why it's a major moderation job, but not a huge percentage). And consider that for a long time there was no moderation in place, which had seriously detrimental effects on the site's quality, leading in part first to an invite-only site and then to a reopened site with moderation.
What I'm getting at is that sub-Contrib level uploads from the current moderation era, which is where most of those deletes would be from, do have a significant failure rate. It really is possible that it could be over 50%, but there are too many unknown variables to be sure.
So it just would not mesh with Danbooru's philosophy to accept them all by default and try to prune the resulting clusterfuck.
We currently have 12 +/- 2 heads going through the queue on a daily basis. I post too much in the forum so I can't really dig up my past numbers but iirc we had 14 heads going through the mod queue last time I looked and 21 before that (shortly after I was promoted to full janitor) so yes we could probably use a few more janitors.
I also have a feeling some of the hides are just mass-hides. Or that some approvers only approve a small subset of select things, and ignore the rest. Otherwise I just can't understand how so many of the acceptable images in the deletion appeal thread were really deemed unworthy by nearly a dozen people.
Focusing approval efforts only on certain things isn't necessarily bad, it just means we need more people to get better coverage of the other things.
Methinks a lot of hides has to do with the content and not quality, for example Ive seen touhou pictures of questionable quality get approved and alternative pictures of good quality get ignored.
I agree with the suggestion of getting more janitors. I remember the queue moving a lot more smoother when we increased the number of staff members. I'm pretty sure that out of all our regular contributors, tag gardeners, regulars and translators etc there are good janitor candidates that can keep the queue moving.
