Rethinking flagging

Flags and appeals are a perennial source of drama on Danbooru. I think a large part of it is disputes that are subjective in nature with regards to art quality. Notably, the wiki says the following is a valid reason to flag a post:

• Poorly drawn (bad anatomy, bad perspective, bad proportions) or overall low quality.

Poorly drawn is not a binary decision. There's a gradient. My opinion is that if you think an upload is ugly, you should either ignore it or down vote it. There are lots of reasons to not rely on flags as a signal:

• The majority of flags are from a few people.
• Therefore these few people hold a disproportionate amount of power.
• This leads to a power conflict between groups taking turns flagging and appealing.
• Arbitrating these ultimately comes down to a subjective decision anyway.
• In practice, this process doesn't change anyone's minds about what they think is good or bad.

Some arguments for keeping flags:

• A moderating force on limited uploaders
• A feedback mechanism for uploaders

But I would argue these are not good enough reasons. The paltry number of deletions resulting from flags don't appreciably affect limit rates. And the feedback comes off as being adversarial and accusatory rather than helpful.

I suggest we remove poor quality as a valid reason for flagging. People who persist in flagging for this reason should be warned and then banned.

albert said:

I suggest we remove poor quality as a valid reason for flagging. People who persist in flagging for this reason should be warned and then banned.

Are you suggesting we remove the line "Poorly drawn (bad anatomy, bad perspective, bad proportions) or overall low quality." from the wiki entirely or just shorten it to "Poorly drawn (bad anatomy, bad perspective, bad proportions)."?

albert said:

I suggest we remove poor quality as a valid reason for flagging. People who persist in flagging for this reason should be warned and then banned.

I don't want to sound disrespectful or insulting, but I present you an argument from the other side: what about the randomly picked "lottery" approvers you yourself promoted that people have been complaining about not so subtly for a while now? Is post #2870590 or post #3026177 the kind of content we want to keep on site?

I find it hard to agree with the sentiment that flags are out of control when the people who are on the receiving end of the flagging are too the same usual suspects, and it always ends up with three or four flags to the same post because some particularly nonsensical content keeps getting reapproved by people who got approval privilege via lottery or were promoted by the same people who uploaded the flagged post. I'd give examples but I don't want to namecall - there's plenty to be found via the search.

My opinion is that if you think an upload is ugly, you should either ignore it or down vote it.

Why have the approval queue at all then?

From the first page of "poor quality" flags only six posts were uploaded in 2018. Looking at the last 100 posts with that reason as flag, most are for old low quality scans, scribbles, heavy JPG artifacts or stuff approved by Not One of Us. I really don't see how this constitutes a problem, given that there's at least ~13 approvers active at all times who care enough to go through pending posts and mark them as not interested. It's rare enough that posts are flagged more than once, so again the "power" any flagger might have is negated by a single approver.

And finally, even if "poor quality" was made an invalid reason, people would still flag under "bad anatomy", which is really the reason for flagging most of the posts that don't fall under the categories I mentioned earlier. And besides that, how would one even flag a post like post #427050 besides as "poor quality"?

Well, what nonamethanks said.

albert said:

My opinion is that if you think an upload is ugly, you should either ignore it or down vote it.

Asking users to simply look past posts they don’t like and getting rid of flagging poor quality posts pretty much means removing moderation. There are enough other ’boorus with little or no moderation. Why should Danbooru become another one of them?

The strict moderation and resulting selection of mostly high quality posts is what sets Danbooru apart from other ’boorus and, if I correctly remember the poll from a while back, it’s one of the main reasons why the majority of users come to Danbooru. Giving up that unique feature seems like a bad idea.

Personally speaking, it’s also a main reason why I use Danbooru: I simply don’t want to sift through all the garbage on Pixiv or other ’boorus to find the few gems.

Minor thoughts that refuse to go into an [expand] block for some reason:

• The majority of flags are from a few people.

From what I see, most users don’t flag anything because they want to avoid the drama you mentioned and want to avoid getting on anyone’s bad side.

• Therefore these few people hold a disproportionate amount of power.

I disagree with that. Any approver can override a flag (except the previous approver or uploader, IIRC). For a post to be (re)approved, a single approver is enough. For a post to go unapproved/deleted, every approver has to dislike it. If anything, the current system is biased towards keeping posts, not deleting them.

My opinion is pretty much what nonamethanks and kittey already said: Removing flagging for poor quality post would be a huge mistake and would only serve to steer Danbooru down the path of destruction.

I find myself agreeing with kittey/NNT/unb, I think removing flagging for poor quality would be a bad idea

(Sorry, unbreakable, I didn't see you'd commented :P)

I had a couple paragraphs going in my head about this, but it turns out noname and kittey laid it out pretty much the same/better than I could have.

I'm not very active on the site these days, apart from the 5-ish tags I obsessively garden, but I still browse enough to be sure that removing the flag option would almost certainly lead to a rapid decline in quality of posts. Most annoyingly, it would remove the ability of the community to further prune ancient posts that never should have been permitted in the first place and/or do not meet modern quality standards. While it may be the opinion of some that such posts are under some kind of "legacy" clause, I'm in the camp of "bad is bad no matter when it got here".

Here are some changes I *could* get behind:

1) remove approval privileges from individuals not endorsed by the majority of the current moderation team. Listen to the rest of the moderation team and established users giving feedback about problematic individuals.

2) If there are any "unspoken rules" about flags, make them spoken rules.

3) Make it clearer that flagging a post sends it back to the moderation queue among any other queued posts. (iirc, we made it so that people viewing the queue couldn't tell the difference between a flagged post and any other queued post, didn't we? or am I misremembering?) If it were up to me, I would completely remove any outward signs differentiating a flagged post from a regular queued one (other than the upload date), including hiding the reason from anyone without approval privileges. Being able to see the reason only feeds fires, and the appeal thread exists for a reason if someone disagrees with the ruling in any case.

disclaimer: this post was written with a stream playing in another tab, on an unreasonable amount of sleep and following an 8-hour work day, so apologies for any typos/nonsensical gibberish

Flags and appeals are a perennial source of drama on Danbooru. I think a large part of it is disputes that are subjective in nature with regards to art quality. Notably, the wiki says the following is a valid reason to flag a post:

Poorly drawn (bad anatomy, bad perspective, bad proportions) or overall low quality.

To me the question here is what do want to preserve in the site, if you want to preserve high quality works then give an own meaning to that, using real references or advices from people who draws, not random fans of certain copyrights. Danbooru has artists visiting the site, most of them with neutral relations with us, but still we can talk to them trying to get feedback about "quality" and start to integrate their concepts here.

Therefore these few people hold a disproportionate amount of power.

This is truly a point here, and whats more, is the one who was elected knowledgeable about drawing and digital drawing? or at least has the competence to judge the work of an artist?, and it is not a simple user who only spam the top 200 from pixiv or the most voted artists here in danbooru?.

The flagging system is ok, but what I think we should do is start to determine how to flag each style and type of work as a solution.

Lannihan said:
This is truly a point here, and whats more, is the one who was elected knowledgeable about drawing and digital drawing? or at least has the competence to judge the work of an artist?, and it is not a simple user who only spam the top 200 from pixiv or the most voted artists here in danbooru?.

Most approvers are chosen because of the quality of their uploads, because there's no other way to determine if they're fit. You'd assume if someone only uploads good posts, they'd also only approve good posts.

This is why most builders consider lottery approvers to be a terrible idea (not that they'd say that publicly in the forums or even mention names explicitly, for fear of getting banned - though I've seen plenty of people complain about it on IRC and discord).

The fact that danbooru is considered to be the highest-quality anime imageboard on the internet is well enough proof that the current system works. Take the quality control away and what's the difference between this site and gelbooru or e621, besides the fact that on danbooru you must pay to see loli and use more tags?
Sure, you could say the quality of tagging, but I'd argue that's precisely because danbooru has a good average quality of posts. Nobody wants to spend hours tagging self uploads or broken spines.

I apologize if I'm being blunt, it's 5 AM here so I might be coming across as more confrontational than I'd like to be, but I really don't think lowering the bar is a solution, or that it's even addressing an actual problem. In the past year ~5% of the posts (21872/401841) ended up being deleted, and only the 5% of that 5% (1183/21872) was deleted as the result of a flag. In short, only 0.2% of the posts on site were flagged successfully in the past year. Furthermore, only an overall infinitesimal fraction was due to poor quality - most were very old posts and old scans, or image samples (about 10% of those 1183 are tagged as image sample so there might be more). That doesn't seem like a problem at all, especially since, again, we have a good number of active approvers and a single approver can deny a flag.

Harsh judgment about something that a user will take seriously will lead to strife and that's an inevitability. Besides, it's usually a vocal minority that complains and raises drama about these flags anyway.

-1 to the idea of reworking flags. People need to grow thicker skin. I've had my share of stings that come from post flags and/or deletions, but I vowed never to complain or involve myself in their appeal. IMHO if they were really that great, then someone else will appeal them.

BrokenEagle98 said:

-1 to the idea of reworking flags. People need to grow thicker skin. I've had my share of stings that come from post flags and/or deletions, but I vowed never to complain or involve myself in their appeal. IMHO if they were really that great, then someone else will appeal them.

I've had my stings as well, though it was largely image_sample deletions due to danbooru file-size limitations at the time of upload, and before image replacement was a thing (Those deletions are like 70-80% of my deletions Dx). For quality-related flags I don't think I've ever appealed one of my own posts, though I do post why I think it's not that bad in the comments.

That said, I'd say there is a bias that forms from a flag, wherein the problematic issue is overblown because it's being focused on so minutely, which can lead to images with minor anatomical issues being deleted because of it.

CodeKyuubi said:

I'd say there is a bias that forms from a flag, wherein the problematic issue is overblown because it's being focused on so minutely, which can lead to images with minor anatomical issues being deleted because of it.

This is partially why I think that flag reasons shouldn't be visible to the general public, but also a good argument for having flag reasons visible to approvers only after the image has been viewed, which can be achieved as simply as placing the flag text below the image rather than above it. I haven't approved anything in something like 6 years (back whenever we did that give-all-builders-approval-powers trial run that ended poorly), so I don't recall where that option is, but it should also be below the image if it isn't already. I don't think this kind of bias is all too big of an issue for the statistical reasons already mentioned in posts above, but it's true that there IS a kind of bias that is fostered by viewing something already knowing there's supposed to be something wrong with it, and I think that efforts to eliminate bias outside of the general bias used for quality control are a good thing.

On a tangentially-related note:
If we're concerned about biases brought on by flag reasons, it might be worth it to consider an approach where, rather than being allowed to enter free-form elaborative text, the flagger could ONLY choose from the list of acceptable flag reasons. Because if an issue in an image is so minor that one only notices if it's specifically pointed out to them, it might not be disruptive enough to warrant deletion. This, of course, works on the assumption that users won't be attacked for flagging something as low-quality without providing a multiple-sentence justification for their feelings, which requires a level of trust in the approvers that I would at least hope we can have here.

That isn't to say that I don't think malicious flagging exists, or that mods shouldn't question it when a user flags a suspicious number of posts that share a trait in common - just that users should not feel they are in danger of banning just for thinking that something is low-quality. As has been stated, quality is subjective, and if the approval team disagrees with the flagger, the image will be quickly re-approved; there's no need for ill-feelings against the flagger unless they continue to repeatedly flag that post out of spite. In my opinion, all members should feel safe contributing to the continued application of quality standards. That's a mark of a healthy community.

I am the user that flaggs the most images on the site (I find a lot of tat while gardening tags on old posts). If any one has a problem with my flagging please contact me, I would gladly change my practice. However without feedback I can only assume that Im doing it right.

I may be a cynic but I do believe that if you take away the ability to flag posts many people will instead atack "bad" approvers with feedbacks, comments and forum posts, causing far more drama.

albert said:

Flags and appeals are a perennial source of drama on Danbooru. I think a large part of it is disputes that are subjective in nature with regards to art quality. Notably, the wiki says the following is a valid reason to flag a post:

• Poorly drawn (bad anatomy, bad perspective, bad proportions) or overall low quality.

Poorly drawn is not a binary decision. There's a gradient. My opinion is that if you think an upload is ugly, you should either ignore it or down vote it. There are lots of reasons to not rely on flags as a signal:

• The majority of flags are from a few people.
• Therefore these few people hold a disproportionate amount of power.
• This leads to a power conflict between groups taking turns flagging and appealing.
• Arbitrating these ultimately comes down to a subjective decision anyway.
• In practice, this process doesn't change anyone's minds about what they think is good or bad.

Some arguments for keeping flags:

• A moderating force on limited uploaders
• A feedback mechanism for uploaders

But I would argue these are not good enough reasons. The paltry number of deletions resulting from flags don't appreciably affect limit rates. And the feedback comes off as being adversarial and accusatory rather than helpful.

You're missing "A moderating force on unlimited uploaders" from the second list, which is initially what I thought you meant, but it seems you actually do mean this based on the following paragraph.

This moderating force on unlimited uploaders isn't some minor point either - it singlehandedly overwhelms every one of the in terms of potency and significance. Take away the ability to flag for poor quality, and suddenly there are no quality controls whatsoever on those who have been granted unlimited uploads other than getting randomly viewed by someone with approval privileges who find it so utterly awful that they need to delete it - and it's likely people would start pointing out these posts to them to delete, in which case you go from "everyone with approval privileges gets a chance to save it before it is deleted" to "if anyone with approval priveleges agree with the person who wants it deleted then it will be deleted".

A similar but related point would be "A moderating force on moderators". Can I just point out that many of the lowest scored images that have been deleted were approved at some point in time?

I also agree with everything Kittey said earlier, and the second part of nonamethanks's post (some of these things are repeated in this post).

If someone can't cope with the idea of something they uploaded being flagged and possibly deleted then that is their problem, and not the problem of the flagger or flagging. If someone flags a decent picture out of pure spite then the picture will just get reapproved, so there is no problem to the site from this. In the 5 years or so since I first discovered this site there have only been a couple of incidents of people going on sprees of flagging of large numbers of images that don't warrant deletion, and in all cases the images with bad or dodgy flags were reapproved and the user in question (temporarily) banned. So this isn't actually causing problems for the contents of the site. Meanwhile flags for bad art have removed hundreds or even thousands of images that don't live up to quality standards that are enforced on every user who hasn't been granted unlimited uploads (yet), thus serving an essential function in preserving the standard of art on the site which is Danbooru's USP.

Echoing most thoughts thrown about here, most notably NNT/kittey/Unbreakable and anyone else against the reworking of flags.
I think the flagging system is doing its job just fine, and few pictures have been the subject of a flag/approval battle on the grounds of quality.
Generally if a picture gets flagged for poor quality, either the entire approval team (or at least the 10+ that see it while reviewing) agrees and it gets deleted, or a single approver disagrees, reapproves it, and it doesn't get flagged again.

nonamethanks said:

I don't want to sound disrespectful or insulting, but I present you an argument from the other side: what about the randomly picked "lottery" approvers you yourself promoted that people have been complaining about not so subtly for a while now? Is post #2870590 or post #3026177 the kind of content we want to keep on site?

Not at all. If you think a particular approver is bad, point me to them and I'm more than happy to demote them. I've done it before. I myself haven't promoted anyone recently precisely because of reactions like the one you're giving here. And you're not wrong.

But pointing to one or two bad uploads isn't sufficient. Everyone approves bad stuff once in awhile, us included. You need to prove that a large fraction of their approvals are bad. I'd say at least 10%. Stuff like a high percentage of negative scores is good evidence, or even a low median score.

I find it hard to agree with the sentiment that flags are out of control when the people who are on the receiving end of the flagging are too the same usual suspects, and it always ends up with three or four flags to the same post because some particularly nonsensical content keeps getting reapproved by people who got approval privilege via lottery or were promoted by the same people who uploaded the flagged post. I'd give examples but I don't want to namecall - there's plenty to be found via the search.

You say this, but look at the flagger distribution. Just four people flagging more than 10 posts. There is no signal there. These four people could easily be victims of confirmation bias or other behavioral fallacies.

Why have the approval queue at all then?

I ask myself this all the time. If you don't check Danbooru's equivalent of /new often it doesn't come into play. But I understand not everyone has my browsing habits and so it's useful to others.

From the first page of "poor quality" flags only six posts were uploaded in 2018. Looking at the last 100 posts with that reason as flag, most are for old low quality scans, scribbles, heavy JPG artifacts or stuff approved by Not One of Us. I really don't see how this constitutes a problem, given that there's at least ~13 approvers active at all times who care enough to go through pending posts and mark them as not interested. It's rare enough that posts are flagged more than once, so again the "power" any flagger might have is negated by a single approver.

I honestly don't see the point in flagging these uploads from 2012 when the average user is never going to go far back enough to come across it. So the fact we wouldn't be flagging these isn't a huge loss. And by your own admission, if poor quality flags aren't common, then the real world impact of not permitting them would be minimal.

And finally, even if "poor quality" was made an invalid reason, people would still flag under "bad anatomy", which is really the reason for flagging most of the posts that don't fall under the categories I mentioned earlier. And besides that, how would one even flag a post like post #427050 besides as "poor quality"?

You're right. Bad anatomy shouldn't be valid flag reason either.

kittey said:

Well, what nonamethanks said.

Asking users to simply look past posts they don’t like and getting rid of flagging poor quality posts pretty much means removing moderation. There are enough other ’boorus with little or no moderation. Why should Danbooru become another one of them?

The strict moderation and resulting selection of mostly high quality posts is what sets Danbooru apart from other ’boorus and, if I correctly remember the poll from a while back, it’s one of the main reasons why the majority of users come to Danbooru. Giving up that unique feature seems like a bad idea.

Honestly, even browsing through new with all the pending and flagged posts, the average quality is pretty good. I think a large part of that isn't flagging but upload limits and attracting a particular kind of user. The kind of user who cares more about quantity will probably goto Gelbooru or Sankaku because of the better search features available to basic users.

Regarding unlimited uploaders:

Flagging is just a bandaid solution. I don't think a dozen deletions a week is a real deterrent to heavy uploaders. If you perceive one of the unlimited uploaders to be bad, then you should message me and we can discuss whether it makes sense to revoke the privilege. It's usually the case that while their average quality is low, and you can always point to a few egregiously bad examples, their uploads are a net positive for the site. But that's an assumption I'm making. It should be handled on a case by case basis.

Like other users, I am also of the opinion that removing the option to flag images for low quality would be a really bad idea especially if there's any desire to keep this site focused on high quality artwork.

Also, if there's drama nearly anytime something gets flagged, then why not hide the flag reason completely? I believe names were hidden to moderators because flaggers kept getting harassed about it so why keep the flag reason visible if it only seems to cause trouble too?