Donmai

Post upload limits

Posted under General

I made a minor deployment to fix the flag/appeal history and deployed the new formula accidentally. It was reverted but I still intend on deploying the new formula in a few weeks.

The formula is this:

(Base * (1 - (chance of deletion / 15))) - (uploads in past 24 hours)

Some major changes:

  • Base is 10, 20, 30, 40, or 50 depending on account age
  • Uploads are capped to 50 a day
  • Everyone can upload at least 10 posts a day
  • All upload counts are scoped to 120 days, so if you are heavily capped now the effect will lessen with time (if your deletion chance goes down)
  • Instead of counting only pending posts, all uploads will be counted. This is to remove the bias for certain time zones where approvers are heavily active
  • Deletion chance is the confidence that your upload will be deleted. At high upload counts it is effectively the percentage of your uploads that ends up deleted. At lower counts it is weighted closer to 0.
  • The limit is heavily biased for deletion chance. If 5% or less of your uploads are deleted you probably won't be affected by the new formula much. If it's closer to 15%, you'll be more heavily penalized. The majority of uploaders effectively have 0% deletion chance.

My belief is that if 15% of your uploads are getting deleted, you should not be able to upload as much as you want. If you are a heavy uploader with a low deletion chance and you want a higher cap, then you should petition for a promotion to contributor.

albert said:
The formula is this:

(Base * (1 - (chance of deletion / 15))) - (uploads in past 24 hours)

I get the explanation but the formula's a bit incorrect from a mathematical point of view. At 15% deletion rate my (Base * (1 - (chance of deletion / 15))) comes out to be zero, meaning my upload limit is zero, though in truth I should have a limit of 10. Why not change it to something like:

limit = 10 + (age modifier * (1 - min(1,chance of deletion/15))) - (uploads in past 24 hours)

where age modifier would be 10, 20, 30 or 40 (depending on account age). Meaning even with a deletion rate of 15% (or higher), one would have a maximum upload limit of 10.

Of course, users won't be penalized for deletions in their first month, but that's how the current formula is, too.

_Aneroph_ said:

...

I imagine instead of the formula you put forward, they'll use the Max function that they do right now, in other words, they have only been showing you a portion of the formula for simplicity and ease of understanding. As I understand it, the formula will go from this:

max(10 + (approved/ 10) - (deleted/ 4), 4) - pending

To this:

max((Base * (1 - (chance of deletion / 15))), 10) - (uploads in past 24 hours)

If that wasn't the plan, then they would need to adjust the formula to either the above or the one you suggested, otherwise someone could end up with an upload limit of zero.

------------------------------------

To either Albert or Toks, is there a way we could see what the "chance of deletion" formula is? Will it be posted on this website or on the Github website? From what's been said, all I can understand is the following:

limit upload_count->large_number_of_posts, chance_of_deletion -> (deleted_posts/upload_posts)
limit upload_count->zero, chance_of_deletion -> zero

albert said:
My belief is that if 15% of your uploads are getting deleted, you should not be able to upload as much as you want. If you are a heavy uploader with a low deletion chance and you want a higher cap, then you should petition for a promotion to contributor.

So is there going to be a more formal path to become a contributor?

BrokenEagle98 said:

To either Albert or Toks, is there a way we could see what the "chance of deletion" formula is? Will it be posted on this website or on the Github website? From what's been said, all I can understand is the following:

limit upload_count->large_number_of_posts, chance_of_deletion -> (deleted_posts/upload_posts)
limit upload_count->zero, chance_of_deletion -> zero

Examples:
0 deletions, 0 total uploads: 0%
1 deletion, 10 total uploads: 1.7%
10 deletions, 100 total uploads: 5.5%
100 deletions, 1000 total uploads: 8.2%
1000 deletions, 10000 total uploads: 9.4%
10000 deletions, 100000 total uploads: 9.8%

See:
http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval
Danbooru's code for it

Just my personal opinion, but I think at least some of their upload slots should still be based on pending as opposed to being tied to a daily limit. That would potentially allow a user to contribute more in a day than their daily limit would allow, as the daily limit seems to me to punish users who can only be active a few days of the week.

This also hits again against comic type post uploads. Given the constant policies that almost excessively hit these types of posts harder than others, is the intention to simply try and discourage uploading comics to begin with? It seems to be a running theme in several of the past changes before someone points it out, so it'd be nice if this was a goal that it be openly stated and not hidden behind other justifications.

Updated

Albert pointed out why he doesn't want it to rely on pending. How about letting upload slots get carried for up to 2 days later? Or something like that. I myself usually gather stuff I want to upload then upload it all in one go, so it does seem worth looking for a solution

NWF_Renim said:

Just my personal opinion, but I think at least some of their upload slots should still be based on pending as opposed to being tied to a daily limit. That would potentially allow a user to contribute more in a day than their daily limit would allow, as the daily limit seems to me to punish users who can only be active a few days of the week.

This also hits again against comic type post uploads. Given the constant policies that almost excessively hit these types of posts harder than others, is the intention to simply try and discourage uploading comics to begin with? It seems to be a running theme in several of the past changes before someone points it out, so it'd be nice if this was a goal that it be openly stated and not hidden behind other justifications.

Could you be explaining this a bit more as this now caught my attention.

-----------------------

I think this proposed system does have a number flaws

I had a rather high failure rate if only because I posted a handful of images in those last 120 days and had two not approved (with one being one which was a misupload I requested deletion of) But that game me a huge penalty, but when I posted a pair of images which were approved and each gave me almost 10 posts to my limit just from those.

This seems to really be making the balance extremely jumpy as unless you're frequently posting it leads to a problem of each outcome moving the meter a huge amount and such a heavy handed feeling may not even fix the problem.

The Old (current?) system does a better job at providing a 'punishment' for posters with high deletion rates as it's a steady up ticking of the penalty compared to the rate where the additional slots are rewarded out.
This proposed system leads to the problem of a 'bad posters' not being incentivized to go for 'better posts' as there isn't a real reward for 'good posting' due to the system only remembering 120 days.
I'm not sure proposed system using square roots would be better as it is a bit more 'punishing' but I admit it doesn't 'reward' half as much as the Old (Current?) system.

albert said:

  • Base is 10, 20, 30, 40, or 50 depending on account age

Since it wasn't mentioned before, if this will be implemented like in this commit , this means with each month of account age one will get an additional +10 to one's base upload limit up to a maximum of 50. Therefore probably most users will have a base upload limit of 50.

BrokenEagle98 said:
I imagine instead of the formula you put forward, they'll use the Max function that they do right now, in other words, they have only been showing you a portion of the formula for simplicity and ease of understanding.

There is actually a max(computed_limit,10) see here but if I didn't miss something a wrong value was possibly shown to the user on their profile, see here

NWF_Renim said:

This also hits again against comic type post uploads.

Why do you think that it hits comic posts harder? Because one will have to upload more images at once or because comics have a higher chance of not going to be approved?

If it's the former Borrator's suggestion might work.

Zelinkokitsune said:
The Old (current?) system does a better job at providing a 'punishment' for posters with high deletion rates as it's a steady up ticking of the penalty compared to the rate where the additional slots are rewarded out.

The old system doesn't punish users with hundreds or thousands of approved images (and at the same time high deletion rates) at all, because with the old system such a user will "only" have a upload limit of say 200 instead of 500 but for all intents and purpose it won't matter if you still have such a high limit.

-------

I did some number crunching (disclaimer: I only looked at the posts from 2015-08-15, so it isn't really representative, but should still show tendencies):

On 2015-08-15 there where 752 posts by 123 different users.

36 user(s) with 1 uploads.
48 user(s) with 2-5 uploads.
25 user(s) with 6-10 uploads.
8 user(s) with 11-20 uploads.
3 user(s) with 21-30 uploads.
1 user(s) with 31-40 uploads.
1 user(s) with 41-50 uploads.
1 user(s) with 51+ uploads.

Assuming:
a) Most users have a base upload limit of 50 (see above)
and
b) Most users will have a deletion chance near 0% (see albert's post) and therefore their real upload limit will be near their base upload limit
I don't think a lot of users will be negatively affected / hit the limit.

Also of mention is that a relatively large number of members will probably have a higher limit than before, namely those with a low number of uploads and at the same time with a low chance of deletion and an "old" account.

-----
Edit: code used to compute this: http://pastebin.com/bc8yJEmT

Updated

SD-DAken said:
Because one will have to upload more images at once or because comics have a higher chance of not going to be approved?

They're more likely to take out a large chunk of your fixed uploads in a given day (the same is true for image sets), and if you're only active a limited number of days a week having them take out a large chunk of your fixed cap can act as a deterrent to uploading sets of images. Users may feel more encouraged to upload non-set posts because they'd feel that they're getting the most out of their limited slots in the day that they can upload anything.

The pending system at least gave them the chance of lucking out with fast approvals that renewed their slots, so there was value in taking the gamble of filling your slots with something that would consume a large chunk of them. It's kind of basic psychic 101. Doing a behavior that still occasionally rewards out still encourages a person to do that behavior, even if it doesn't work out most of the time.

There are fewer people (from my perspective) that approve comics and there is inherently lower scores and the like tied to comics, so the reward for uploading them was already lower from the get go. Something like this merely makes the reward even lower and further discourages people from uploading them.

Any new formula where the post throttle gets removed over time is good in my book. Sometimes people just get unlucky with approvals and they shouldn't ultimately be cut off for that. I've seen tons of good stuff die in the queue over the years. But like Renim is saying, this could kill comic/doujin pools and I'd also like to see a definitive statement on whether or not that's intentional.

Updated

Sacriven said:

Well tbh, this new formula punishes non-Contributor old users too hard. For old users, their upload limits can be called as their "pride" or "worth" for them. They work hard by uploading quality pics in daily basis, even some of them do it on hourly basis. All of it only to raise their upload limits, so it can keep them motivated to upload even more quality pics, forming a cycle. Albert said that the old formula is broken and unfair for new users, but I beg to differ. These old users (except Contributors) worked really hard and spent their precious time to reach that upload limit level. This new formula is just too much, it's just like mowing all of their efforts in a single go (in my eyes).

I think the new formula's con is outweighed the pros, but that's just me. I have no power or anything to change that.

Just leaving my opinion here.

Sacriven said:
These old users (except Contributors) worked really hard and spent their precious time to reach that upload limit level.

Do you think people just become Contributors on accident? Frankly having a high upload limit should be something you don't want, as it means you've been uploading a lot but you still can't maintain a reasonable level of quality.

ShadowbladeEdge said:

Do you think people just become Contributors on accident? Frankly having a high upload limit should be something you don't want, as it means you've been uploading a lot but you still can't maintain a reasonable level of quality.

I thought I did when I first got it (and I still do).

Anyways, I do agree. Quality over the quantity (upload limit).

ShadowbladeEdge said:

Do you think people just become Contributors on accident? Frankly having a high upload limit should be something you don't want, as it means you've been uploading a lot but you still can't maintain a reasonable level of quality.

Of course not. What I emphasize here is the "upload limit level" not the "hard work". Seems like my choice of words is a bit off, sorry.

Anyways, I do agree. Quality over the quantity (upload limit).

You said that as if quantity is always a bad part. Let's just say, a quantity of quality images? Is it a bad thing?

Updated

1 2 3 4 5 6 7 9