Danbooru 2 Issues Topic

Strobe_Red said:

Looking to flag a post, but the Flag option is missing from where it should be. Anyone know any work arounds?

Member-level users can flag one post per day (see help:users). If you've already flagged a post today, wait until tomorrow or try posting in the Deletion Request Thread. If you haven't flagged a post today, then there might be a bug happening

Yeah I've never flagged anything before. I checked later and it's there now though.

Been getting near constant 502's this past hour or so.

Ars said:

Been getting near constant 502's this past hour or so.

Seems like there are problems with https://shima.donmai.us

I'm getting a "that page does not exist" error when trying to browse for tags. Seems to be any tag.

I'm going to implement rate limiting set to 4 requests per second on anything underneath /posts. I can tweak this value based on how often people are hitting 503s.

albert said:

I'm going to implement rate limiting set to 4 requests per second on anything underneath /posts. I can tweak this value based on how often people are hitting 503s.

Ouch. I don't have any metrics on my API use, but I'm probably going to be hitting that daily. My monthly bad pixiv id scans will be hitting it hard.

Just to chime in myself, I've already started seeing odd behavior with my scripts, getting random HTTP 404 errors instead of HTTP 500-600 errors. This is a bit of an issue as a 404 signals to my script that something is drastically wrong and thus causes a hard stop. If a HTTP 429 (rate limit) or something similar was sent instead, that would be better as I've already programmed my scripts to handle 429 errors.

https://imgur.com/a/8KTxD9R. This time it was someone scraping the users/show endpoint on shima, not posts. They peaked at ~50 requests per second and user profiles are slow (~500ms per request).

The rate limiting is more like 1 per sec rather than 4 per sec. Doing a Ctrl+Left-Click twice in half second intervals will trigger it. Is it it limited to loading 1 picture at a time?

The rate limiting returns 404s, not 503s, 429s or similar.

Is the server under load, ddos or something?

Getting real sick of sites implementing shittier changes in the name of "better", pixiv going for js bloat, and the panda UI changes.

@albert this rate-limiting is sub-optimal, I'm getting 404s during *browsing*, and I'm not even doing it that fast...like, I'm literally looking at an image, adding it to my favorites, then going to the next, after several seconds.
I'm pretty sure @deltarz had it right in his post previously. This method isn't working correctly.
Edit:
Using hotkey nav makes it 404, but opening the next image in a new tab by middle-clicking on 'next' doesn't, how odd.

DeusExCalamus said:

@albert this rate-limiting is sub-optimal, I'm getting 404s during *browsing*, and I'm not even doing it that fast...like, I'm literally looking at an image, adding it to my favorites, then going to the next, after several seconds.
I'm pretty sure @deltarz had it right in his post previously. This method isn't working correctly.
Edit:
Using hotkey nav makes it 404, but opening the next image in a new tab by middle-clicking on 'next' doesn't, how odd.

Not only the hotkey. Just pressing the next post button grants you a 404.

I got a flood of "405 Not Allowed" trying to add tags to posts--is that the new rate limiting?

I've bumped the limit on posts to 10 requests/second (up from 4) and added one for users to 5 request/second.

albert said:

I've bumped the limit on posts to 10 requests/second (up from 4) and added one for users to 5 request/second.

Still getting 404 errors when going to ex user:deusexcalamus and using the right/left hotkey nav to go back and forth through them.

Now you get a 404 if you open 3 images. Could you explain how requests per second actually translate to loading a /posts/ page? For sure it isn't 4 images or 10 images.

And what brought about the rate limiting? Server load, costs, ddos??

EDIT: Never mind, don't know how missed that, thanks evazion.

Read the thread. The site was going down because of heavy bot activity. That's what all the complaints about 502 errors were about.

I've tweaked it to keep it back to 10 reqs/second but with a burst of 20. From observing the logs it seems like the only thing triggering it now is Atom feeds and webcrawlers. I've also updated the config to have Nginx return a 429 when throttling.

Can't upload from DeviantArt:
https://danbooru.donmai.us/uploads/2418223