Donmai

Danbooru 2 Issues Topic

Posted under General

This topic has been locked.

I've restarted the workers and cleared out ephemeral jobs.

There are a few old post download jobs that are slow that are clogging up the queue but things should return to normal within the hour.

albert said:

I've restarted the workers and cleared out ephemeral jobs.

There are a few old post download jobs that are slow that are clogging up the queue but things should return to normal within the hour.

Confirming that my saved search is now working again. Many thanks, Albert.

albert said:

I've restarted the workers and cleared out ephemeral jobs.

There are a few old post download jobs that are slow that are clogging up the queue but things should return to normal within the hour.

That is great, it worked. For a day... This seems to be rather constant, for a bug.

James-Sarrowtail said:

That is great, it worked. For a day... This seems to be rather constant, for a bug.

It still works. The saved searches seem to work on demand now, with a delay. For example, visiting my profile page in the morning typically shows empty saved searches. However, after waiting ~30 seconds and refreshing the page, all of the saved searches show up. I imagine the same is true when using the search: metatag, i.e. you have to perform the search once, perhaps wait 30 seconds, then search again.

BrokenEagle98 said:

It still works. The saved searches seem to work on demand now, with a delay. For example, visiting my profile page in the morning typically shows empty saved searches. However, after waiting ~30 seconds and refreshing the page, all of the saved searches show up. I imagine the same is true when using the search: metatag, i.e. you have to perform the search once, perhaps wait 30 seconds, then search again.

It’s been like that for a while already. Unfortunately, viewing your profile or using the search: metatag doesn’t guarantee that all your saved searches will update. Usually, half of my saved searches haven’t updated after 30 seconds. Some even haven’t updated after a minute, so I basically keep reloading my profile several times for five minutes or so, hoping that that will be enough. Loading my profile once and then again five minutes later is definitely not enough.

I’d really like to have some kind of “last update” timestamp on the list of saved searches or something like that.

Having a ridiculous amount of errors this entire night from basic upload page not even completing "Loading similar...", a bunch of parse errors ,and now I'm just being whaled upon by 502s.

I'm going to roll out a change to saved searches that makes them update immediately, meaning you should never see a blank page again. This will only be enabled for gold users to see how it impacts performance, then I might roll it out for everyone.

albert said:

I'm going to roll out a change to saved searches that makes them update immediately, meaning you should never see a blank page again. This will only be enabled for gold users to see how it impacts performance, then I might roll it out for everyone.

It takes 45 seconds for the profile to load now, and that's with only 69 saved searches. Since Platinum+ can have up to 1000 searches, I imagine that some users' profiles may never load now.

It might be better to do some kind of load-on-demand with Javascript, where it will only update the search queries once the search label becomes visible in the profile.

Alternatively there could be a button or something below the favorites and uploads that lets you populate all your saved searches, visiting your profile to check your mail is a real pain in the ass now although it can be bypassed by going through settings.

albert said:

As a compromise I could just cap it to 3-5 updates and delay the rest.

There still seems to be a lot of delay. It still takes around 20 seconds at longest, and it requires about a dozen page refreshes before the profile load takes it's expected load time of around 5 seconds.

I've been monitoring the delayed jobs queue over the last week when loading my profile, and I've noticed that some queries seem to get executed even after multiple page refreshes. I'm guessing that if a query times out, that it gets executed again at the next page refresh?

BrokenEagle98 said:

I'm guessing that if a query times out, that it gets executed again at the next page refresh?

Incidentally, what timeout do saved searches use? The same as the user who first saved it? The one who triggered it to be run? Always a fixed value?

Looks like the profile loads are still excessively long. I tried loading my profile this morning when it was most likely that all of the queries would have expired, and it took close to a minute for the profile page to load.

kittey said:

Incidentally, what timeout do saved searches use? The same as the user who first saved it? The one who triggered it to be run? Always a fixed value?

For the ones that are run synchronously, I believe it's the user that triggered it. Unless explicitly specified otherwise, every action on behalf of a user is run under that user's context. For the delayed jobs it's most likely system, though that still has the same timeout cap as Platinum (12 seconds). Also, when a ton of searches get executed at the same time (like on a profile load), the server gets overloaded and the queries are more likely to time out.