Posted under Bugs & Features
I'm trying to use Grabber (https://github.com/Bionus/imgbrd-grabber) to do searches using my login and API key. However, Cloudflare has started serving up an anti-bot challenge page which obviously blocks programmatic API access (I can still access the site with a browser with javascript enabled). I assume this is a misconfiguration of cloudflare somehow. Or is programmatic access using login and API key just up to the whims of cloudflare?
Is this the same issue mention in topic #22822?
Unbreakable said:
Is this the same issue mention in topic #22822?
Looks like it, but I am using an API key and am still getting blocked.
It appears that Grabber is using the user-agent "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0" if that helps.
i was having the same issue earlier; managed to get it working by manually setting a user-agent header for danbooru in grabber.
to do this in grabber:-
kittey said:
Instead of pretending to be a browser and potentially getting that browser banned, you should set your user agent to something like “Danbooru user [yourusername]” so that Evazion knows who to yell at when you’re going overboard.
If they're using an api key then that's already possible
kittey said:
Instead of pretending to be a browser and potentially getting that browser banned, you should set your user agent to something like “Danbooru user [yourusername]” so that Evazion knows who to yell at when you’re going overboard.
tbf i had no idea you could just set your user agent to absolutely whatever you wanted
Updated
its.ok said:
i was having the same issue earlier; managed to get it working by manually setting a user-agent header for danbooru in grabber.
to do this in grabber:-
- navigate to danbooru site options in grabber
- click the headers tab and click add to add a blank header
- double click the empty boxes to set the header's name to "user-agent" and value to whatever your browser reports as its user agent (or if u cant be bothered, the latest chrome ua is "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36").
Thank you so much bro！
kittey said:
Instead of pretending to be a browser and potentially getting that browser banned, you should set your user agent to something like “Danbooru user [yourusername]” so that Evazion knows who to yell at when you’re going overboard.
By the time I get my usual download manager patched to be able to do that without spoofing the user-agent with a local proxy, using Grabber with that trick seems to have done the job. Thanks for the tip.
its.ok said:
i was having the same issue earlier; managed to get it working by manually setting a user-agent header for danbooru in grabber.
to do this in grabber:-
- navigate to danbooru site options in grabber
- click the headers tab and click add to add a blank header
- double click the empty boxes to set the header's name to "user-agent" and value to whatever your browser reports as its user agent (or if u cant be bothered, the latest chrome ua is "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36").
Still no luck. Earlier the source used to work fine but one day it just stopped working.
I am still getting the same 'possible reasons: server offline' message while searching on Danbooru from Grabber. I did the header thing, used latest chrome ua value from my browser. experimented with changing user-agent name.
I also tried deleting the source, readding it, putting my username and api_key from danbooru site. Then did all things above again. Still no luck. Earlier the source used to work fine but one day it just stopped working.
On my log it is showing- ''[Warning] [danbooru.donmai.us] No valid source of the site returned result''. any idea for fix, I am out of any idea.
hawkeye223 said:
Still no luck. Earlier the source used to work fine but one day it just stopped working.
I am still getting the same 'possible reasons: server offline' message while searching on Danbooru from Grabber. I did the header thing, used latest chrome ua value from my browser. experimented with changing user-agent name.I also tried deleting the source, readding it, putting my username and api_key from danbooru site. Then did all things above again. Still no luck. Earlier the source used to work fine but one day it just stopped working.
On my log it is showing- ''[Warning] [danbooru.donmai.us] No valid source of the site returned result''. any idea for fix, I am out of any idea.
Did you follow the suggestion to use the Chrome user agent? If so, that’s why it’s broken now. It’s no longer allowed to pretend to be a browser. Try setting the user agent to “Danbooru user hawkeye223” instead and see if that helps.
I just tried using a head formatted as suggested - with my own username obviously - and I still get the same results. Even tried with an API key and no difference.
Just returns the error: [Error] [danbooru.donmai.us][Xml] Cloudflare wall for 'https ://danbooru.donmai.us/posts.xml?limit=20&page=1&tags=&login=Grahf&api_key=[my api key which I'm not sharing here for obvious reasons]
kittey said:
Did you follow the suggestion to use the Chrome user agent? If so, that’s why it’s broken now. It’s no longer allowed to pretend to be a browser. Try setting the user agent to “Danbooru user hawkeye223” instead and see if that helps.
yup tried that too, didn't worked. Someone has even a raised a issue for this on imgbrd-grabber github
https://github.com/Bionus/imgbrd-grabber/issues/2912
If you use imgbrd-grabber app too, can you confirm if its working on your side or everyone is in the same boat?
