Posted under General
I am trying to get a count of various posts (for a little side project of mine, basically graphing a count progress of some tags), but when I try to GET http://danbooru.donmai.us/counts/posts.json?tags , it returns the "No Access-Control-Allow-Origin header" message. Is this intentional because of post count being computationally expensive or is this just overlooking something (from my or devs' side)?
What URL are you trying to GET exactly? Just that with actually no tags? I’m getting various unexpected results (such as “204 No Content” with actual content), but not the same as you.
Are you using your API key? If not, anything that requires Gold access will fail.
Which application or library are you using for HTTP access?
kittey said:
What URL are you trying to GET exactly? Just that with actually no tags? I’m getting various unexpected results (such as “204 No Content” with actual content), but not the same as you.
Are you using your API key? If not, anything that requires Gold access will fail.
I tried out various stuff and it all returned results so far (like rating:safe or test - which returns 101 results http://danbooru.donmai.us/counts/posts.json?tags=test ). Just in case, it returns this JSON string to me for test: {"counts": {"posts": 101}} - but only if I visit it manually, if I call it from a script, I meet the error mentioned above.
Tried it signed out, with cleared session and in anonymous window, and it always returned a result (I think it's a normal level, though not 100 % sure).
EDIT: I am using simple $.get from jQuery - I can reach other API endpoints (like http://danbooru.donmai.us/posts.json?tags=test ), but not the count.
Mh, can’t check that. Maybe one of our user script masters will have an idea.
Oddly, though, viewing http://danbooru.donmai.us/counts/posts.json?tags=test in a browser when logged in gives me a count of 100 and viewing the same URL when logged out or downloading it with wget or w3m gives me 101. Reproducible every time. :-?
The tag search says that 101 should be correct.
Edit: Nvm. I have the “Deleted post filter” on and test has exactly one deleted post.
Updated
It sounds like you're running into browser cross-origin restrictions. Javascript running on one site isn't allowed to make AJAX requests to other third-party sites. Browsers restrict these requests for security reasons. Imagine if I hid $.get("http://danbooru.donmai.us/dmails.json") on a site I owned and tricked you into visiting my site. I'd be able to read your dmails if that were allowed.
Sites can allow cross-origin requests by sending a Access-Control-Allow-Origin: * header. Currently only /posts.json is whitelisted in this way, which is why that works but other endpoints don't. Other endpoints could be whitelisted but there are security issues to keep in mind. I'll make an issue for it.
In the meantime, if you happen to be doing this within a userscript you can bypass cross-origin restrictions by using GM_xmlHttpRequest. See here for how to patch jQuery to work with GM_xmlHttpRequest. Otherwise, you'll have to run your script outside a browser environment.
PS: if you just want to scrape tag counts then look at http://danbooru.donmai.us/cache/tags.json.gz. It's a full dump of the tags table updated nightly.
I see - I am aware of the header issue, but thought the API was completely free to read. I am not sure what I am doing is worth being a userscript, though, and scraping tags sounds unusable to me, since I planned to use it along with date meta-tags (i.e. call the tag with age:1d, then age:2d, up to age:7d and then graph the differences, for a week graph, with month/week and year/month alternatives).
Thanks for the explanation anyway o7
