[Userscript] IndexedAutocomplete

IndexedAutocomplete switches the Danbooru storage mechanism for saving autocomplete entries, using IndexedDB instead of Local Storage.

Installation

• Chrome/Firefox: Install Tampermonkey

Project page

• https://github.com/BrokenEagle/JavaScripts

Main script

• https://raw.githubusercontent.com/BrokenEagle/JavaScripts/master/IndexedAutocomplete.user.js

Usage notes

It's used mostly the same as autocomplete. The primary difference is under the covers, in how the data is stored and retrieved. The autocomplete also covers more inputs than are normally covered by Danbooru.

Note: It only enables on browsers that support IndexedDB, which for sure includes both Chrome and Firefox.

With Version 16, a new functionality has been added beyond what is provided by Danbooru. Whenever a user selects a data item with Tab or Enter, that choice will appear at the top of results for at least the next 48 hour period (the period is changeable by the user). Reusing that same choice renews that period.

With Version 19 the different sources for the tag autocomplete can be grouped and arranged by source.

With Version 20 an alternate weighting scheme can be applied for the various sources.

With Version 24 an already used mechanism shows which tags have already been added to the tag edit box.

With Version 27 alternate related tag comparisons can be used. Additionally, tag autocomplete can now be used on any non-autocomplete text field, which is activated/deactivated with the hotkey Alt+A.

With Version 28, alternate options are now available with text autocomplete with the hotkeys Alt+1, Alt+2, Alt+3. (See forum #168588 for more info)

Final

Any suggestions or feedback is appreciated.

Latest edits

• (2022-09-04)
• Updated script link in OP
• (2022-09-05)
• Version 29 Added two new options to adjust to the new word match autocomplete (forum #220660)

Versions

Pushed Version 3, which stores the pool autocomplete results, i.e. the autocomplete when you start a tag with pool:. This marks a greater departure from the original autocomplete, as with that, only the regular tags were being stored.

Pushed Version 4, which does the user autocomplete results (user:). This also does the autocomplete for @ mentions.

Edit:

Pushed a minor update to this version, which fixes the program loader.

Pushed Version 5, which does the favorite group (favgroup:) and saved search (search:) results.

Edit:

Pushed a minor update, which fixes a bug with the user source.

Pushed Version 6, which stores the related tag results. This includes the buttons Related tags, General, Artist, Copyright, Character.

Pushed Version 7 which varies the expiration time by type and count. Below are the approximate values/ranges.

• tag (count):
• < 100: 7 days
• 100 - 1000: 7 - 14 days
• 1000 - 10000: 14 - 21 days
• 10000 - 100000: 21 - 28 days
• > 100000: 28 days
• pool (count):
• < 10: 7 days
• 10 - 100: 7 - 14 days
• 100 - 1000: 14 - 21 days
• 1000 - 10000: 21 - 28 days
• > 10000: 28 days
• user: 28 days
• favgroup: 7 days
• search: 7 days
• relatedtag: 7 days

User was left high as that's not that often to change, whereas favgroup/search could be changed often so were made low, and relatedtag was left low since no post count is returned with the results.

Pushed Version 8 which stores the wiki page results. Additionally, it fixes an issue where the autocomplete doesn't function on wiki page version pages.

The expiration schema is the same as the tag category if a tag exists, otherwise it uses an expiration time of 7 days.

Edit:

Pushed a minor update, fixing a bug with handling blank wiki entries.

Pushed Version 9 which stores the artist results.

It uses the same expiration schema as wiki page results.

BrokenEagle98 said:

Pushed Version 9 which stores the artist results.

It uses the same expiration schema as wiki page results.

Edit:

Pushed a minor update, fixing how pools save data.

Pushed Version 10 which stores non-tag pool results. These are the results that come from the pools page and also from the Add to pool dialog box. It uses the same data as used by the pool: tags, so something stored by either can be used by either.

So the reason why someone would use this script is for a faster autocomplete? I don't really get the real hook for using this besides the script utilizing a new technology.

tapnek said:

So the reason why someone would use this script is for a faster autocomplete? I don't really get the real hook for using this besides the script utilizing a new technology.

Not an easy answer without getting too technical, but I'll give it a try.

Current autocomplete results are stored on your computer using what is called local storage. The reason it does this is that otherwise it would have to make a network call every time which is significantly slower (~250-500ms). Additionally, you might have noticed that this becomes completely non-functional when Danbooru is very slooooow.

Local storage has the advantage of speed, but is very limited in size. To support all browsers, Danbooru limits itself to just below 5MB. This might sound like a lot, but works out to be around only 4000 autocomplete results. This, again, might sound like a lot but it's not. Every letter and combination of letters that you type in the tag box or search box counts as an entry. For myself, it seemed like I was reaching the 5MB size limit once every 3 days.

Currently when Danbooru reaches the limit, it purges all results, going back to an empty state. So all the advantage you have gained in speed by storing those results locally are lost.

IndexedDB on the other hand is slightly slower than local storage (1ms vs 15 ms average), however it has the benefit of being near limitless in size, at least from what I've read and what I've tested (up to 35 MB). This means that results are no longer lost every few days, but instead they can last almost indefinitely, although to prevent stale tag information they currently expire from 7 days to 28 days depending on the post count.

Overall, this means that on average the autocomplete should perform faster than the current setup. Additionally, to reduce the speed penalty for IndexedDB, the results are cached in session storage which is just as fast as local storage.

The above covers just the usage of stored autocomplete for the tag box and search box, and does not cover any other usage of autocomplete on this site. Those results are currently not stored, which requires going to the network every time which is very slow. With this script, the results are stored so that the lookup penalty is only encountered on the first iteration. This pretty much guarantees that the results will on average always be faster than without. Besides autocomplete, it currently also stores the related tag lookup, which otherwise would require a network call every single time.

TL;DR yes it is faster on average.

When will the script support other usages of autocomplete?

tapnek said:

When will the script support other usages of autocomplete?

Well, I've pretty much exhausted all of this site's current usage of autcomplete, so I've started to look where else it could be used. Right now I've been testing the addition of it to the Dmail TO and FROM username text fields.

If you have any other ideas, feel free to share and I'll see what I can do.

Pushed some minor fixes to account for the latest deployment of Danbooru (forum #141030).

• Minor versions
• (2017-12-27)
• 10.1 Fixed reliance on missing short category names meta
• 10.2 Fixed reliance on missing related tags button meta
• (2017-12-28)
• 10.3 Fixed for wiki page versions autocomplete inclusion

Pushed Version 11 which adds in user autocomplete to all available username inputs. This adds autocomplete in over a dozen different pages that mainline Danbooru currently doesn't support.

• Minor versions
• (2018-01-01)
• 11.1 Fixed missing autocomplete
• On post version User searchbox
• On artist version Name searchbox
• (2018-01-02)
• 11.2 Fixed missing autocomplete on post index saved search dialog
• 11.3 Fixed IndexedDB save of data
• (2018-01-02)
• 11.4 Fixed tag antecedent name

Pushed Version 12 which adds in a full client-side data validation in addition to some major code refactoring.

Pushed Version 13 which adds the artist finder data to session storage. It's not being stored in persistent storage since URLs are so unique that it's not likely to generate very many hits. The primary use case is when you edit a post multiple times, or when you expand/contract the Related Tags section in the detachable edit box.

• Minor versions
• (2018-03-05)
• 13.1 Fixed deprecated use of UI autocomplete

Pushed Version 14. Like my CurrentUploads script (topic #15169), I updated this script to use my library, which overall reduced the code size by ~25%. Additionally, I fixed a bug in the artist finder button that was introduced by 13.1 and also the latest jQuery version deployment.

• Minor versions
• (2018-04-27)
• 14.1 Fixed bug on checked cached values
• (2018-04-29)
• 14.2
• Fixed bug with highlighting some autocomplete values
• Applied same fix as issue #3688
• (2018-05-15)
• 14.3
• Fixed bug with wrong name of a function
• (2018-07-23)
• 14.4
• Updated to use newest library (Version 4)
• Added back in program checks accidentally removed in a prior version
• 14.5
• Set debug log levels
• (2018-08-02)
• 14.6
• Add more variable specifications
• Mostly disabled script due to issue #3795
• Turn off debug messages by default
• (2018-08-03)
• 14.7
• Used new method of overriding Danbooru functions
• Added temporary patch because of Danbooru's inaccessible code (issue #3795)
• (2018-08-04)
• 14.8
• Added more patching to fix translated tags
• Fixed issue with missing artist link (issue #3799)
• 14.9
• Added more patching to fix artist tags
• 14.10
• Fixed usage of read cookie function
• (2018-08-06)
• 14.11
• Added pruning function
• (2018-08-15)
• 14.12
• Removed temporary patch as the needed functions were exported

Pushed Version 15 which adds forum topic autocomplete. Additionally, it fixes a few input boxes missing autocomplete, as well as some code refactoring that reduced code size by ~10%.

For the forum topic autocomplete, it adds a quick search input labeled "Search topic" next to the existing "Search forum" input. The old input was kept as it performs a different search looking in the forum bodies for the text. Also, the "Search topic" input was only added to forum topic pages and not forum post pages (check the URL address for which one you're on). The last place it was added to is the Title input on the Forum post search page.

The expiration time for forum topic data is currently at 1 week. I thought about making it 1 month, but the forum results are retrieved in most recently updated first order, and the forum can change quite a bit in 1 month leaving the results stale.