Donmai

[Userscript] ValidateTagInput

Posted under General

Pushed Version 12, which fixed Unbreakables issue from forum #136728.

Basically, the script would sometimes gain execution before Danbooru. This would cause it to have double-binded events when Danbooru came in afterwards and rebound those same keys.

The above fix forces the script to wait until Danbooru has reached a certain point in its processing.

Pushed Version 13 which adds a check rating validator.

It basically does the same thing that Danbooru does when you don't enter a rating after submitting an upload, but in contrast it doesn't reload the page and thus remove the image preview from view.

This came about from the multitude of times that I accidentally submitted an image and then had to deal with not having the image preview anymore. There are ways to deal with that, the primary being to just submit the image with the tags as is and then finish tagging the post upon upload completion. With the update though, the script will stop you from reaching that state unless you check the Skip validation while keeping the rating empty.

Pushed Version 14 which has no new external features, but which changed how data was stored and also added a validator for that stored data, which will keep bad stored data from causing script errors, as well as facilitating changes to the data model in the future.

Pushed Version 16 which adds a Reset cached data link to user settings (under Basic Settings). This will delete all of the data being stored by the tag validator as well as autocomplete.

The reason to do this for either is if there are new approved aliases or implications that aren't being accounted for in either the autocomplete or the tag validator.

Another reason for autocomplete is after the populating of a new tag. Autocomplete will store the first value it gets, which for a new tag will most likely be a count of 1. This will make it unlikely that the tag will show up in autocomplete until those values expire (1 week).

Updated

Pushed Version 17 which includes a migration to using IndexedDB when possible for storing data.

This change is related to issue #3335, when it became apparent that there was a contention for LocalStorage space between Danbooru's autocomplete and the tag validator. Danbooru currently uses a maximum cap of 2.5M characters tailored to the lowest capable browsers out there. After this limit is reached, it prunes all of the values and starts from scratch. As the tag validator's storage needs grow over time as well, this would likely cause more and more pruning actions from Danbooru.

To deal with this, the tag validator's storage is being migrated to IndexedDB where possible, and when not possible it limits the tag validators data to no more than 1M characters, doing a pruning action similar to Danbooru when it reaches that size. When using IndexedDB, it makes use of the mostly unused SessionStorage, which is pretty much LocalStorage but limited to a single tab.

For those interested in the performance tradeoffs between IndexedDB and LocalStorage, the following are an example of numbers seen when looking at the performance metrics on one of my slower systems. Also, although it says SessionStorage, the numbers seen for LocalStorage were pretty much equivalent.

NameNumber of RequestsAverage Time (ms)Outliers Removed
SessionStorage360.382
IndexedDB285810
Network22432.376

Pushed a minor update with Version 18, mostly due to a minor bug found that prevented the local storage pruner from working on data stored with Versions 16 and prior. Besides the above, the debug statements and code are now conditionally run. The performance increase for this would be minor, however it also reduces the clutter in the debug console for when the script is not being tested.

Pushed Version 19 to account for the new Meta tag category.

As an update, issue #3352 was submitted to make these validations baseline, and two fixes have already been submitted (issue #3357, issue #3362). These two add in the Tag Add validator and the Rating Exists validator, although with the former it set to notify the user after the post save for now. If needed, it may be moved to before the post save as it is with the ValidateTagInput user script.

Pushed Version 20 primarily since it fixes a bug in the data model check. The rating existence validator has also been modified to check for the rating in the tag box.

Edit:

Pushed a minor update to this version.

Updated

Pushed Version 21 fixing a bug pointed out by evazion on Discord, which affected the post index page for Member-level users and those not logged in as Anonymous.

Edit:

Pushed a minor update to this version.

Updated

Pushed Version 22 which removes the rating existence validator. This was done in response to issue #3362 being merged and deployed. So the check still exists, but now it's being done by Danbooru. Additionally, that check also does the file/source existence validations as well.

Pushed Version 24 which had a major code refactor, but function-wise it also adds a Check button that will check the tags but not submit them. The reason I added this is because I have encountered at least several times where I was batch uploading images as quickly as possible to try to maintain continuity, and the validation check would sometimes halt the uploading process. Besides that use case, one might also want to check the tags as they're going along instead of at the very end.

As a side note, the clear cache option has been removed for now (user settings), as I want to have a solution which can accommodate all of my scripts.

  • Minor versions
    • (2018-07-23)
      • 24.1
        • Updated to use newest library (Version 4)
    • (2018-08-02)
      • 24.2
        • Add more variable specifications
        • Turn off debug messages by default
    • (2018-08-03)
      • 24.3
        • Converted script to use available functions instead

Updated

Pushed Version 25 which adds a settings menu available on the same page as the Danbooru settings (My Account >> Settings) under Userscript Menus. The specific settings for this script are under the tab ValidateTaginput. The settings "should" instantly reflect in all other open tags with the latest version of the script running.

Minor versions
  • (2018-09-09)
    • 25.1 - Updated to use newest library (Version 5)
      • Added the rating validation back in as optional (including source validation)
      • Several code fixes/refactors
    • 25.2 - Fixed factory reset broadcast
    • 25.3 - Fixed not removing typetags from consideration
  • (2018-12-31)
    • 25.4 - Fixed incorrect pruning parameter.
  • (2019-01-03)
    • 25.5
      • Updated library version
        • ~30% reduction in code size
      • Refactored code to standard

Updated

Pushed Version 26 which adds a bunch of new validations on post loads, many of which are already used by Danbooru after creating a new post. Warning messages are setup to occur at most once per post per tab session, although this can be changed in user settings.

Besides that, the cache editor added can now edit program data. This makes it possible to transfer settings around.

New validators
  • Artist validator
    • Posts without an artist tag
      • Posts with official art are ignored
      • Posts from a first-party source (Pixiv, Twitter) get a warning
      • Posts without artist request get an additional add-on warning
      • Posts with an available artist tag to add get a warning
    • Posts with an artist tag
      • Artists that have no artist entry get a warning
  • Copyright validator
  • General validator
    • Posts without a certain level of general tags get a warning
    • There are 3 levels with different warning messages
    • The thresholds for those levels can be configured in user settings
Other new items
  • Added the cache editor being prototyping (forum #154196, forum #154240)
    • It can now edit Program Data the same as Cache Data
    • User settings can now be copied from one browser/machine to another
    • Changes to user settings are immediately reflected on the settings page
Minor versions
  • (2019-01-30)
    • 26.1 - Fixed issue with program data validator
    • 26.2 - Fixed issue with unavailable flat function in Firefox
  • (2019-01-31)
    • 26.3 - Added caching to artist data
  • (2019-02-01)
    • 26.4 - Fixed issue with artist name parser

Updated

worldendDominator said:

(Moving from my report in site issues thread)

I opened a few random posts from my userpage, and they all got warnings.

post #3400296 - all 3 (gentags, copyright, artist)
post #3400263 - copyright and artist
post #3398804 - artist
post #3231136 - copyright and artist
post #3398653 - all 3
post #3369437 - copyright and tags

Gentags warning seems to work correctly, but other 2 are broken.

The only warnings I got were on post #3400296, post #3398653, post #3369437, and only the general tags warning since they all have less than 30 general tags.

I tested it out on both Chrome and Firefox, and got the same with both browsers. I did find an issue which caused the Artist validator to fail on Firefox that I fixed in Version 26.2, but that wouldn't cause the notice to be fired like you're showing.

Which browser are you using? Also, what other user scripts are you running? There may be a conflict.

One thing you can do is turn on debug statements and see what shows up in the development console (F12). This can be done by altering one of the first lines in the script to change the value from false to true.

JSPLib.debug.debug_console = true;

I just thought of this, but another issue could be that certain elements aren't available for whatever reason when the script is run.

For the copyright validator, you can run the following in the development console on the post page and see what it says.

$(".copyright-tag-list .category-3 .wiki-link").length

For the artist validator, you can do the same with the following.

$(".artist-tag-list .category-1 .wiki-link").map((i,entry)=>{return decodeURI(entry.search.split("=")[1]);}).toArray();

Also, when you're getting the artist validation warnings, which ones are you actually getting. Same for the copyright warnings.

worldendDominator said:

Turns out it was some weird interaction with Better Better Booru. I guess I'll just disable those warning for now.

Ah okay, thanks for the information. I'll try to debug the script with Better Better Booru in the next week or so to see if I can replicate and get around those issues.

Pushed Version 27 which primarily integrates the new library code that was finalized today. This is coming out as a major version since a large majority of the code was changed while updating to the new library version and standardizing all of the affected userscripts.

Additionally, all of the menus were standardized, and fully working cache editors were incorporated. Also, a new control was added under cache settings which shows the size of all the userscript data amongst all of the various data storage.

Besides the above, the Artist validator was fixed so that the "Create new artist entry" link on existing artist tags would incorporate the tag name into the new artist entry.

Minor versions
  • (2019-01-30)
    • 27.1
      • Account for auto-implications in tag removal validation
      • This includes the *_school_uniform and *_(cosplay) tags
  • (2019-05-11)
    • 27.2
      • Make use of "only" parameter
      • Increase implication limit to 200
  • (2019-05-19)
    • 27.3 - Fix network function not working without ID
  • (2019-05-29)
    • 27.4 - Only check nonexistant tags for aliases

Updated

1 2 3