Donmai

[Userscript] CurrentUploads

Posted under General

CurrentUploads regularly checks the count of your uploads for the last 24 hours and compares it against the copyrights you uploaded as well as various time frames. The intended purpose is to allow a user to actively track how many uploads they've uploaded recently as well as the variety.

Installation

Project page
Main script
Usage notes

Clicking the Toggle Upload Table link will show or hide the upload count table. Clicking the STASH link will completely hide the table. To restore it, click the green link that now appears in the footer.

Program details

The time frames currently shown are 1 day, 1 week, 1 month, 1 year and all-time. The following is how often each time frame is queried:

  • 1 day: 5 minutes
  • 1 week: 1 hour
  • 1 month: 1 day
  • 1 year: 1 week
  • all-time: 1 month

The different copyrights it displays are from the user's uploads from the previous 24 hour period. Additionally, it will only show the base copyright, e.g. it will show Splatoon but not Splatoon 1 or Splatoon 2.

The statistics data is gathered automatically for the Daily period, but must be individually queried for other periods by clicking the name in the period header. The following is how long the statistics data is valid for each period.

  • 1 day: 5 minutes
  • 1 week: 1 day
  • 1 month: 1 week
  • 1 year: 1 month
  • all-time: 1 year
Member-level caveats

Member-level users do not have the copyrights shown. The reason is that it takes 3 tags to show the copyright information for a user by period (Ex: user:BrokenEagle98 girls_und_panzer age:..1d). Member users can only search by 2 tags though.

Final

Current suggestions/ideas are:

  • Add actual approval time statistics

Any suggestions or feedback is appreciated.

Latest edits

  • (2022-09-04) Updated script link in OP

Versions

Show
  • (2018-04-16)
    • Version 2 - Initial release
    • Version 3 - Adjusted table layout
      • Gave the table a fixed height <Discord: nonamethanks>
      • Fixed the copyright ordering <Discord:Unbreakable>
      • Rate limit network requests to reduce browser performance degradation
      • Present user with feedback during loading phase
  • (2018-04-18)
    • Version 4 - Heavily reduced network requests
      • Only checks user upload counts when post IDs change
      • Only checks site upload counts based on weekly average
  • (2018-04-21)
    • Version 5 - Updated to use new library
  • (2018-04-23)
    • Version 6 - Added statistics to day data
  • (2018-04-23)
    • Version 7 - Added ability to check other users <forum #145409: Provence>
      • Added ability to stash UI with the STASH link
      • When stashed, a link appears in the footer that can restore it
      • Changed how data is stored to save space
      • Made several other minor style changes
  • (2018-07-23)
    • Version 8 - Updated to use library
  • (2018-08-11)
    • Version 9
      • Added full stats for weekly/monthly periods
      • Added partial stats for yearly/all-time periods
      • Added daily pruning of expired data
      • Fixed usage by Member/Anonymous
      • Various other fixes and code refactor
  • (2018-09-05)
    • Version 10 - Add setting menu
  • (2018-09-13)
    • Version 11 - Add time statistics
  • (2018-10-01)
    • Version 12 - Add approver option
  • (2018-10-25)
    • Version 13
      • Added override option
        • For users with no uploads/approvals over previous day
      • Added copyright selection <forum #151552: Shinjidude>
        • Can select copyrights from the last week or month
        • Can add tags manually (if they exist)
  • (2019-01-31)
    • Version 14
      • Sortable columns - <Discord: nonamethanks>
      • Fixed position table headers
      • Data viewable in chart form
      • Added cache editor
  • (2019-02-14)
    • Version 15 - Updated to newest library
  • (2019-12-25)
    • Version 16 - Updated to newest library

Updated

Pushed Version 3 which fixed a few sorting/data issues as well as adjusting the table layout and appearance. Also limited the network requests to reduce browser performance degradation.

Edit:
  • Minor versions
  • (2018-04-16)
    • 3.1 Fixed bug where notice box was attaching to the incorrect header <Discord:Unbreakable>
  • (2018-04-17)
    • 3.2 Added versioning to the library

Updated

Pushed Version 4 which heavily reduces the network requests on regular renewal. The previous schema had a high number of network calls, and the more varied the copyrights being uploaded were, the higher the number of network calls.

The new schema primarily only rechecks counts when it detects a change in the user's uploads, which it can do as it now keeps a record of the previous network call to get all posts for the 24 hours. If a change is detected on a particular copyright, it will also recheck the site count for that copyright.

Otherwise, it uses the velocity of post uploads as a guideline for when to recheck the site copyright counts using the average upload count for the previous week. For example, if the average upload velocity for the original copyright is 8 minutes, then it will check the original copyright every 8 minutes even if it doesn't detect any change in the user's uploads. The maximum duration for this is currently set not to exceed a day.

Note: One caveat to the new schema is on new tabs or windows as the sessionStorage starts out blank, and so the program currently reverts to using the previous schema once for the first opening of the table.

Pushed Version 5, which while nothing major externally, internally it significantly revamped how library functions are called. Besides that, there were several minor bugfixes. Additionally, how the page unload event triggers code was also altered to allow for more than one program to use this event.

Could it also be used as a tool to look up what other user have uploaded?
That's especially useful for moderators when looking up the upload diversity of promoteable users, because a rule f thumb is: The more diverse, the better.

Provence said:

Could it also be used as a tool to look up what other user have uploaded?
That's especially useful for moderators when looking up the upload diversity of promoteable users, because a rule f thumb is: The more diverse, the better.

It already sort of can, and just by changing one line of code. However adding a user interface for that is a bit more work, and it will also need to be adjusted as things are currently programmed under the assumption that they will run only once. I'll add it to the list of suggested ideas in the OP and add it to my To-Do list.

Pushed Version 6 which adds statistics for the day data, as the day data is always available since it is used by the program. The different metrics are score, upscore, downscore, favcount, tagcount and gentags. For each metric, the following statistics are calculated.

  • Maximum (Max): The highest data in the set
  • Average (Avg): Mean
  • Standard Deviation (StD): Standard deviation
  • Outliers (Out): Removing all of the data that is more than 2 standard deviations from the average iteratively until either there are no more outliers or all are outliers
  • Adjusted (Adj): The average of the set with all outliers removed

Note: I'm working on the other time periods, but they require a little more work as it can take a while to collect all of the data and so the data collection function needs to be reentrant as well as being locked to use on no more than one tab at a time.

Pushed Version 7 which added the ability to query other users from the UI. Also changed how data was being stored as some storage limits were being hit early on when querying several other users. Finally, added an ability to stash the UI out of the way rather than turn it off when it won't be used in a while. Restoring the table can be done by clicking a link (green-colored) that now appears in the footer.

  • Minor versions
    • (2018-04-26)
      • 7.1 Fixed conflict with IndexedAutocomplete
    • (2018-04-29)
      • 7.2
        • Fixed naming of metric
        • Now closes the table when it's stashed
        • Other minor changes
    • (2018-05-12)
      • 7.3 Fixed naming of required variables
    • (2018-05-30)
      • 7.4 <Unbreakable Discord>
        • Fixed bug with table overflow not adjusting to new users

Updated

Pushed Version 8 that uses my library like my other scripts, which reduced the overall code size by ~12%.

  • Minor versions
    • (2018-07-30)
      • 8.1 Various bug fixes
        • Checking statistics would fail because of a faulty regex
        • Checking any user was crashing because of faulty validation
    • (2018-08-02)
      • 8.2
        • Add more variable specifications
        • Turn off debug messages by default
    • (2018-08-03)
      • 8.3
        • Converted script to use available functions instead

Updated

Pushed Version 9 which adds in statistics for the remaining time periods. It also finally adds in pruning of expired data, so that the database doesn't grow too large. Besides those two major items, there were several other fixes and code refactors.

Usage notes

To get the statistics for a time period other than Daily (which is automatically collected), click the column header for the desired time period. A countdown will start that will inform how many network calls are remaining.

Warning!! Every 100 posts takes about 1 second. If the tab loses focus, the network requests will progress slower as the tab will no longer have priority.

The Week and Month time periods, like the Day period, show the statistics for every copyright. As the storage requirements go up a magnitude of order for each period beyond that, the final two periods, Year and All-time, are more limited, only showing the statistics for all of the uploads for that time period.

As this data takes more resources to collect, the expirations for the various time periods have been lengthened compared to the count expirations shown in the OP.

  • Day: 5 minutes
  • Week: 1 day
  • Month: 1 week
  • Year: 1 month
  • All-time: 1 year

Updated

Pushed Version 10 which adds a settings menu available on the same page as the Danbooru settings (My Account >> Settings) under Userscript Menus. The specific settings for this script are under the tab CurrentUploads. The settings "should" instantly reflect in all other open tags with the latest version of the script running.

Minor versions
  • (2018-09-08)
    • 10.1 Updated library version
      • Alternate load for jQuery addon
      • Several minor code fixes and refactors
  • (2018-09-11)
    • 10.2 Fix not fully waiting for all network calls to finish
      • Was causing multiple N/A's to show up in the table, the default when data is not defined

Updated

Pushed Version 11, which adds the time statistics of week and day. All times are calculated using UTC.

For week, it shows the distribution of uploads for the various days of the week.

For day, it shows the distribution of uploads for various time periods of the day. 0000-0400 represents the 24-hour clock representation of Midnight - 4 A.M. UTC. The period intervals were limited to 6 to avoid having too many stats show up, plus 4 divides into 8 evenly which is a standard breakdown of daylight hours.

As a side note, I thought it would be pretty easy to implement approver statistics as well, so I'll be pushing that in some future release.

Pushed Version 12, which adds in an approver option. When that option is checked, it'll query that user's uploads for the various time periods instead of their uploads.

Note: The time periods for the posts are from when the post is created, not when it was approved.

Because of the above, the one thing it doesn't capture are the periods of time and the days when an approver is most active in approving, though I may add something in the future in order to capture and display that information.

I just decided to check this userscript out. It's pretty cool and pretty powerful in the summarization it can do for you.

I was wondering if it might be possible to expand the copyright list to include those posted within the longer time-frames beyond a day, or to make it customizable so you could add specific copyrights to get stats on for those longer windows. Would that be a reasonable feature to consider adding? Is it something already possible, but I'm missing the means of doing it?

Thanks for the tool!

Shinjidude said:

I was wondering if it might be possible to expand the copyright list to include those posted within the longer time-frames beyond a day, or to make it customizable so you could add specific copyrights to get stats on for those longer windows. Would that be a reasonable feature to consider adding? Is it something already possible, but I'm missing the means of doing it?

Yeah, something like that should be possible. Going for longer timeframes could be doable, but I'd like to avoid that as each additional copyright automatically can add anywhere up to 10 network requests, and I'd like to avoid hammering the server and/or significantly increasing the delay until the report completes.

What'll probably be easiest for the user would be to take the list of copyrights for a chosen period and present them to the user somewhat similar to how the related tags get presented. So currently shown copyrights would be highlighted, and clicking a tag would toggle it on or off. I'm not sure I'd want it to be immediately responsive though, so I'll probably make it so that the Submit button needs to be clicked in order to show the selected copyrights.

In addition to the above, I'll also add an override selector for when a user hasn't uploaded anything over the previous day, so that a report can still be generated.

Pushed Version 13 which adds both a copyright selection feature and an override feature.

The override can be used if a user has had no uploads/approvals over the previous 24 hours.

The copyright selection feature can be used to show more copyrights than just those from the previous 24 hours. The time periods available for this are weekly and monthly. Besides those two options, tags can also be manually added. When all of the copyrights have been changed as desired (either by turning them on or off), clicking the Refresh button will repopulate the table with the updated data.

Minor versions
  • (2019-01-03)
    • 13.1
      • Updated library version
        • ~20% reduction in code size
      • Refactored code to standard

Updated

Pushed Version 14 which adds a whole set of new features. The two biggest items though are that columns can now be sorted, and the statistics data can now be displayed in chart form.

New features
  • Sortable columns <Discord: nonamethanks>
    • Click the grey area in any of the period headers to sort by that period
    • There are four sorts for every period:
    • Sort by user or site postcount, both H -> L and L -> H
  • Statistics data viewable in chart form
    • Click the cyan area in the first row to show the chart for that metric
    • The data will be underlined in the table if the statistics data is available
    • To get the statistics data, click the name in the period header
  • Table headers are now statically placed
    • The headers now won't move when scrolling through an oversized table
    • Plus several adjustments to the table to make it fit better
  • Added the cache editor being prototyped (forum #154196, forum #154240, forum #154285)
    • No major changes externally, although internally there were a few
    • Also adjusted the settings menu to be more compact
  • Added settings which affect the maximum number of copyrights shown
    • The main benefit of doing this is that it takes less time to calculate the table
  • Added user autocomplete when IndexedAutocomplete is also installed
Usage help

Since there are a lot of clickable/hoverable items in the chart now, the following graphic was created as an explanation for what things do.

https://drive.google.com/open?id=1iYV0F_JoInaNDS_CYI_kwKUDvqE4euo5

Minor versions
  • (2019-02-08)
    • 14.1
      • Fixed missing specifier for approval data
      • Kept stale approval data from being pruned

Updated

Pushed Version 15 which primarily integrates the new library code that was finalized yesterday. This is coming out as a major version since a large majority of the code was changed while updating to the new library version and standardizing all of the affected userscripts.

Additionally, all of the menus were standardized, and fully working cache editors were incorporated. Also, a new control was added under cache settings which shows the size of all the userscript data amongst all of the various data storage.

  • (2019-05-12)
    • 15.1
      • Make use of "only" parameter
      • Update library version

Updated

Pushed Version 16 which primarily includes an update to the library code plus a few additions and fixes:

  • Additions
    • Add new menu library elements/functions for cache editor
      • The raw program data can be extracted using the cache editor, which facilitates transferring program data to another browser/domain
      • Controls which are not applicable are hidden, for example Local storage must be selected before the raw data option becomes visible
  • Changes
    • Added coloration of usernames and copyright tags
    • Settings menu now adapts to the color changes of the chosen theme (light/dark)
    • Current uploads user interface now adapts to the color changes of the chosen theme (light/dark)
  • Other
    • Multiple internal code changes and refactors
1