Age | Commit message (Collapse) | Author | Files | Lines |
|
The new filter framework hasn't been extensively tested yet, but seems
to mostly work.
No real changes to the homepage, but I did manage to make a few queries
a little bit faster. The most significant source of slowness is the
releases listing, but I can't optimize that without adding indices.
|
|
|
|
There's a lot more unreferenced code in VNDB::DB::*, but I'll not spend
too much effort cleaning that up right now. All of it can be deleted
in bulk at some point.
|
|
Not used anymore.
|
|
Those stats havent been displayed on the site for ages, anyway. The API
will now report '0' for these stats.
|
|
I had already rambled on the current composite type solution in
583ae868dfd3c882a8d2dd40b5d5ed099170c1c2 and I had already explored a
few alternatives. This was the one alternative I hadn't yet explored
because I wasn't sure the operational complexity was going to be worth
it, but after seeing how bad PostgreSQL was at optimizing queries with
composite types, I figured I might as well just go with this approach.
It improves performance of some queries by a *lot* (especially the image
selection query) and it's pretty elegant and convenient to work with.
Only downside is the complexity of compiling, installing and maintaining
a vndbid.so library for PostgreSQL.
|
|
So that I can now work on adding a few more fields.
|
|
Ought to simplify the image flagging UI.
|
|
This is the first major editing form to be converted, so I'm expecting a
little breakage. A good chunk of this code has been copied from v3.
In terms of the UI there has been a small change: aliases that are still
referenced do not have the 'remove' link and instead have a flag that
shows that they are still referenced. This ought to be a bit friendlier
than throwing an error message after the user has submitted the form.
Some other things I'd like to improve in this form:
- BBCode preview
- Pasting in external links and letting the form figure out the Pixiv
ID, etc.
- Or perhaps even: Integrate AniDB/Wikidata search/autocompletion.
|
|
Only place where this isn't applied (yet?): Sorting user lists still
goes by the old username and board names don't use the new unicode
names.
I have to say, I quite like the sql_user() and user_() pattern. It's not
without problems when applied to everything, but it's good enough for
several use cases.
|
|
Most of this is copied from v3. I did improve on a few aspects:
- db_edit() and db_entry() use VNDB::Schema rather than dynamically
querying the DB. This has the minor advantage of a faster startup.
- The Elm code generator now writes to multiple files, this avoids
the namespace pollution seen in v3's Lib.Gen and makes the dependency
graph a bit more lean (i.e. faster incremental builds).
- The Elm code generator doesn't update the timestamp of files that
haven't been modified. This also speeds up incremental builds, the elm
compiler can now skip rebuilding unmodified files.
- The Elm API response generator code now uses plain functions rather
than code references and all possible responses are now defined in
Elm.pm. Turns out most API responses were used from more than a single
place, so it makes sense to have them centrally defined.
The doc page preview function is also much nicer; I'd like to apply this
to all BBCode textareas as well.
(Elm.pm itself is ugly as hell though. And we will prolly need some HTML
form generation functions in Elm to make that part less verbose)
|
|
I copied and modified VN3::DB to VNWeb::DB, but it isn't used yet. Now
that I think about it, that module isn't "web" specific at all and
should perhaps be in the VNDB:: namespace. Oh well.
|
|
+ Automatically fetch other links from Wikidata
|
|
This touches a bunch of things:
- Adds a new first-class database entry type
- Removes the d+.+.+ BBCode link syntax, adds a new d+#+ and d+#+.+
link syntax (references have been updated where possible)
- Adds a new dependency on Text::MultiMarkdown
|
|
Two main improvements:
- Filtering on (non)hidden items now doesn't join any of the item
tables, instead it looks up the latest revision from the changes table
itself, using the index on (type,itemid,rev). It's still not super
fast, but a pretty large improvement nonetheless.
- The item titles/names are obtained in a separate query. I tried to
modify the main query in various ways, but couldn't make it as fast as
I'd have liked.
I also removed the 'what' flag while I was at it, all uses of the method
request all information anyway.
|
|
This changes quite a bit to the way the editing functions work. Because
these functions are very repetitive and it's easy to keep things out of
sync, I created a script to generate them automatically. I had to rename
a few function and table names for consistency to make this work.
Since database entries don't have a 'latest' column anymore, and since
the order in which tables are updated doesn't have to be fixed, I
dropped many of the SQL triggers and replaced them with a
edit_committed() function which is called from edit_*_commit() and
checks for stuff to be done.
Don't forget to run 'make' before importing the update script.
|
|
This basically makes VNDB browsable again, but editing entries is still
broken.
I split off the get-old-revision functionality from the db*Get() methods
into db*GetRev(). This split makes sense even with the old SQL schema:
db*Get() had to special-case some joins/filters when fetching an older
revision, and none of the other filters would work in that case. This
split does cause some code duplication in that all db*GetRev() methods
look very much alike, and that the columns they fetch is almost
identical to the db*Get() methods. Not sure yet how to avoid the
duplication elegantly.
I didn't do a whole lot of query optimization yet (most issues require
extra indices, I'll investigate later which indices will make a big
difference), but I did fix some low hanging fruit whenever I encountered
something.
I don't think I've worsened anything, performance-wise.
|
|
|
|
|
|
Renamed 'changes' join to 'h' instead of 'c'
|
|
|
|
The Perl code and SQL-revisioning code only handles the name, original,
alias and desc fields at the moment. There is a basic /i+ and /i+.+ page
for testing, which should have all the functionality required for the
revisioning framework.
|
|
The code is a bit more complicated now, and it's not a lot faster, but
at least this helps a bit.
|
|
This is implemented by adding ihid (item hidden) and ilock (item
locked) columns to the changes table,
The (vn|release|producer).(hidden|locked) columns now work as a
cache, refering to the changes.(ihid|ilock) columns with
changes.id = (vn|release|producer).latest.
The cached columns are updated automatically each time a new revision is
inserted.
This is a pretty large change, bugs are quite likely.
|
|
|
|
Also added a little sanity checking on the edit_(vn|release) table,
and added a default value for releases_rev.released.
|
|
This will make it easier to do automated edits, either from cron jobs,
Multi, update scripts, or from within SQL triggers.
So far only the VN related functions have been defined/updated, trying
to edit/add releases or producers will not work at the moment.
The functions for editing or adding a new database entry have been
merged, as the procedure is rather similar.
util/dump.sql will be updated later on.
|
|
This column was used to differentiate between automated edits and user
edits, but that later changed to checking for changes.requester = 1.
The column has since never really been used, and due to a bug introduced
in VNDB 2.0, it has never been updated, either. Meaning it's not even
accurate for any database changes made after december 2008...
|
|
And also changed the way the item_table.latest column was updated: it is
now only updated after the revision insert has completed, making it
easier to write trigger functions in SQL.
|
|
|
|
This is a very important column in a very important table, I hope I
didn't forget to update a piece of code somewhere...
|
|
Performance improvement of ~15ms for all release and VN browse pages.
There are in total 20 known languages in the DB, and 12 of them are
actually used (i.e. a release in that language exists). Which means 8 of
the listed language filters won't produce any results (yet), but I'd say
that's an accaptable trade-off.
|
|
That was the last one. I hope I haven't forgotten to update anything.
|
|
The 'language' column in releases_rev has been replaced with a
releases_lang table. As this is quite a big change, there may still
be bugs floating around somewhere.
|
|
|
|
|
|
No idea why, but it looks like a mess :-(
|
|
DB/{Votes,VNList,WishList}.pm into ULists.pm
|
|
We're getting there...
|
|
Because I can't say no to a performance increase of 4 to 7ms
for -every- pageview!
Makes use of postgresql triggers and stored procedures.
|
|
go to Multi again
|
|
They pretty much all work the same anyway
|
|
History browser is now pretty much done
|
|
|
|
The query is starting to get pretty heavy, might want to cache a few
columns to get rid of all those table joins
|
|
Currently relies on the fact that Multi is the only one making the
automated edits
|
|
Doesn't work for r+ and v+ yet. Filter options pending...
|
|
...and those URL regexes are getting more and more complex >.>
|
|
|
|
...and I created yet another *::Misc module...
|