summaryrefslogtreecommitdiff
path: root/.gitignore
AgeCommit message (Collapse)AuthorFilesLines
8 hours!BREAKING! Restructure directory layout for generated & runtime filesYorhel1-28/+4
My testing, deployment and backup scripts were getting more complicated with files from various stages being lumped into a single directory structure, so all generated files (= anything touched by 'make') and runtime files (= anything touched by the web backend or multi) have now been moved into separate directories. These directories are also configurable with $VNDB_GEN and $VNDB_VAR, making it possible to manage multiple instances from a single source checkout. I also got rid of *data/* and *dl/* while I was at it, and moved *static/st/* (that is, the screenshot thumbnails) to */sf.t/*, to be consistent with the newer and more flexible image directory naming scheme. This commit breaks all existing installations and upgrading requires manual action. General upgrade instructions: # BEFORE doing a checkout of this commit make clean # AFTER make mkdir -p var/static var/log mv static/st var/static/sf.t mv static/{sf,ch,cv}{,.orig} var/static/ mv data/conf.pl var/ mv data/log var/ mv data/hibp var/ mv dl var/ util/setup-var.sh Use `git status` find leftover files to clean up or move. Don't forget to update conf.pl and web server configuration to make sure they access the new paths.
2024-03-10Improve processing of uploaded imagesYorhel1-0/+5
My goals, in particular, were to: - Make use of a more efficient jpeg encoder; jpegli in this case. - Sandbox the image handling process to protect against codec vulnerabilities; implemented with seccomp. This isn't easy to do with software packaged in typical Linux repos, so I had to improvise a bit. I switched to libvips as that is easier to sandbox than ImageMagick and provided a Makefile to fetch and build a custom libvips that uses jpegli. None of this is very portable, so to simplify testing setups a simpler "imgproc-portable" binary is built instead. Also experimented with different resizing and encoding options, but the defaults were mostly okay. See https://vndb.org/t21692 (This breaks the Docker setup, I'll fix that later. I'll also see if I can drop the ImageMagick dependency that's still used for pngsprite.pl, as it seems a little redundant)
2023-08-06Save original image uploadsYorhel1-0/+3
So that they can be recompressed or resized to new formats, in case we ever add support for alternative sizes or formats.
2023-04-25Merge platform icons into an SVG spriteYorhel1-1/+0
Couldn't get fragment identifiers to work satisfyingly, so ended up going back to old-fashioned CSS background-position. Oh well.
2023-04-08JS: Reorganization to prepare for an Elm->Preact migrationYorhel1-0/+1
This change also increases browser version requirements by quite a bit, though it should still be fair. Pale Moon is an easy "old enough" target to test. This new organization is, in a way, pretty similar to what we had before the switch to Elm, except with more experience I might be able to do this in a more manageable way. I have to admit, I like arrow functions more than I had expected.
2023-03-04Add scripts to create and update the dl/ directoryYorhel1-0/+3
These aren't new, but for some reason I thought they weren't interesting enough to keep in git.
2022-12-03Switch to the HIBP password databaseYorhel1-1/+1
Currently has 851 million password hashes, taking about 8G of space with the current approach. It's simple and fast, so should be worth it. inb4 complains about "why can't I use my password anymore!?"
2021-01-17Use SASS for CSS + static asset reoganizationYorhel1-16/+1
Had been planning to use a more powerful preprocessor for CSS for a while, so that I can also reorganize and clean up the CSS a bit. The cleanup will come later, this is the first step to reorganize the build system a bit and remove skingen.pl. I moved all generated static assets to static/g/ (for _g_enerated), including icons.png and js files. This simplifies management of static/f/ and static/s/, which are fully in git. Skins are now defined as sass files in css/skins/ with their images in static/s/ using plain directory structure.
2021-01-15Docker: Upgrade to Alpine 3.13Yorhel1-0/+1
I was waiting for a new stable release so we can get Postgres 13, which I've already been using in my own dev & prod environments. I don't think Postgres 13 specific features are being used yet, but that will no doubt change. Image::Magick removed as it isn't used anymore. The image is somewhat more bloated, though, as postgresql-dev pulls in llvm+clang now.
2020-09-17Feeds: Move feed generation form Multi to VNWebYorhel1-1/+0
This means the feeds are now generated on demand rather than every 15 minutes. Main reason for originally implementing this into Multi was because RSS feeds tended to get requested a *lot* and I didn't want those requests to impact site performance, but now that RSS is almost dead it doesn't really matter that much anymore. A caching layer can still be added anyway. Be sure to restart Multi and delete www/feeds/ after this change.
2020-05-29JS: Split v2rw.js into plain.js and elm.js, only include JS files when necessaryYorhel1-3/+6
The trace_log showed that the vast majority of pageviews do not need any Elm modules at all, so we can save a lot by not loading them on the page. Some pages (most notably: VN pages for not-logged-in visitors) do need JS but don't load any Elm modules. I've added a 'js' flag to HTML::framework_() to indicate that and ensure that plain.js is still loaded.
2020-04-05SQL: Add C-based "vndbid" type and use it for image IDsYorhel1-0/+3
I had already rambled on the current composite type solution in 583ae868dfd3c882a8d2dd40b5d5ed099170c1c2 and I had already explored a few alternatives. This was the one alternative I hadn't yet explored because I wasn't sure the operational complexity was going to be worth it, but after seeing how bad PostgreSQL was at optimizing queries with composite types, I figured I might as well just go with this approach. It improves performance of some queries by a *lot* (especially the image selection query) and it's pretty elegant and convenient to work with. Only downside is the complexity of compiling, installing and maintaining a vndbid.so library for PostgreSQL.
2020-04-04SQL reorg: Move util/sql/ to sql/ as a top-level dirYorhel1-10/+1
Been wanting to do this for a while... I've kept util/sql as a symlink for compatibility with the devdump, old update scripts and other code I may have forgotten. I'll remove it later.
2019-09-25v2rw: Add Elm & db_edit framework + Convert doc page editingYorhel1-0/+5
Most of this is copied from v3. I did improve on a few aspects: - db_edit() and db_entry() use VNDB::Schema rather than dynamically querying the DB. This has the minor advantage of a faster startup. - The Elm code generator now writes to multiple files, this avoids the namespace pollution seen in v3's Lib.Gen and makes the dependency graph a bit more lean (i.e. faster incremental builds). - The Elm code generator doesn't update the timestamp of files that haven't been modified. This also speeds up incremental builds, the elm compiler can now skip rebuilding unmodified files. - The Elm API response generator code now uses plain functions rather than code references and all possible responses are now defined in Elm.pm. Turns out most API responses were used from more than a single place, so it makes sense to have them centrally defined. The doc page preview function is also much nicer; I'd like to apply this to all BBCode textareas as well. (Elm.pm itself is ugly as hell though. And we will prolly need some HTML form generation functions in Elm to make that part less verbose)
2019-09-12Add VNDB::Config module and unified conf.pl config fileYorhel1-2/+1
This gets rid of global.pl, config.pl and config3.pl and uses the cleaner config3.pl format for the config file. The config is easily accessible from anywhere by importing the new VNDB::Config module; The global $VNDB::S,O,M,ROOT variables have been removed. Sorry for all the churn...
2019-09-09spritegen.pl: Move compression to "make prod"Yorhel1-0/+1
I think this is the last one. 'make' in a development environment is now beautifully fast and 'make prod' generates nicely small assets. (arguably we could have an even faster dev setup by not generating an icons.png in the first place, but then we'd need more code to differentiate between dev & prod, which is also a pain) This change does remove the "slow" option that would use the compressed image size in the optimization algorithm, but I hadn't used that option for a while anyway, it takes an hour and only saves about 100 bytes.
2019-09-09JS/Build: Move minification and compression to "make prod"Yorhel1-1/+2
Similar to previous commit.
2019-09-09CSS/Build: Move minification and compression to "make prod"Yorhel1-1/+2
This rather significantly speeds up development builds. Also simplifies skingen.pl and its config a bit. The new compressed files are written to /s/*/style.min.css{,.gz}, it is up to the web server to serve those instead of /s/*/style.css.
2019-07-25Merge the v3 branch into separate namespace + fix Docker stuff (again)Yorhel1-0/+11
I was getting tired of having to keep two branches up-to-date with the latest developments, so decided to throw v3 into the same branch - just different files (...which will get mostly rewritten again soon). The two versions aren't very different in terms of dependencies, build system and support code, so they can now properly share files. Added a section to the README to avoid confusion. This merge also makes it easier to quickly switch between the different versions, which is handy for development. It's even possible to run both at the same time, but my scripts use the same port so that needs a workaround. And it's amazing how often I break the Docker scripts.
2019-07-17Docker: Store Postgres DB in data/ + simplify the setup a bitYorhel1-1/+1
I initially wanted to move the static/.. files onto the Docker volume, so that all dynamic site data is stored in a single place. But that turned out to be impossible to do without some really ugly hacks. So instead I went with the opposite approach: get rid of the 'vndb-data' volume and instead store everything in the source directory. This also requires running PostgreSQL as the 'devuser', but that's fine for a development setup. All of this makes it more obvious what is going on and simplifies the init script.
2019-05-16Add password dictionary checkYorhel1-2/+3
This affects the following: - API login with a weak password is disallowed, affected users will have to change their password through the website to continue using the API. - Registration, password reset or password change forms require the new password to not be in the dictionary. - Attempting to log in to the website with a weak password will force-redirect to a password change form, allowing a new password to be set (using the weak-but-still-valid password as check).
2018-01-05Various fixes to the release icons featureYorhel1-0/+2
It's been a while since I had static/f/ in git, so I had to adjust .gitignore a bit. The CSS changes are purely opinion, but it does integrate better with the existing layout. Everything else are bug fixes.
2016-01-10Use atomic replace when writing .gz assetsYorhel1-0/+1
2015-10-21SQL: Fix editing + func.sql + triggers.sql + autocreate editing funcsYorhel1-0/+1
This changes quite a bit to the way the editing functions work. Because these functions are very repetitive and it's easy to keep things out of sync, I created a script to generate them automatically. I had to rename a few function and table names for consistency to make this work. Since database entries don't have a 'latest' column anymore, and since the order in which tables are updated doesn't have to be fixed, I dropped many of the SQL triggers and replaced them with a edit_committed() function which is called from edit_*_commit() and checks for stuff to be done. Don't forget to run 'make' before importing the update script.
2015-05-11Generate icons.png and associated CSS automaticallyYorhel1-4/+6
It became a bit of a hassle to keep updating that file manually in Gimp. This script performs surprisingly well for our set of icons.
2013-01-04Added daily-generated JSON dump of the tag informationYorhel1-0/+1
This isn't documented yet.
2011-02-16chardb: Added image to character entriesYorhel1-0/+1
2010-11-25JS: Split script.js into a separate file for each languageYorhel1-1/+1
This greatly reduces the size of the Javascript file. The compressed size has been reduced with about 9kB, and is now a total of 14kB for en.js. A nice property of this is that more translations can be added without increasing the JS size. While I was at it, I made jsgen.pl also replace mt() function calls in cases where an exact TL string was requested without any additional arguments and/or formatting codes. This helped reduce the compressed size by about 1kB. My aim is to keep *all* the JS code of VNDB smaller than the jQuery core library, as a general "fuck you" towards users of large and bloated JS libraries. We must keep the VNDB page loading times lower than that of other sites, after all!
2010-11-13Multi::Feed: Added Atom feedsYorhel1-0/+1
TODO: add links to these feeds from the site
2009-11-15docs: Deleted d8 from the git repoYorhel1-0/+1
This page is pretty specific to the main VNDB, and each VNDB would probably have it's own d8.
2009-11-03gitignore: Added data/multi.pidYorhel1-0/+1
2009-10-07JS: Moved script.js to data/ and added jsgen.pl to process itYorhel1-0/+1
While this 'processing' is currently limited to minifying the file if JavaScript::Minifier::XS is available, this change would make it a lot easier to make the strings in the JS code translatable.
2009-09-25Use inline SVG for relation graphsYorhel1-1/+0
The graphs are now stored in the DB in SVG format, the static/rg/ directory can be removed (not used anymore). SVG data is stored using the xml data type, so now I can say for sure you'd need at least PostgreSQL 8.3. This feature still needs some tweaking, though. Current state isn't perfect.
2008-12-21Basic skin generator + example skin for testingYorhel1-0/+2
How it works: Create new directory in static/s/ Create a 'conf' file (see the test skin for a template) Run skingen.pl, which will generate a style.css and boxbg.png This process will probably be automated using a simple web interface or something... There's no skin selector yet, so Util/LayoutHTML.pm has to be modified to view the generated skin.
2008-10-26Removing all files we're not going to use with the rewriteYorhel1-1/+0
...this is basically everything we're going to rewrite
2008-10-26Adding .gitignoreYorhel1-0/+9