@me thanks :) If I may, unless you plan to have a lot of users or must store files on NFS or CIFS, just use SQLite and you'll get all your files in one place :)

I would be a big performance hit to store files in the database since everything is served directly from the zip files that Readeck creates.

@readeck@mastodon.online disclaimer: what follows below is my particular form of insanity and doesn't claim to be a rational software design choice. You are under no obligation to share it

Conceptually, I am very attracted to the idea that all the data I care about is neatly stored in one place, for example in a database, which just happened to be Postgres for me. Primarily because that makes backups trivial - I don't have to think about all the paths that need to be backed up or restored, just dump the whole database, and store it away. If I am looking at using Readeck (or any other software) in isolation, it doesn't apply, but when we are considering managing a collection of services, it becomes more of an inconvenience that each of them wants a different database.

Now, is it a bit deal? No, not really, SQLite would work just as well for me. And even if I have a dozen different apps each with its own SQLite database, it's still not too hard to write a script that backs them all up. I'll have to remember to update it every time I add something, but we aren't talking about thousands of them. The only victim is my sense of elegant

Maybe a slightly more rational argument could be related to splitting the state between the database and file system: making an consistent backup becomes harder without resorting to custom mechanisms. Admittedly, it would only be a serious consideration for large multi-user instances where state changes happen multiple times a second. On my personal instance I can just copy all the files at 3am, and the odds of inconsistency are really slim.

I'm not sure that serving from the file system is necessarily such a performance win, https://www.sqlite.org/fasterthanfs.html suggests that storing blobs in the database might be just as viable, but the balance may change with networked databases like Postgres.

@me I actually tested this by implementing sqlar and, even though it's good, it doesn't beat opening a file and copying data from some position to a writer.
And the sqlite driver Readeck uses doesn't provide the functions to read directly from a blob (sadly) and you end up with your 30Mb image in RAM before you can send it :)

My point was: with sqlite, your whole dataset is one folder and that's it. Plus, having a collection of files gives you the benefit of deduplication.

@me That said, I get your point and I actually agree. Only, if I have to choose between performances and backup convenience (on the hottest path), it's an easy choice :)

PS: the import / export commands are mostly there when you need to move from pg to sqlite or the other way around (and for portability). You can still use pg_dump, it'll work the same

@readeck@mastodon.online now you got me curious, how are you testing performance? Not storing 30mb images in RAM I can appreciate, after cramming readeck into a 2G vps with 10 other services on it already :)

@me for the specific case of zip vs sqlar, I made some extensive benchmarks. I was actually thinking of publishing a go-sqlar package yesterday, instead of having it rotting in a branch I'll never use :)