fuchsiaaaaaaaaaaaaaaaaa
ok lol my synapse-media-proxy project in progress is apparently the new code thing I brainstorm about in bed and under the shower
wonder if this structure will work nicely, a local component that runs on the Synapse machine with filesystem and database access, and a remote component that caches files and handles user requests
nice, I have basic file upload to the same memory cache as other served/proxied files working now :)
Remote media download just works (tm) too, very easy to get the remote homeserver base url properly with https://www.npmjs.com/package/@modular-matrix/autodiscover-client-configuration
now working on access token validation for upload, which will need some work on the <local> server bit too, accessing the synapse db for things.
But this standin works, much secure such wow :P
- proper database access token validation
- saving uploaded files to <local> server disk
and initial commit published!
https://git.pixie.town/f0x/synapse-media-proxy
added progress explanation so far to readme as well
did a bunch of code cleanup and refactoring, tomorrow hoping to get started on thumbnailing, and url previews after that, which would finish the spec compliance for the media repo :)
now to limit the in-memory cache on <remote> and check up the federation spec, and then /upload and /download are fully implemented
oh yeah and also verifying the symbols in the requested mxc for 'security reasons' lol
I love how the server-server spec for the media repo is "yeah just call their client-server endpoint" which is also how i implemented it already
ah fuck i reallly stumbled into a bad wormhole doing punycode domain testing, seems neither Synapse, Conduit nor matrix-media-repo can fetch media from those properly (and I can :3)
when the punycode is in the url literally like mxc://xn--puny--59d2hgc.dev.cthu.lu/testmedia, Synapse still fails, but Conduit and matrix-media-repo do fetch it correctly.
None of them do url decoding of the mxc right however..
time to write a nice memory cache
nevermind i can't be bothered anymore, I'll just make it remove the oldest entry...
the most basic of caches, it's a Map with an array tracking access order, removing the oldest accessed item from the map when it's about to get bigger than maxEntries
hmm debug() logging says trans rights? and the Validator says you're valid too!
:O I think I implemented the whole https://spec.matrix.org/unstable/server-server-api/#server-discovery Matrix server discovery flow!!! It's rather complex, with .well-knowns and SRV records and combinations of those.
Also nice that I could fork of the client-spec counterpart already made by someone else :) https://www.npmjs.com/package/@modular-matrix/autodiscover-client-configuration
now just have to wait for some responses from a few homeservers asking if I could add their servernames to my example.js, to show off the different flows and the method response you get
swapping out client discovery for server discovery in synapse-media-proxy makes it so I actually follow spec correctly there :)
some basic memory usage reporting, but memory management is an enigma so can't really see immediate free-ing when removing stuff from cache etc
submitted 2 things to TWIM https://matrix.org/blog/category/this-week-in-matrix for the first time in aaaages :3
soo good to just come across a library that does what you need to *perfectly*. I was messing about with regexes to parse Content-Disposition stuff, and with this library I can do both the parsing and formatting sooo much nicer (and it's used by express.js so it's Good(tm))
https://www.npmjs.com/package/content-disposition
I have a nice ttl invalidating cache for server lookups, and the content-disposition lib is fully integrated
I also used vscode's incredible git integration to split those two changes into 2 commits after I had written both, with the suuper good visual cherry-picking of lines to commit
nice nice nice file uploads work very well now, properly stored where Synapse would normally expect them too, and a listing in the database