and I've heard 0 complaints and some people probably didn't even notice the deployment :D

ooh up to 26.6%

except that my server discovery code *did* have a bug in certain server configurations so failed to fetch media from those...
should be an easy fix today, and due to the drop-in nature it was incredibly easy to temporarily disable the media proxy

fixed and deployed :)

@forestjohnson yeah, I think pixie.town's users are in a rather diverse set of rooms, so media tends to get accessed only a few times

synapse's own media API is *only* used to fetch local content when it's no longer in the RAM cache (shown as 'synapse' in the top right gauges)

All other requests are handled by the proxy, which directly contacts federating servers (remote)

A graph would be good yeah, when I have some more time

https://stats.pixie.town/d/rPBvoh6Gk/synapse-media-proxy?orgId=1

@f0x ok I think get it, so there are two different kinds of cache-misses, the ones where it has to talk to your synapse (content that was posted on your server) aka "synapse" in your grafana dashboard.

Then there are the ones where it has to talk to other matrix servers (i'm assuming those are the ones labeled "remote" in your grafana dashboard).

So it looks like most of the time it's talking to remote servers, not yours -- "synapse" is only like 10% of requests. Isn't that a big win then? 90% bandwidth load reduction for the synapse server?

@f0x 90% load reduction sounds a lot cooler than 26.4% load reduction. Gotta look on the bright side lol 😀

@forestjohnson actually yeah, that's a big difference hah, I should add a gauge for that :)

@forestjohnson https://stats.pixie.town/d/rPBvoh6Gk/synapse-media-proxy?orgId=1&refresh=15m

now there's a large happy number (and a cache hit ratio that dropped even lower :') )

@f0x It might be illuminating to collect a metric for "cache misses caused by previous cache evictions". You would just have to store the cache keys for longer than the actual cache entries, or something.