- DB size is under 50gb still (we had to overprovision on DB b/c of cpu issues that have since been resolved but we can't scale back down b/c of how digitalocean links this stuff)
- the render server is really CPU bound so unfortunately we have to replicate that part up higher than we'd want (it can currently only reliably handle 1rps per-replica; this is an area of active research)
- the API server, on the other hand, is exclusively DB bound and can do 25rps per replica no problem, although we have our scaling target set lower than that for safety.
- we're running on a kubernetes cluster of digitalocean
s-8vcpu-16gbnodes. that node size has been a good sweet spot for us thus far. - total image storage is about 500gb
ok. now for big numbers, all of these are over the last month.
- we're transferring an average of 13gb per hour for uploaded images with a cache hit rate of 51% (we wanna improve this second number). fastly also gives us "cache coverage", which is the percentage of requests that are theoretically cacheable. our cache coverage is 99.6%. i don't know what that extra 0.4% consists of but it's going to haunt me.
- the main app is 7.9gb per hour with a cache hit rate of 95.30% (static assets like the client-side js have really aggressive cache rules for this exact purpose). our cache coverage here is only 13.63% since dynamic pages (aka Everything That Isn't Javascript) isn't really cacheable.
that said, over the last few days we have seen a fairly substantial increase in overall traffic. our hourly average for images prior to that was 10gb; if we count just since then it's up to 33gb per hour. we see a similar increase with main app traffic.
big weekend for us



