jkap

jae

@jkap

CEO of posting

butch jewish dyke
part of @staff, cohost user #1
married to @kadybat

This user can say it
osu stats

log inask
🔵 tumblr
yrfriendjkap.tumblr.com/
🐘 mastodon
xoxo.zone/@jkap
🖼️ icon credit
twitter.com/osmoru
🐦 twitter
not anymore lol
🎬 letterboxd
letterboxd.com/yrfriendjkap/
jkap
@jkap
staff
@staff
staff
@staff

(finally) known bug: occasional UNAUTHORIZED when loading pages

hey folks! you may have seen a bug occasionally where, when loading a page, you would see an UNAUTHORIZED error page. we hadn't figured out what was causing this yet, but we were occasionally seeing it around deploys and large periods of site load.

good news: we finally figured out what it is! the current system we use for our feature flags has an oversensitive rate-limit system, which means that sometimes newly starting instances will fail to get the list of flags. when this happens, if a user tries to reach something behind a feature flag, they'll get a (until now, very unhelpful) error message.

we already have a fix written that will deployed on monday morning. (it's a jae change and it's already 4:30pm their time). the fix is twofold:

  • migrate our feature flags to a different provider that doesn't have the rate limit issue
  • remove all now-unnecessary feature flags1

until then, if you see this error, waiting a bit before refreshing should clear it up.

thanks for using cohost, and have a good weekend! :eggbug:

  1. like trees, software has rings. an example of this is that access to notifications is still gated behind a feature flag. we shipped notifications in march. whoops.

You must log in to comment.

in reply to @staff's post:

vogon
@vogon

incidentally, since I realized after jae posted this that the concept of "feature flags" might be new to some folks: the way you'd traditionally turn on or off functionality in a piece of software is to deploy a new version of the software that had the code added or removed. for certain reasons you'd want to do deploys and certain software development organizations, this is impractically slow, so a pretty common pattern is to invert the design, so the software hits an external service to tell it whether it should provide the functionality or not.

this also enables you to do fancy modern web development stuff like gradual rollouts (the thing where twitter gives 1% of people a feature before everyone else), A/B testing (giving some people one version of a feature, and other people another version of it), etc.

login to reply
georges
@georges

My last job had warnings for feature flags that stayed on too long (with a recommendation to turn them into kill switched, which are features flags, but upside down so that they're off by default). I've never seen the feature flag service go down myself, but it's interesting to see the reason for that in the wild.

login to reply
Pinned Tags