i can't go to hell - i'm all out of vacation days. i watch space rocks and yell at computers for my day job. probably too old for any of this

 

i think i might be burned out on internet social. it's hard to keep doing it. it's hard to even maintain the amount of attention i'm already giving it

 

i am the cause of most of my own problems

 

furthermore, capitalism must be destroyed

 

birdsona: ?????

 

🌎 Ontario, Canada


webbed site
egrets.ca/

Some thoughts on preservation and ephemerality and impermanence in light of that article from this week where about 87% of classic games were found to be unavailable or at risk of going missing

And there's been a lot of thoughts about that, and I think, a lot of thinking about a lot of different things in the same light as these video games - photographs, personal documents, journals, posts, tweets

I'm not sure that we can equate these things? I'm not sure it's possible to equate these things?

And ultimately, i'm not totally convinced that we can, or even should preserve all of this.

We make too much stuff

So, before anything else, I think we need to talk about the sheer scale of stuff we produce on a day to day basis. i can't actually claim to have any idea how much of it there is - I can't give you an exact number, or even a truly informed estimate. But it's a lot!

I'm not willing to turn this into a major research project, but as just a small slice, let's ballpark some twitter numbers - with minimal cursory searching, we can make some very rough estimates here

I'll put this behind a cut because it's a lot of ultimately unimportant numbers based hand-waving
  • as of 2023, there are about 200 billion tweets per year (demandsage)
  • in 2010, there were about 70M tweets per day (approx 25.5 billion per year) (twitter by the numbers)
  • in 2010, twitter generated about 12GB of data just for tweet text alone per day (twitter by the numbers)
  • but from the same presentation, in 2010, twitter generated about 8TB/day of data in total (twitter by the numbers)

The reason there's so much more total data than tweet text alone is likely because supplementary data from database structure, indexing, attached media, metadata, metrics & profiling, etc. add up to far, far more storage cost than the actual text content itself.

So I'm going to do some back of the napkin math here, assume that things have scaled fairly linearly, that storage requirement per tweet has not changed that much (almost certainly not true lol) to estimate that in 2023, twitter's numbers have scaled something like:

8TB/day / 25.5B tweets/day * 200B tweets/day
= 62.7 TB/day
= 22902 TB/year
(or 22.9 petabytes)

Now this is just one copy of the data at rest, in cold storage. Twitter almost certainly keeps at least two at-rest copies for disaster recovery, and at-rest copies are rarely ever touched by API systems, but rather moved into several different levels of caches (nearline, hot, etc.) at different levels of locality (regional, datacentre-local, API server local), ultimately meaning that there are a fuckton more copies than just the at-rest data. But the at-rest size is probably all we're really concerned here if all we want to do is be able to save a copy.

The takeaway from all of the above is that as a rough back of the napkin estimate, twitter is shoving at least three of the highest capacity drives commercially available hard drives into their storage backend every single day, per replicated copy, just to keep up with all the data they're generating. And that number seems to be accelerating roughly at the same pace storage is growing, so we are not gaining ground.

And twitter isn't the only site out there. Facebook generates a colossal amount of data. Instagram generates a colossal amount of data. Tumblr generates a colossal amount of data. cloud storage generates a colossal amount of data.

The world over is peppered with facilities up to an entire square kilometre in size, that are nothing but enormous air conditioned boxes for storing racks and racks of high density boxes full of hard drives to sustain all of this.

So how can we save it all?

Short answer - i don't think we can.

Somewhere between 1% - 1.5% of those drives are failing every year and need someone to actively replace them.

These facilities are a deeply interconnected system, any of the parts of it can fail. All of the parts are constantly failing. Someone always needs to be fixing something. Power distribution, climate control, backup batteries, backup generators, hardware, software.

If you leave any of it alone for probably on the order of a week, something will go wrong.

Sure, we can keep it all running as long as it's in our interests to do so. you can define interests however you wish - monetary, academic, cultural - in the end, it takes constant and real material resources and human labour to keep this entire house of cards standing. In the end, something will happen to bring it to an end.

Twitter or whichever website will close up shop. We know this will happen, we've seen it hundreds of times before. And when it goes down, there's nobody left to maintain these data. Nobody to swap in replacement drives and replace old batteries or maintain the generators or fix the air conditioners. There's nobody to pay for the datacentre floorspace, or if you're (questionably?) lucky enough to own your own infra, pay for the expenses of running it - the things that again, come with real, unavoidable material and labour costs. The staff, the parts, the electricity.

And this is just big megasites on the internet, that's saying nothing of small personal sites or your files at home.

Small personal sites and your files at home

Oh my god, this situation is even more dire.

How many redundant hard drives do you have in your computer at home? You can survive a catastrophic storage failure without having to rebuild and reinstall your computer, right?
OH DANG

How many of your photos and videos only exist on your phone? A device which you probably replace wholesale every few years because they are, by design, not servicable or maintainable?
OH SHIT

Okay, okay, but you have backups right? Oh, it's on a portable external disk? When was the last time you tested them? Wait, is that disk older than ~5 years? Does it even work?
OH NO...

Wait, wait, let's not panic - you have offsite backups, right? Maybe with a major online cloud storage provider in a datacentre much like the ones we just described above?
OKAY, GOOD

But you're in the vast minority now. And how confident are you that your online backup provider is going to outlive you? Will they be around in 20 years?
What happens to your archived files when you die and stop paying the bills?

In the last 20-30 years, we have already seen so many websites rot and decay away. Most of them are just gone. The ones that survived are some of the simplest sites that live in on some corner of an academic webserver somewhere, and have probably been replicated across so many generations of replacement servers simply because they are simple files with virtually no requirements other than a webserver, and too small to notice/care about, so they get carried along with each successive backup & migration.

We preserve the things we care about

I feel like this copost, "on preservation" by @thecommabandit is relevant at this point.

It goes into a lot, but I think the single most important takeaway from it, at least for me, is the single sentence, "societies only expend effort to preserve things they thought were important,"* or the shorter version of it that has stuck in my mind, "we preserve the things we care about"

* with the important caveat that often, it is the wealthy or powerful that decide which things are important on a societal scale.

Soft, organic artifacts like clothes, wooden tools, etc. decay. Even stone left exposed to the elements will weather and erode away over time. And every single method of archiving all of this digital stuff we're making is even worse than any of these.

Hard drives? Probably good for 5-10 years at most. Some last longer, but you're really gambling.

SSDs? Absolutely unusable. If not actively powered and used, the charge stored in every tiny NAND cell dissipates over time. They are guaranteed to forget every single thing you have ever stored on them.

Writable optical media. Wouldn't count on it for more than a decade.

Professionally mastered optical media. You've got maybe 20-30 years realistically. Some of my commercial discs from the 1990s are already starting to show signs of bitrot.

Things look a little better when you start considering purpose-built archival media, stored in a perfectly environmentally controlled vault, not exposed to humidity or light. The numbers seem to suggest that we can probably get entire decades of reliability out of rigorously environmetally controlled optical media - up to half of them might even survive a few hundred years!

Ephemerality

So, it's all hopeless then? All of this will be lost? All your photos, all your posts? All these recordings and shows and media?

Yeah, probably, a lot of it will. I don't think that's a bad thing.

Look, we make so much stuff. The amount of stuff we're making is growing at probably an exponential rate. Our ability to store it is getting shorter and shorter. The numbers don't work out. But I don't think it's a contentious position to take that we know infinite growth, forever, is unsustainable. Something has to give.

And I don't necessarily think that there's as much value as we might think in keeping around every thought, every photo, every experience we have ever had, in perfect, lossless clarity, forever.

We forget things over time, we age and long distant memories get fuzzy. This is natural. We hold onto the ones most important to us - we preserve the things we care about.

We change over time, thoughts an opinions we had ten years ago probably no longer represent us as we mature and develop new understanding. The parts of our old selves that we built our present day selves on, we keep around - we preserve the things we care about.

Perhaps we need to think of our online corpus in the same way. The art we truly value, that we truly want to hand down to whatever cultures come after ours - the things we personally care about, we have to selectively preserve in a way that will last after our hard drives rust away and optical discs rot.

We have to selectively choose what art, texts, thoughts, artifacts we care about and preserve them, because we can't save it all, and the best we can do is to save the things we care about.

And maybe we have to accept that even some of the most powerful figures of the past couldn't preserve their legacy against the ravages of time, that most things will be lost, that ultimately it's out of our control. All we can do is hand down the few things we care about most to the next generation, and the culture after them, and the culture after them, and trust that they will preserve the things they care about.

We preserve the things we care about (redux)

So what about those 87% of games? They're just gone? Should we care?

I think so, yeah. Even more so than a single photograph or personal site, these things are cultural artifacts. And being fairly early titles, a lof of them represent early works in a nascent form of media, that tell the story of its origins and how it came to be what we know today.

They may not all be an equal influence, and they may not all have equal cultural importance, but as a collection, they do represent the work of a culture, and they're such recent works, that it feels maybe a little disappointing that so many of them have been left to rot already.

I know it's hard when the death cult of capitalism wants to forget everything so you can be sold the next new thing, or so the past can be remastered and resold to you again. Or when they demand that everything worth doing produces a profit. And when oppressive intellectual property regimes interfere with anything that isn't their own perpetual profit machine.

I know that in the face of our modern dystopia, we don't always have the ability or agency to decide these things, and that on balance, the impact one person can have is often very limited - we can't save the world on our own.

But maybe, if these things are important to us, then we collectively, as a culture, have to find a way to preserve what we care about? In spite of it all?


You must log in to comment.

in reply to @SomeEgrets's post:

I like these thoughts. These are good thoughts.

People who are interested in lost media sometimes talk about this in ways that are strange to me. They'll sometimes mourn for media they personally never saw, or things that were never even published. So it's often not even about the virtues of the particular pieces of media, it seems to be about an ideology where nothing we create ought to ever be erased.

Archives are great, and efforts to preserve media, records, and artifacts of culture are so valuable. I'm also thinking about my grandmother's daily journals she kept for about 50 years, which are an incredible family treasure. But it's just not feasible, or even worth the effort and resources, to save every human creation and record every event indefinitely.

I think it's also worth mentioning that artists choosing to not publish a work or remove copies of their work online is something they should have every right to do.

This is a topic I've thought and cared intensely about for some time. I think you're right overall, but one thing that I think is fundamentally important to examine critically is where the stuff we make comes from.

Most of it? Spam. Advertising. Things to make you click the link to make a few cents on an ad. More ads.

Most of it is not the result of a personal creative endeavor - rather, at best the creative endeavor of a purely capitalistic pursuit. Once we narrow the scope to exclude that class of stuff, we are left with a much more manageable amount. I'd venture a guess that the growth rate of this class of stuff is much closer to flat. We could, if we cared, save the things which are important to people - at least, we could for as long as they cared, and then some.

hm yeah, and i think the section on capitalism being actively antagonistic to preservation proooobably deserves more than a short paragraph at the end too

and i think maybe it's missing the concept of lifespan - that preserving something while it's still active, fresh, new, in the zeitgeist, relevant to someone, etc. is easier and much lower cost than it is as it starts to age? and how as things drift towards the latter, it takes effort, resources, and intentionality, and it becomes an active choice to do so

(and specifically how these two things tie into the tragedy of foundational works being lost when they're still so new, on balance)

felt like it was kinda running long already though. maybe i'll add to it sometime

Pinned Tags