Vosyl

Black-Tailed Jackrabbit

Known Obscurant ▼ Anti-Social ▲ No Label
.
Psychology & Criminology Student.
.
A Trans Woman in her early thirties. I write,
draw, and even play music. An avid comicbook nerd,
a chess geek, and indie ttrpg enjoyer.
.
I'm also a part-time supervillain.
.
∍⧽⧼∊
ϴ⨺



gretchenleigh
@gretchenleigh

Vanishingly few people on Earth are as close to web archival as I am. How many people are lucky enough to call it their full-time job? And I have a small bit of depressing news: Yes, the web is getting worse.

Websites that used to be open are now requiring registration to just view content and forcing users onto proprietary apps. APIs that once encouraged creative usage of data are tougher to access and offer less access. Restrictions on crawling via robots.txt and heavier-handed tactics such as Cloudflare are making it harder to preserve crucial content. Ads are getting more intrusive and frequent.

Everything is being turned into a fine paste of hallucinated LLM slurry. The web is failing at the thing it used to be best at: Directing you to the information you seek as provided by a human who knows about the topic and has taken care to write about it accurately.

It's made it a huge challenge to archive the web, but it's also, obviously, made it a lot harder to find true community in the way that we used to. It's really tough for a distinctly human website like Cohost to thrive in a content slop world. So much money has flowed into the worst possible version of Tech, and building sustainable community-driven projects is really challenging in that environment.

I mourn the loss of Cohost and the web of the past. I don't think we're ever going to go back to what things were like before the money and reactionary rich guys ruined everything, but I'm hoping that we can continue to hold on to each other, build nice things for ourselves, and ensure they're preserved for the future.


You must log in to comment.

in reply to @gretchenleigh's post:

i wonder how much of the robots.txt restrictions is attempts to block scraping by people building datasets for LLMs. maybe sites are being overly restrictive with crawlers in response, but also a lot of LLM scrapers don't seem to be particularly interested in playing by the rules so that situation is definitely not helping here

i ended up setting my forum to no robots, partially because of what you mention; i don't want the history of that site to just turn into fuel for some llm garbage. but also, at what point do we let things go? how much should someone's posts from 17 years ago be visible to someone casually searching the internet?

i don't think i can make that decision, but i trust google and openai to do it even less; not that it stops those who do it anyway. my forum has gotten a consistent crawler from hauwei for the last several months. what can one even do about it other than give up?

the search engines don't have to care any more. why should they? google and amazon own everything. you search on google, you get google ads. you go to another page, even if it isn't what you actually wanted, google's ads are there too. so you go back and look at another result, maybe it'll be better. nope, slop too. what's the incentive to get it right? nobody else is.

I mourn the loss of Cohost and the web of the past. I don't think we're ever going to go back to what things were like before the money and reactionary rich guys ruined everything, but I'm hoping that we can continue to hold on to each other, build nice things for ourselves, and ensure they're preserved for the future.

in my experience, having done this sort of thing for 20 years now... it's a losing battle. there are little pockets here and there, but all of it is against the tide. the "small internet" is life trying to survive in a hostile environment, and every year it gets more hostile. the big services try to keep users on their platforms through any means they can; smaller platforms and places are getting pummeled by bad actors (llm shit, bots, spam) constantly.

it's... it's like. you can either give users the easy thing that provides no real substance or nutrition, but tastes great and is readily available. or they have to do actual work themselves, actual effort. and even though it's worth it, when everything is constantly grinding down your will to keep doing that, when everything is smoothed to the level of "just scroll for more, forever, never leave"... how do you combat that?

basically the same conclusions but a lot of words. blame weed

some of it's the divide and conquer strategy. most sites, platforms either focus entirely on yourself (YOUR followers, YOUR feed) or make it trivial to make your own space (discord servers, channels).

and you can't have a community in a world where nobody shares the same space (the former) or where groups become so fragmented they die (the latter)

the massive inter-connection has left us as a huge ocean of people, all together yet, ironically, lonely.