Staring into the Void

First, let’s get one myth out of the way: The Internet Archive has not been up, rock-steady and with no loss of service or connection, for twenty-eight years.

Starting out as a project to archive online materials, with a lot of speculative ideas of how to handle data at scale, the archive.org website was hosted at a shifting set of locations across its early years. It ran at razor-thin margins while rubbing hardware and software elbows with all sorts of then-famous sites; it directed its staff towards nebulous and aspirational goals while trying not to burn through its resources.

Stand back, we’re not sure how big this Archive is going to get.

A lot changed in October of 2001, when the Wayback Machine was introduced to the world at a ceremony at the Bancroft Library in Berkeley, and the Web spontaneously developed something it hadn’t really had before: a memory.

That Memory went from a feature to a core utility for the internet.

Collections such as the Prelinger Library and the Live Music Archive were also coming along for the ride, providing a way for people to just get to the good stuff and not face down web banners and pop-up ads just to listen and watch culture from a growing set of sources and reaching back farther in time, to before the web itself.

Serving a massively-enlarging set of data to a massively-increasing audience became an engineering and cost problem, and ultimately the problem – how do you retrieve and provide terabytes, then hundreds of terabytes, then petabytes, then dozens of petabytes of data to your patrons without, again, falling to a thousand potential problems?

Photo by Ben Margot of Associated Press, 2006.

The short answer is that you work very hard with a very dedicated crew with a shared vision, but the longer answer is that sometimes, issues arise.

Many issues.

Network equipment crashes, power strip failures, unexpected configurations and firmware upgrades gone wrong. Unaccounted growth in files, surprise operating system limits, and countless other snags and roadbumps have hit the archive over nearly three decades. These problems are definitely not unique to the archive’s existence – many other websites and computers in the world experience the same snags.

Some of the snags have been localized – an item stops loading, or a filetype renders wrong in some browsers. Others will take out a rack of machines, a fleet of drives, and late nights or long days bring them back to service.

Further issues are even more generalized: Power outages due to weather or fire, or a cable (power or network) is sliced through by a misinformed construction crew. A solid heatwave takes some of the machines out for hours at a time.

Across the years, the Archive has had outages lasting minutes, hours, and even days.

In 2024, for the first time in recent history, it was weeks.

The Archive staff was now spending long days and nights auditing, assessing, and improving the entire infrastructure of the Archive, top to bottom. To the public, we looked completely down, and to some, waiting patiently and then less-patiently for the return of the site, they came to a conclusion: this was it.

For some people, the era of Internet Archive was over. The Wayback Machine, Open Library and the Internet Archive were, in one shocking stroke, gone.

This was, it turns out, not true. And it was also something surprising: an opportunity.

Among the things it is very difficult to do is attend your own funeral. You don’t get to stand among the mourners and hear their thoughts, and to find out what about you mattered to them, and what difference you made over the course of them knowing you.

You don’t hear the proclamations, the dedications, the thoughts about what inspirations and warnings your life held.

But in October, we did.

There is, naturally, an entire ecosystem dedicated to taking news about sites like the Archive being down and stretching them into 30 minute presentations, and there are articles and editorials about any events of note online.

But during this period of weeks, we also got to see the conversations, statements and posts of long-time users, who otherwise would not have communicated about their relationship with the holdings and offerings they’d used for so long.

For many people, the Archive is a standard part of their browsing life – a vast and complex shelf of media and pieces of culture that they reach out to in the process of their day.

For others, it’s a critical tool in their toolbox of research, be it verifying a source for an assignment or tracking down long-otherwise-removed sources that would be near impossible if not for the Wayback Machine or the stacks within the main site.

And the amount of people who spend their days and nights walking the collections, browsing idly and finding inspiration or entertainment or relief flipping through the items, is very significant.

The inherent invisibility of the Archive, however, can’t be ignored.

It’s clear that, for many patrons, when they look for something, they search for “SOMETHING internet archive” in their search engine or go directly to archive.org to search, but the existence of a “there” related to the archive had drifted into the background. The outage had brought the bulk of our collection and presence, the depth of it, into the foreground.

In this new attention came bewilderment at the downtime, and then a protective anger.

The Archive represents a shrinking population of sites on the web – it is not “for” a company or “for” shareholders, but is run and available “for” everyone, as much as it can afford, and facing down all the challenges that come with a constantly growing site being visited by millions of patrons, daily.

As time has passed and the years have progressed, it feels like the air you breathe and the water you drink: the place you walk through on your way to knowledge.

Staring into the void of a lost Internet Archive, people took to social media and communities to be scared, bothered, worried, and angry – and for many to recognize what part it plays in many people’s lives.

At the end of 2024, after a pretty tough year, with often-unsung employees within the Archive working incredible long and stressful hours to minimize the outages and downtime, it’s the comments from donors, posts on social media, and supportive communications (e-mail and otherwise) that have helped make everyone excited to face 2025 and beyond.

Usually, the tidal wave of users that pass through our machines remain as blinking lights on servers, and the Archive is simply a website that many people use. In this period of darkness and loss of access, everyone was reminded of the many other parts the archive plays in life, and that, at least, is a precious knowledge.

We’re glad to be back, and to be back with you. Here’s to the next year and the years to follow.