Internet Archive Blogs

Over the past couple of years the Wayback Machine has been written about, or referenced, by journalists, researchers, academics and students in more than a thousand published news articles.

This week a CNN article used the Wayback Machine to bring to light writings of a public figure, that otherwise would have been lost, in a relevant and current context. Reading the article made me the happiest about leading the Wayback Machine project since I started 3 years ago.

I think it is fair to say that this article, written by Andrew Kaczynski, @KFILE of CNN, makes the case stronger, and more clearly, than any other, of the importance of cultural memory in general, and the Wayback Machine in particular, in the role of supporting a healthy political discourse and helping to hold those in power accountable.

The article cites two columns of now Vice President Mike Pence that were posted about 17 years ago and that can be read via the Internet Archive’s Wayback Machine here and here. However they return a 404 (page not found) error when accessed via the “Live Web” here and here, and have been gone from the live web for more than a decade.

The fruits of the Wayback Machine are the result of thousands of people over the past 20 years, working, volunteering and otherwise contributing to the Internet Archive’s efforts to preserve our cultural heritage and helping to make the web more useful and reliable.

If it were not for the Wayback Machine, the cogent and earnest writings of a columnist who became Vice President of the United States might not be available for us to reflect on, and benefit from, today.

To all those who value journalism, memory, context and perspectives, supporting the Internet Archive’s mission of Universal Access to All Knowledge is necessary now more than ever in our digital age.

Baffled by blockchain? Decentralization left you dazed and confused?  We’ve answered some frequently asked questions to help you make sense of this new area of technology.

Q:  Why create a Decentralized Web?

A:  The way we code the Web shapes how we live our lives online.  Ideally, that code should protect user privacy, freedom of expression and universal access to all knowledge.  Instead, centralized points of control make it easier for governments that are so inclined to censor and conduct surveillance, and for private companies to collect, share and monetize more personal information than many users would like.

A goal in creating a Decentralized Web is to reduce or eliminate such centralized points of control. That way, too, if any player drops out, the system still works. Such a system could better help protect user privacy, ensure reliable access, and even make it possible for users to buy and sell directly, without having to go through websites that now serve as middlemen, and collect user data in the process.

Q:  So how do you build an alternative? What are the components of a Decentralized Web?

A:  A new Decentralized Web requires a decentralized way to store and retrieve the files that make up websites, decentralized log-ins so users can interact, and a peer-to-peer payment system.  A distributed authentication system (proving you are who you say you are) could end the need for centralized usernames and passwords. Public key encryption could protect privacy, so users could have more confidence they weren’t being spied on.  Decentralized databases could allow information to ‘live’ in many different places, so information can’t easily be blocked or erased. The Decentralized Web should also have a time axis, making past versions of the Web accessible, similar to what the Internet Archive’s Wayback Machine does now.

Q:  Why does it matter?

A:  Online activities are now an important part of life for much of the world’s population. The original vision of the World Wide Web was to empower users, but many users now complain that too much power and user data is concentrated in the hands of too few corporate and government players, making it easier to conduct warrantless surveillance, feed the public disinformation and impose censorship.  It also makes it possible for state-sponsored or criminal hackers to scoop up personal data and passwords of millions of users at a time, and to use that information to create false identities, steal money and more. And over time, huge amounts of creative content—essays, musings, personal messages, photos, videos and other data— have disappeared when commercial entities shut down, or even when they just change their protocols. The average life of a webpage now is about 100 days before it’s changed or deleted. The Decentralized Web aims at least to mitigate, ideally to reverse or correct many of these trends, by putting control and ownership of data back in the hands of those who create it.

Q:  What is the significance of the Decentralized Web Summit? What’s the outcome you are hoping for?

A:  The Decentralized Web Summit, Aug. 1-2, 2018, will bring together hundreds of people interested in building a better Web. The Summit will include the creators and builders of the original Internet and World Wide Web, plus other developers of cutting-edge decentralized protocols and representatives of civil society, human rights and government from around the world. It’s our belief that technology alone cannot change society; it takes laws, policies, market forces, and the right set of values to make meaningful change. So we are convening people from many sectors to consider how to build the Web we want, and the Web we deserve.

This is the second installment; the first Decentralized Web Summit was produced by and held at the Internet Archive in San Francisco, June 7-9, 2016.

“The first Decentralized Web Summit was basically a ‘Hey! Did you know this is possible?’  It was a call to action,” says Brewster Kahle, founder of the Internet Archive. “The idea now is to try to get some coordinated effort to move this forward. There have been great advances in this direction over the past couple of years.  People are starting to show real working code and real projects. They’re building whole technology stacks that are more decentralized, in large part fuelled by the excitement of the cryptocurrency systems. The altcoins and Bitcoins are proving that interesting and complicated systems are starting to work out there.”

Q:  Why is the Internet Archive involved?

Builders, archivists, and civil society leaders discuss building a decentralized Web at the 2016 Decentralized Web Summit at the Internet Archive.

A:  The Internet Archive has been archiving the World Wide Web for 20 years, saving different versions of webpages over time, and making them openly available to anyone using the Wayback Machine (available at https://archive.org/web/).  The Decentralized Web would build the Wayback Machine into the DWeb, and the DWeb into Wayback Machine, says Internet Archive founder Brewster Kahle. “The Internet Archive is dedicated to the open world,” says Kahle. “We’re only going to survive if the open world is more interesting than closed app worlds on cell phones, or a dystopian world of closed, segmented, siloed, corporately-owned little pieces of property.  We would rather see an open, next-generation Web succeed.”

Q:  What’s meant by ‘decentralized storage’?  Is the blockchain the same as the Decentralized Web?

A:  Decentralized storage means that rather than the current system under the World Wide Web, where a file exists in one physical location on a server, a file can instead be stored in multiple computers around the world, possibly with just bits of the file in each one, that all come together and get sent to a user who requests that file. BitTorrent was one early example of this.

A particular form of decentralized storage is a verification system of transactions that uses blockchain technology. One company that uses the blockchain is Ethereum, which allows multiple users to agree to store and verify data on their computers, and in exchange for doing this, they get value in a cryptocurrency called Ether. This is different from Bitcoin, says Ethereum’s creator Vitalik Buterin because, as he said at TechCrunch SF 2017, “with Bitcoin, the protocol is in service of the currency, but with Ether and Ethereum, the currency is in service of the protocol.” That is, the currency provides an incentive for its holders to store small bits of data that can help create a Decentralized Web.

A challenge, Buterin says, is that blockchain is vastly less efficient and more costly, in terms of using energy and computing time, than centralized Web offerings, and that approaches that could make blockchain more efficient might do so at a cost to privacy or security, while efforts to increase privacy and security could further reduce efficiency. These are challenges for developers and builders of blockchain-based protocols to work out.

There are many decentralized protocols that are not based on a blockchain. But in the overheated publicity over blockchain technology, the public has started to conflate blockchains and the Decentralized Web.  The two are related, but not synonymous.

Q:  If creating a Decentralized Web is a decentralized effort, how does it all come together? Who’s in charge?

A:  Creating common coding standards would allow independent developers to go off and create their own component parts, that can fit together in various ways, like the way Lego pieces can be assembled and reassembled. This approach worked for the original World Wide Web. Common standards allowed developers to contribute, without having to go through a central authority for approval. As long as new websites or apps used the standards, they could go online and become part of the Web.

Q:  How will all a Decentralized Web change a user’s online experience? Do we all have to relearn everything? That sounds like a lot of work.

A:  The hope is that new Decentralized Web components will gradually be integrated into existing and new servers over time, so that the experience for individual users is as a smooth, if not seamless, transition to a better, safer online experience in which a user retains ownership and control of his or her own data.

Q:  How does net neutrality relate to the Decentralized Web?

A:  Net neutrality is the principle that no data online is advantaged or disadvantaged, in terms of access or flow. The closer you can get to net neutrality, the better for a Decentralized Web. It would be difficult if not impossible to have full Decentralized Web access under an authoritarian government that controls and censors the internet, and can cut off access if it sees a user trying to use Decentralized Web tools. China, in particular, has already proven adept at slowing or stopping internet connections, and blocking access to VPNs and other tools meant to circumvent surveillance and censorship. Even non-authoritarian governments may demand a backdoor to do online surveillance. The end of net neutrality in the United States could also make it possible for an internet provider to slow access to DWeb tools and websites. That’s the bad news. The good news is that increased efforts to control the Web may motivate developers and builders of the Decentralized Web to work faster and more creatively to find work-arounds.

Q:  Will the Decentralized Web replace the current Web, be a parallel, separate entity, or exist as an integrated part of the existing Web?

A:  It’s impossible to know what will happen in the long run—quite possibly some combination of all three. In the short-term, component parts of the Decentralized Web are already starting to become available online, via the existing Web, and may eventually be integrated into browsers.

Q:  How does the Decentralized Web scale up, so enough people are using it to make it more than just a niche interest of a small segment of Web users?

A:  Some revolutionary technologies are adopted quickly, others more gradually.  World Wide Web creator Tim Berners-Lee said in his 2014 TED talk that adoption of the World Wide Web went from five percent of the world’s population in 2000, to 40 percent in 2014. The hope is that the promise of the Decentralized Web, to provide users with more control of their online experience and of their own data, and to better preserve data online, will rapidly draw people to adopt and use Decentralized Web tools.

Q:  So that’s how the Decentralized Web can make life better. How might it make life worse?

A:  Any technology is a tool, and many tools can be used constructively or destructively. The same technology that protects users from central surveillance, might also protect criminals  and hide their activity. That’s one thing. Another is, if information is stored in decentralized ways—say, with bits of each file stored on multiple computers around the world, and/or embedded in a blockchain—how do you ever truly get rid of information you no longer want to have online?  For Europeans, who have advocated for a ‘right to be forgotten,’ that could be a concern.

Q:  Aren’t big online companies going to push back against this, since it will affect their business model of monetizing their users’ data? How will social media and other companies related to online activity make a profit?

A:  Big online companies may well push back, because their current core business model of monetizing user data will not work well in a Decentralized Web environment. But it needn’t be a zero-sum game. It’s likely that other business models will emerge to take advantage of the opportunities offered by the Decentralized Web, much as companies have, over time, found ways to profit from open source endeavors.  Whether existing social media and internet-related companies choose to try to obstruct the development of the Decentralized Web, or to recognize this is the kind of Web users actually want and find ways to profit within the new model, is a decision each will have to make.

Q:  Ok, sign me up. How and when can I get onto the Decentralized Web?  

A:  Some apps and programs, built on the decentralized model, are already available, and you can sign up and use them at will. But the Decentralized Web, as an envisioned ecosystem, might not be fully functional and integrated for another five or ten years. Remember how it felt to use the early World Wide Web? Apps and features came and went. Some of it was buggy, and some of it was revelatory, and users helped developers figure out what needed to happen, and how it could better come together. Expect another era like that, building on what we’ve already learned, to create a better Web for everybody.

Still have questions?  Visit this page to see more quotes from key Decentralized Web players.

Now you may be ready to sign up for the Decentralized Web Summit 2018:  Global Visions/Working Code.

More questions still? Contact Dwebsummit@archive.org.

On June 13, nearly a hundred developers, technologists and decentralized web enthusiasts met up at the Internet Archive to learn from each other and share their progress towards building towards a new, decentralized web. Fueled by pizza and beer, the conversation segued into a series of presentations by six keynote speakers interspersed with 1-minute lightning talks by eager participants. Filled with both vigor and technical nuance, the demos provided a sneak peek into next month’s Decentralized Web Summit.

Internet Archive founder Brewster Kahle set the tone by welcoming all to the Archive and defining the web as a “radical sharing experiment” that has encountered major issues of privacy, surveillance and manipulation. He challenged the audience with the toughest questions, asking “What if we can evolve the Web? What if we can bake our core principles into the Web? What if we can lock the Web open? What if we can build a Decentralized Web?” Brewster’s emphasis on the DWeb’s sharing and collaborative creative process echoed throughout the rest of the night.

Part of the network map of the Decentralized Web ecosystem by the Digital Life Collective’s Christina Bowen.

The Digital Life Collective’s co-founder and knowledge ecologist Christina Bowen was the first keynote speaker up, presenting an interactive map of the decentralized web ecosystem. Sortable by maturity, platforms and other characteristics, the map is a transformational tool for understanding the progress of the DWeb and gaining lateral awareness of our peers. To add your organization to the map, please fill out this survey and check out the map’s progress here!

Feross Aboukhadijeh, Founder of WebTorrent, was up next, informing us about the progress of WebTorrents since the last DWeb Summit in 2016. To much excitement, he explored how P2P torrenting ideas can be integrated into web browsers while also presenting the success of WebTorrent’s desktop application. Feross continually focused on the community aspect of WebTorrent and presented features made by other members. Check out his slides.

Arkadiy Kukarkin of Protocol Labs, the parent organization of IPFS, Filecoin and many other projects, then proceeded to present the recent research they have been conducting, in particular demoing Peerpad, a P2P collaborative editing tool. Other topics presented touched upon IPFS gateways and linking the IPFS network. For more research see here.

The Internet Archive’s Lead for Decentralized technologies, Mitra Ardron, presented his progress making the Archive more decentralized, elaborating on how the Archive’s decentralized platforms allows users make copies and use IPFS and WebTorrent links when putting out metadata. Check out for yourself here !

Joachim Lohkamp, founder of Jolocom followed by asking the audience a question: “how many account log-ins do we have online?” With the answer in the hundreds, Joachim presented the case for self-sovereign digital identity to control our online presence and security in a world where we have hundreds of accounts, passwords, and companies holding bits and pieces of our personal identities. Check out Joachim’s open source project.

Mozilla’s Irakli Gozalishvili, Dietrich Ayala and Tantek Çelik presented their work on browser support for decentralized web projects! These new APIs for decentralized development received thunderous applause, especially from the other protocol teams. The overall sense of collaboration and mutual support between groups was best demonstrated in IndieWeb member, Tantek’s call for longevity, for building a sustainable web together.

The lightning talks included an array of projects, from Johannes Ernst’s elucidation on the need for self-hosting and home servers to Devon James’s Open Index Protocol to Christopher Allen’s W3C credentials community. These talks demonstrated the vigor and liveliness of the space, both from the early pioneers and eager new faces.

This was an exciting prologue to what is panning out to be a groundbreaking conference on August 1st and 2nd. To sign up for the Decentralized Web Summit, please visit the conference website !

Video of the event and links towards all the speakers is available here https://archive.org/details/DWebMeetUp