We’ve heard you: “Fix the Wayback Machine!”
Here’s what’s going on. The Internet Archive’s Wayback Machine has been hit by waves of high-volume automated traffic, and we’ve put protections in place to keep the service running. One recent change: we rewrote the message you see when a request is blocked with a 429 error — the HTTP code that means “too many requests.”
Those protections sometimes catch real people by mistake. If that’s happened to you, we’re sorry, and we appreciate your patience while we work to reduce the errors.
We’re getting better at telling abusive bots apart from the people who depend on the Wayback Machine every day. If you think you were blocked in error, email info@archive.org with your operating system, browser, and IP address, and we’ll look into it.
“High volume automated traffic”
Do you think that’s somebody training a model? Or an attempted DOS, possibly from the Trump regime?
LLM scraping seems a lot more likely to me. If so, it is really sad. They have plenty of other training material and websites to scrape: no need to do it so abusively to IA, a service that does is so helpful for so many people.
Everyone, including Donald Trump’s administration, values the archival records provided by Brewster Kahle and IA.
P.S. Mark Graham, thanks for fixing it so we don’t get the 429 error when logged in! That was considerate of you.
Just to clarify: are these DOS attack?
What is the policy for when “bot” traffic is good and when it’s not? I use AI to search the archive for historical data about certain products that is no longer available on the companies’ websites. I don’t want to drown out other traffic but also don’t want to have to do all of this manually. (And btw, I am a regular donor to the Internet Archive.)
Many people like me use mullvad vpn for example and almost always blocked. You can add a cloudflare like protection
Well I remember hearing the first time the old time Radio show and it was the Shadow in my Dad’s truck on the radio back in 1979 and now I listen to the old radio shows instead of listening to music.
The problem is that VPNs are completely blocked. There probably isn’t an easy way to deal with this.
Sharon Lois Bram I really love that you have them on here and love those cool interviews. I would love to know if you have any more hidden surprise Sharon Lois and Bram interviews concerts from 1984 to 1988 especially. Because that was the elephant show series time.
Did you know that there is a documentary about Sharon Lois Bram being created? I hope you can release some stuff of theirs like that from the 80s to celebrate that. It’s gonna be there almost 50 years of Sharon Lois and Bram. You guys need to celebrate that!
AIs are ruining everything
Harsh times
what would be a reasonable timeout, for consumers of the Wayback API or CDX API?
Is one request ever 0.1 seconds reasonable?
Thanks!
Has the Internet Archive considered denying requests to the Wayback Machine for recent content (e.g. younger than 1 month)? This might help prevent abuse from scrapers and crawlers while still preserving the core functionality of the Wayback Machine as an archive.
AI is actually ddosing the entire internet at this point
I have that impression too. Some websites get relentlessly ddos’d by scraper bots.
Then there are resident proxy networks. Just learned about them. They subvert consumer devices to make them a network node for AI scrapers. The FBI warned about it in July.
Thanks Mark for your work on this treasure of the internet.
Thank you for keeping this wonderful thing running!
Awesome work and completely understand what you’re trying to do.
Keep up the good work with the archives.
Sincerely
An avid archive.org user and contributor
Hello! I’ve lately been blocked by the wayback machine, likely due to using Mullvad, librewolf and a large collection of anti fingerprinting extensions.
Please consider adding a captcha or similar for blocked ip ranges.
I value access to information that I cannot find elsewhere, the clearnet is not an option for me.
Thank you for maintaining one of the most valuable pieces of the internet!
Hey man, don’t apologize! Screw them bots. Keep em out!
Everyone has been hit by the AI swarms(see the recent post by the linux git team)
It’s a rough situation, and kudos must be given to any effort made <3
Why has uploading from the U.K. so slow, and has been for past several years?.
E.g. 1 MB per 5 seconds. (No internet connection issues)
(I know it is not a problem at some places outside UK)
Can you fix it?…
https://archive.org/details/instructionbooka0000easy why can i not access this it worked earlier today
I go on all the books. None work. There is a blank.
Please reply to me and explain why this happened. Please reply to me.
My e-mail address is nfterno@aol.com
John Politis
Fix the website test
I’m trying to figure out what is going on with Open Library. During the last few days, there will be long periods when you can get to search results, but whenever you click on one of the search results to try to go to a book and borrow it, you get only a black screen. What’s the deal with that? It doesn’t appear to be my Internet connection. Everything else is working. Is this somehow related to the Wayback Machine problem?
Thanks for the update and for working to improve access for legitimate Wayback Machine users.
I truly would like to thank you for all of your guy’s work. The archives worth is priceless and I can’t even begin to show my depth of gratitude
Thank you very much for this update
Just want to encourage all your work and will be patient!!!