Just over 8-1/2 years ago, I wrote a multi-process daemon in PHP that we refer to as “catalogd”. It runs 24 hours a day, 7 days a week, no rest!
It is in charge of uploading all content to our archive.org servers, and all changes to uploaded files.
We recently passed the 100 millionth “task” (upload or edit to an archive “item”).
After starting with a modest 100 or so tasks/day, we currently run nearly 100,000 tasks/day. We’ve done some minor scaling, but of the most part, the little daemon has become our little daemon that could!
Here’s to the next 100 million tasks at archive.org!
We know there will be a few minor breaks here and there especially from some third-party applications that might not handle “301 Moved Temporarily” redirects (if you have something flash-based that needs http://www.archive.org/crossdomain.xml we caught that breakage and that url still works now (that is, it can be either requested either with or without the lead “www.” as an exception now). We’re happy to work with anyone having issues — feel free to reply to this post and let us know.
Best wishes, and now go spend those four characters saved on something fun 😉
“Libraries are our allies in creating the best range of discovery mechanisms for writers and readers—enabling open and browser-based lending through the OpenLibrary.org means more books for more readers, and we’re thrilled to do our part in achieving that.” – Richard Nash, founder of Cursor.
American libraries spend $3-4 billion a year on publisher’s materials. OpenLibrary.org and its more than 150 partnering libraries around the US and the world are leading the charge to increase their combined digital book catalog of 80,000+ (mostly 20th century) and 2 million+ older titles.
“As demand for e-Books increases, libraries are looking to purchase more titles to provide better access for their readers.” – Digital Librarian Brewster Kahle, Founder of the Internet Archive.
From the articles:
“What we wanted to do out of this was to leave a very public cookbook for how to push back. That was our goal in our negotiations with the FBI. We would not have settled without being able to talk about what the letters look like, how to push back and who to call.” -Brewster Kahle
Zoia Horn presented the award and spoke of her own ordeal as the first librarian to be jailed for refusing to divulge information that violated her belief in intellectual freedom during the 1972 conspiracy trial of the “Harrisburg Seven” anti-war activists.
With so much information packed into the Internet Archive, it’s often hard to remember what your favorite items are or to go back to an item that you haven’t finished reading/watching/listening to. A simple way to keep track of your favorites is to use the bookmark feature, which is found on the left hand side of each item under “Resources.”
Clicking here will flag the item and allow you to keep everything you love on the Archive in one spot. You can create your own collection to refer back to or share with friends who may be new to the Archive or looking for some fresh material. You can also use the Bookmark Explorer to see some of the most recent and popular bookmarked items on the Archive.
My bookmarked items can be found at this page. Feel free to share your own bookmarks here!