Tag Archives: Books

Celebrating 100 million tasks (uploading and modifying archive.org content)

Just over 8-1/2 years ago, I wrote a multi-process daemon in PHP that we refer to as “catalogd”.  It runs 24 hours a day, 7 days a week, no rest!

It is in charge of uploading all content to our archive.org servers, and all changes to uploaded files.

We recently passed the 100 millionth “task” (upload or edit to an archive “item”).

After starting with a modest 100 or so tasks/day, we currently run nearly 100,000 tasks/day.  We’ve done some minor scaling, but of the most part, the little daemon has become our little daemon that could!

Here’s to the next 100 million tasks at archive.org!

-tracey

Open Library Buying e-Books from Publishers

The Internet Archive is on campaign to buy e-Books from publishers and authors; making more digital books available to readers who prefer using laptops, reading devices or library computers.  Publishers such as Smashwords, Cursor and A Book Apart have already contributed e-Books to OpenLibrary.org – offering niche titles and the works of best-selling “indy” authors including Amanda Hocking and J.A. Konrath.

“Libraries are our allies in creating the best range of discovery mechanisms for writers and readers—enabling open and browser-based lending through the OpenLibrary.org means more books for more readers, and we’re thrilled to do our part in achieving that.” – Richard Nash, founder of Cursor.

American libraries spend $3-4 billion a year on publisher’s materials.  OpenLibrary.org and its more than 150 partnering libraries around the US and the world are  leading the charge to increase their combined digital book catalog of 80,000+ (mostly 20th century) and 2 million+ older titles.

“As demand for e-Books increases, libraries are looking to purchase more titles to provide better access for their readers.” – Digital Librarian Brewster Kahle, Founder of the Internet Archive.

This new twist on the traditional lending model promises to increase e-book use and revenue for publishers. OpenLibrary.org offers an e-Book lending library and digitized copies of classics and older books as well as books in audio and DAISY formats for those qualified readers.

All Icelandic literature to go online?

Þorsteinn Hallgrímsson, formerly of the National Library of Iceland, had a big idea:  digitize all Icelandic literature all the way to the current day and make it available to everyone interested in reading it. The Internet Archive was eager to be a part of this bold vision. I am in Iceland now, and because the financial crisis and Icelandic reaction to the US Department of Justice’s subpoenaing the tweets and Facebook account of a sitting member of the Icelandic Parliament, this project may have the momentum it needs to happen.

Ingibjörg Steinunn Sverrisdóttir, the National Librarian, and Katrín Jakobsdóttir, the Minister of Culture, met to discuss this possibility this week. I have met with several other ministers and parliamentarians in the last few days to discuss how this could be done.

The total literature of Iceland is under 50,000 books, which is easily scannable in 2 years by 12 people using the scribe scanners of the Internet Archive. David Lesperance, a lawyer from Canada who has helped support the Room to Read project, has offered to fundraise for this project; the Internet Archive has offered scanning technology, training, and backend software; and the Library has offered to administer the project. A digital lending system could be a way that they decide to limit access to a book to one person at a time in order to balance the interests of the writers and publishers while still having some access to everything from anywhere forever for free. Egill Helgason, of the Icelandic TV network, interviewed Brewster about this (photo below, video on the Archive).

If they decide to go ahead, Iceland could be the first country to have its complete literature go online. Fingers crossed.

The next step beyond this that is interesting to many here is to have Iceland become a “Switzerland of Bits,” where the laws will help protect the historical record from foreign or corporate danger. This is being promoted by Birgitta Jónsdóttir, a member of parliament. The Internet Archive works with many libraries around the world, and everyone wants to make sure that the digital copies are safe for the long term. Iceland is taking steps to be a good place for this.

As an aside, with all their inexpensive “green” electricity from their hydro electric and geothermal plants, I found it interesting that they are growing some vegetables under lights in the long winters as a way to become more self sufficient. With LED lights that can be tuned to produce specific wavelengths at different parts of the growth cycle, this approach could be a fairly energy efficient way to grow food for their people.

-Brewster Kahle

Go Books in Browsers from Google!

We are excited to see commercial books from many publishers being made available through web browser technology from Google eBookstore. As a standards based system, reading in a browser offers an opportunity for many more people to actively participate in the evolving digital book ecology.

The advantage of “books in browsers” over dedicated devices and even app store-based selling is that books can come from any website, read on many more devices, and be findable with standard search technologies.

The Google eBook Reader

Buying books that are delivered in a browser is now being demonstrated on a massive scale by Google. This is great news as it shows that the security measures offered are good enough for commercial players.

Lending books through a browser that recreates the traditional library-check-out system was demonstrated at the Books in Browsers 2010 summit at the Internet Archive. Lending and vending of books using browsers can pave the way for many winners:

Authors can find wider distribution for their work.
Publishers both big and small can now distribute books directly to readers.
Book sellers can find new and larger audiences for their products.
Device makers can offer access to millions of books instantly.
Libraries can continue to loan books in the way that patrons expect.
Readers could start to get universal access to all knowledge.

I am especially excited to see the possibilities in platform independent social reading and beautifully designed ebooks that could come from browser based books.

-brewster