Category Archives: Technical

Using Docker to Encapsulate Complicated Program is Successful

The Internet Archive has been using docker in a useful way that is a bit out of the mainstream: to package a command-line binary and its dependencies so we can deploy it on a cluster and use it in the same … Continue reading

Posted in Music, Technical | 3 Comments

Job Posting: Web Application/Software Developer for Archive-It

The Internet Archive is looking for a smart, collaborative and resourceful engineer to lead and do the development of the next generation of the Archive-It service, a web based application used by libraries and archives around the world. The Internet … Continue reading

Posted in Archive-It, Jobs, News, Technical | Comments Off

How to use the Virtual Machine for Researchers

Some researchers that are working with the Internet Archive, such as those at University of Massachusetts, have wanted closer access to some of our collections. We are learning how to support this type of “on-campus” use of the collections. This … Continue reading

Posted in Technical | Tagged | 2 Comments

Site down some of Tuesday and Wednesday for Power Upgrade

[Update:   Upgrade is done, we were offline twice, as we predicted (and are sorry about), but now we have twice the power. Thank you PG&E, Ralf Muehlen, and the Archive engineers.] This week, we are doubling the power coming into … Continue reading

Posted in News, Technical | 3 Comments

Celebrating 100 million tasks (uploading and modifying archive.org content)

Just over 8-1/2 years ago, I wrote a multi-process daemon in PHP that we refer to as “catalogd”.  It runs 24 hours a day, 7 days a week, no rest! It is in charge of uploading all content to our … Continue reading

Posted in News, Technical | Tagged , , , , | 1 Comment

new mp4 (h.264) derivative technique — simpler and easy!

Greetings video geeks!   We’ve updated the process and way we create our .mp4 files that are shown on video pages on archive.org It’s a much cleaner/clearer process, namely: We opted to ditch ffpreset files in favor of command-line argument … Continue reading

Posted in Technical, Television Archive, Video Archive | Tagged , , , , , | 3 Comments

new video and audio player — video multiple qualities, related videos, and more!

Many of you have already noticed that since the New Year, we have migrated our new “beta” player to be the primary/default player, then to be the only player. We are excited about this new player! It features the very … Continue reading

Posted in Audio Archive, Live Music Archive, Technical, Television Archive, Video Archive | Tagged , , , , | 4 Comments

getting only certain formats in .zip files from items — new feature

Per some requests from our friends in the Live Music Archive community… You can get any archive.org item downloaded to your local machine as a .zip file (that we’ve been doing for 5+ years!) But whereas before it would be … Continue reading

Posted in Audio Archive, Live Music Archive, Technical | Tagged , , , | Comments Off

Uploading images for text items (update on *_images.zip format)

The old news Until about a year ago, if you wanted to upload a set of individual page images and have them be recognized as a “book” so we’d create the usual derivative formats from them, you had to mimic … Continue reading

Posted in Books Archive, Technical | 1 Comment

Improved theora/ogg video derivatives!

We’ve made our ogg video derivatives slightly better via: minor bump up to “thusnelda” release “upgrade” from 1-pass video encoding to 2-pass video encoding direct ffmpeg creation of the video (you’ll need to re/compile ffmpeg minimally with “–enable-libtheora –enable-libvorbis” configure … Continue reading

Posted in Technical, Video Archive | Tagged , , , , | 3 Comments

Downloading in bulk using wget

If you’ve ever wanted to download files from many different archive.org items in an automated way, here is one method to do it. ____________________________________________________________ Here’s an overview of what we’ll do: 1. Confirm or install a terminal emulator and wget … Continue reading

Posted in Technical | 34 Comments

IA forums now do “inline reply”

One thing that’s driven me a bit kooky is every time I “reply to this post” in the Internet Archive forums, it drives the browser to a new form page with no context/content of the post I was just looking … Continue reading

Posted in Technical | Tagged | 1 Comment

new off-site video/audio embed codes

We are about to rollout a “new new” video/audio player You can see it in action now with our upcoming embed codes to go with this new player. It will allow for additional much wanted features like: – off-site playlists … Continue reading

Posted in Technical, Video Archive | Tagged , , , | 11 Comments

Open Hardware: Inexpensive Enclosures From Junction Boxes.

I had a need for a cheap, standard enclosure for a humidity and temperature monitoring project. While there are many, many options for enclosures out there, few are cheap AND locally available. It occurred to me that electrical junction boxes … Continue reading

Posted in Technical | 14 Comments

improved h.264 derivatives!

We have thoroughly tested a newer and simpler way to create h.264 derivatives! Changes you’ll notice: More pixels!  previously 320 x 240    goes to 640 x 480 pixels Slightly higher video bitrate — from about 512kb/s   to   … Continue reading

Posted in Technical, Video Archive | Tagged , , , , , | 25 Comments

How Archive.org items are structured

What is an item? An item is a logical “thing” that we present on one web page on archive.org. An item may be one video file along with scans of the DVD cover, one book, one audio file, or a … Continue reading

Posted in Technical | 7 Comments

How Montana State Library Uploaded Batches of Digital Objects to the Internet Archive

by Chris Stockwell for Montana State Library, 12/29/2010 Introduction The Montana State Library (MSL) last year moved a copy of its collection of 3000 born digital state publications to the Internet Archive (IA). Since MSL will be continuing to upload … Continue reading

Posted in Books Archive, Technical | 10 Comments

New BookReader!

By mang We’re pleased to announce the release of our freshly re-designed BookReader on the Internet Archive. The updated BookReader has these great new features (links will take you to a live example): Redesigned user interface that maximizes the amount … Continue reading

Posted in Books Archive, Open Library, Technical | 23 Comments