Category Archives: Video Archive

NSA TV Clip Library

Posted on by


When the American people find out how their government has secretly interpreted the Patriot Act, they are going to be stunned and they are going to be angry.  Senator Ron Wyden May 26, 2011

Recent revelations of the extent of National Security Agency surveillance and weakening of our digital infrastructure give substance to the warnings of Senator Wyden and others. To assist journalists and other concerned citizens in reflecting on these issues, the Internet Archive has created a curated library of short television news clips presenting key statements and other representations.

NSA-issues TV News Quote Library

The experimental, Chrome and Safari only, library launches today with more than 700 chronologically ordered television citations drawn from the Archive’s television news research service. The TV quotes can be browsed by rolling over clip thumbnails, queried via transcripts and sorted for specific speakers. Citation links, context, links to source broadcasters and options to borrow can be explored by following the More/Borrow links on each thumbnail.

new mp4 (h.264) derivative technique — simpler and easy!

Posted on by

Greetings video geeks!  😎

We’ve updated the process and way we create our .mp4 files that are shown on video pages on archive.org

It’s a much cleaner/clearer process, namely:

  • We opted to ditch ffpreset files in favor of command-line argument 100% equivalents.  It seems a bit easier for someone reading the task log of their item, trying to see what we did.
  • We no longer need qt-faststart step and dropped it.  we use the cmd-line modern ffmpeg “-movflags faststart”
  • Entire processing is now done 100% with ffmpeg, in the standard “2-pass” mode
  • As before, this derivative plays in modern html5 video tag compatible browsers, plays in flash plugin within browsers, and works on all iOS devices.   it also makes sure the “moov atom” is at the front of the file, so browsers can playback before downloading the entire file, etc.)
Here is an example (you would tailor especially the “scale=640:480” depending on source aspect ratio and desired output size;  change or drop altogether the “-r 20” option (the source was 20 fps, so we make the dest 20 fps);  tailor the bitrate args to taste):
  • ffmpeg -y -i stairs.avi -vcodec libx264 -pix_fmt yuv420p -vf yadif,scale=640:480 -profile:v baseline -x264opts cabac=0:bframes=0:ref=1:weightp=0:level=30:bitrate=700:vbv_maxrate=768:vbv_bufsize=1400 -movflags faststart -ac 2 -b:a 128k -ar 44100 -r 20 -threads 2 -map_metadata -1,g:0,g -pass 1 -map 0:0 -map 0:1 -acodec aac -strict experimental stairs.mp4;
  • ffmpeg -y -i stairs.avi -vcodec libx264 -pix_fmt yuv420p -vf yadif,scale=640:480 -profile:v baseline -x264opts cabac=0:bframes=0:ref=1:weightp=0:level=30:bitrate=700:vbv_maxrate=768:vbv_bufsize=1400 -movflags faststart -ac 2 -b:a 128k -ar 44100 -r 20 -threads 2 -map_metadata -1,g:0,g -pass 2 -map 0:0 -map 0:1 -acodec aac -strict experimental -metadata title='”Stairs where i work” – lame test item, bear with us – http://archive.org/details/stairs’ -metadata year=’2004′ -metadata comment=license:’http://creativecommons.org/licenses/publicdomain/’ stairs.mp4;

Happy hacking and creating!

PS: here is the way we compile ffmpeg (we use ubuntu linux, but works on macosx, too).

new video and audio player — video multiple qualities, related videos, and more!

Posted on by

Many of you have already noticed that since the New Year, we have migrated our new “beta” player to be the primary/default player, then to be the only player.

We are excited about this new player!
It features the very latest release of jwplayer from longtailvideo.com.

Here’s some new features/improvements worth mentioning:

  • html5 is now the default — flash is a fallback option.  a final fallback option for most items is a “file download” link from the “click to play” image
  • videos have a nice new “Related Videos” pane that shows at the end of playback
  • should be much more reliable — I had previously hacked up a lot of the JS and flash from the jwplayer release version to accommodate our various wants and looks — now we use mostly the stock player with minimal JS alterations/customizations around the player.
  • better HD video and other quality options — uploaders can now offer multiple video size and bitrate qualities.  If you know how to code web playable (see my next post!) h.264 mp4 videos especially, you can upload different qualities of our source video and the viewer will have to option to pick any of them (see more on that below).
  • more consistent UI and look and feel.  The longtailvideo team *really* cleaned up and improved their UI, giving everything a clean, consistent, and aesthetically pleasing look.  Their default “skin” is also greatly improved, so we can use that now directly too
  • lots of cleaned up performance and more likely to play in more mobile, browsers, and and OS combinations under the hood.

Please give it a try!

-tracey

 

For those of you interested in trying multiple qualities, here’s a sample video showing it:

http://archive.org/details/kittehs

To make that work, I made sure that my original/source file was:

  • h.264 video
  • AAC audio
  • had the “moov atom” at the front (to allow instant playback without waiting to download entire file first) (search web for “qt-faststart” or ffmpeg’s “-movflags faststart” option, or see my next post for how we make our .mp4 here at archive.org)
  • has a > 480P style HD width/height
  • has filename ending with one of:   .HD.mov   .HD.mp4   .HD.mpeg4    .HD.m4v

When all of those are true, our system will automatically take:

  • filename.HD.mov

and create:

  • filename.mp4

that is our normal ~1000 kb/sec “derivative” video, as well as “filename.ogv”

The /details/ page will then see two playable mpeg-4 h.264 videos, and offer them both with the [HD] toggle button (seen once video is playing) allowing users to pick between the two quality levels.

If you wanted to offer a *third* quality, you could do that with another ending like above but with otherwise the same requirements.  So you could upload:

  • filename.HD.mp4       (as, say, a 960 x 540 resolution video)
  • filename.HD.mpeg4   (as, say, a 1920 x 1080 resolution video)

and the toggle would show the three options:   1080P, 540P, 480P

You can update existing items if you like, and re-derive your items, to get multiple qualities present.

Happy hacking!

 

 

 

files.xml and metadata API now showing video file’s width, height, and duration

Posted on by

For those video oriented of you who inspect our _files.xml directly, or use a metadata API (JSON) for items, you’ll likely be pleased to know that for all new items, as well as items that update (eg: review posted, metadata changes, rederives), that:

we now detect and stamp into each video file fields for
width
height
length (duration in seconds)

here is an example file listing (in JSON results) demonstrating:
{“name”:”anamorphic.mov”,
“source”:”original”,
“format”:”h.264″,
“length”:”3.03″,
“height”:”720″,
“width”:”1280″,
“mtime”:”1355278126″,
“size”:”375646″,
“md5″:”94a0b479869ab7ee997b36b45d9edb7a”,
“crc32″:”0b8e4162”,
“sha1″:”c21b65ee8e9153f0812d7397fe89224d6f3f80fd”}

we will even attempt to “correct” the width in the case of “anamorphic” videos (eg: the pixels are “rectangular” and not “square”) to the scaled width in square pixel terms (like the one above)

cheers!

A/V Geeks fundraiser to digitize 100 miles of film

Posted on by

A/V Geeks has done a lot of digitization of old film for The Internet Archive. They are trying to raise funds to digitize many more hours of footage to put up on archive.org which will be free to view and use by the public. If you would like to contribute here is some information:

http://www.indiegogo.com/avgeeks100miles

WHAT? The A/V Geeks have over 24,000 old 16mm educational films that we’ve rescued from landfills, dumpsters, closets, school libraries. These films cover topics from Atomic Bombs to Zoo Babies and provide an entertaining yet insightful glimpse into our past. We’ve kept these materials from being thrown out and we want to continue our mission by giving them a new life and sharing them you!

Traditionally, this means that we would have to either travel to you or you would have to travel to us to watch a film on a projector. By digitizing these films we can give you and the world access to these materials! When a film from the A/V Geeks archive is digitized and uploaded to the internet, it can be easily accessed, watched, downloaded, researched and repurposed for music videos, class projects, documentaries and more.

Help us digitize 100 Miles of Film! Instead of traveling miles to give access to these films we’d rather digitize miles of 16mm film. Film length is generally measured in feet (in the US) to where 2100 feet of film roughly equals one hour of content. With your support we can digitize and make available 100 miles of film – over 240 hours of (around 1,000 individual titles) from the archive. So we envision this as sort of a road rally. We have to go 100 miles in a short period of time. We’ll have a small team to keep the machines humming along. We aren’t sure what we’ll find with some of the films – we haven’t seen them yet!

For every $500 we raise, we digitize one mile of film (nearly 2.5 hours of material)!

Improved theora/ogg video derivatives!

Posted on by

We’ve made our ogg video derivatives slightly better via:

  • minor bump up to “thusnelda” release
  • “upgrade” from 1-pass video encoding to 2-pass video encoding
  • direct ffmpeg creation of the video (you’ll need to re/compile ffmpeg minimally with “–enable-libtheora –enable-libvorbis” configure flags)

ffmpeg -y -i ‘camels.avi’ -q:vscale 3 -b:v 512k -vcodec libtheora -pix_fmt yuv420p -vf yadif,scale=400:300 -r 20 -threads 2 -map_metadata -1,g:0,g -pass 1 -an -f null /dev/null;

ffmpeg -y -i ‘camels.avi’ -q:vscale 3 -b:v 512k -vcodec libtheora -pix_fmt yuv420p -vf yadif,scale=400:300 -r 20 -threads 2 -map_metadata -1,g:0,g -pass 2 -map 0:0 -map 0:1 -acodec libvorbis -ac 2 -ab 128k -ar 44100 -metadata TITLE=’Camels at a Zoo’ -metadata LICENSE=’http://creativecommons.org/licenses/by-nc/3.0/’ -metadata DATE=’2004′ -metadata ORGANIZATION=’Dumb Bunny Productions’ -metadata LOCATION=http://archive.org/details/camels camels.ogv

some notes:

  • You’d want to adjust the “scale=WIDTH:HEIGHT” accordingly, as well as the “-r FRAMES-PER-SECOND” related args, to your source video.
  • I made a small patch to allow *both* bitrate target *and* quality level for theora in ffmpeg, after comparing the other popular tool “ffmpeg2theora” code with the libtheoraenc.c inside ffmpeg.  It may not be necessary, but I believe I saw *slightly* better quality coming out of theora/thusnelda ogg video.  For what it’s worth, my minor patch is here:  http://archive.org/~tracey/downloads/patches/ffmpeg-theora.patch
  • The way we compile ffmpeg (ubuntu/linux) is here.  (Alt MacOS version here )
  • (Edited post above after I removed this step) It’s *quite* odd, I realize to have ffmpeg transcode both the audio/video together, only to split/demux them back out temporarily.  However, for some videos, the “oggz-comment” step would wipe out the first video keyframe and cause unplayability in chrome (and the expected visual artifacts for things that could play it).   So, we split, comment the audio track, then re-stitch it back together.

Lost Landscapes SF6: huge success– Next Lost Landscapes of Detroit February 22

Posted on by

Standing room only for Rick Prelinger’s Lost Landscape of San Francisco 6 at the Internet Archive last night.   New films including “process plates” from studios brought a new sharpness to many of the films presented.   Suggested donations was 5 bucks or 5 books, and people brought lots of great books for the Archive.

Next is Lost Landscapes of Detroit on February 22, 2012– this is Detroit without the narratives being imposed on it.   Doors open at 6:30, show at 7:30.

Thank you all!

new off-site video/audio embed codes

Posted on by

We are about to rollout a “new new” video/audio player 😎

You can see it in action now with our upcoming embed codes to go with this new player.

It will allow for additional much wanted features like:
– off-site playlists
– fullscreen in many cases
– subtitles/captions

as well as the standard arbitrary width/height and “autoplay” options.

You can see some examples here:

http://www.archive.org/help/video.php

The rest is coming soon (if you are eager, you can even “opt in” now by clicking here:

http://www.archive.org/details/movies&newplayer=jw

(then take a look at one of your favorite items).

Now relax, sit back, and enjoy an archive video!

Cheers!
–tracey

improved h.264 derivatives!

Posted on by

We have thoroughly tested a newer and simpler way to create h.264 derivatives!

Changes you’ll notice:

  • More pixels!  previously 320 x 240    goes to 640 x 480 pixels
  • Slightly higher video bitrate — from about 512kb/s   to   about  700kb/s bitrate
  • Switching from mp4creator container maker to ffmpeg container + qt-faststart
  • Less back-end commands to make high-quality derivative

Nice things about this derivative (similar to prior derivative):

  • Plays in adobe flash plugin
  • Plays on all versions of iphone and ipad
  • Starts quickly, nearly instant seeking even to unbuffered areas of the video

Here’s a sample of how we do it with just 3 simple commands.  (We do/you should adjust “-r” argument appropriately to your video’s frames-per-second.  We also adjust the “640” in the “-vf scale” argument to be appropriate for the video’s *actual* aspect ratio, etc.  So for example, the 640 might become 852 for 16:9 widescreen video.  Although for our .mp4 specific derivative and playback ability on iPhone (1st gen and thus all versions), we would actually downrez that to 640×360).

ffmpeg -deinterlace -y -i 'camels.avi' -vcodec libx264 -fpre libx264-IA.ffpreset -vf scale=640:480 -r 20 -threads 2 -map_meta_data -1:0 -pass 1 -an tmp.mp4


ffmpeg -deinterlace -y -i 'camels.avi' -vcodec libx264 -fpre libx264-IA.ffpreset -vf scale=640:480 -r 20 -threads 2 -map_meta_data -1:0 -pass 2 -acodec aac -strict experimental -ab 128k -ac 2 -ar 44100 -metadata title='Camels at a Zoo - http://www.archive.org/details/camels' -metadata year='2004' -metadata comment=license:'http://creativecommons.org/licenses/by-nc/3.0/' tmp.mp4

qt-faststart tmp.mp4 'camels.mp4'

our preset file:
http://www.archive.org/~tracey/downloads/libx264-IA.ffpreset

 

For the adventurous out there, you can create this same setup by building ffmpeg on mac, linux, or windows.  Linux is easy, but personally, I’m a mac gal.  So here’s some ffmpeg build tips on the mac.

Happy viewing!

 

Lost Landscapes of San Francisco

Posted on by
San Francisco

The Embarcadero

On April 10, 2011 Internet Archive kicked off the Internet Archive Presents public events series with a hometown favorite, Rick Prelinger’s Lost Landscapes of San Francisco.

Culled from thousands of hours of home, commercial and institutional movies, Lost Landscapes presents San Francisco the way it was. From Gliders in the Outlands to Joe DiMaggio’s wedding to cityscapes of long gone people and places the movie offers both vitality and nostalgia. What makes the event especially vibrant is that the audience participates by shouting out places, events and people they recognize. Occassionally this results in a conversation about the backstory of these clips.

San Francisco

San Francisco Airport

With over 400 people in attendance with a suggested donation of ‘5 bucks or 5 books” it was deemed a huge success. More public events are planned.

Check out video of the event at http://www.archive.org/details/lostlandscapes2011

-Jeff Kaplan