Tag Archives: video

improved h.264 derivatives!

Posted on by

We have thoroughly tested a newer and simpler way to create h.264 derivatives!

Changes you’ll notice:

  • More pixels!  previously 320 x 240    goes to 640 x 480 pixels
  • Slightly higher video bitrate — from about 512kb/s   to   about  700kb/s bitrate
  • Switching from mp4creator container maker to ffmpeg container + qt-faststart
  • Less back-end commands to make high-quality derivative

Nice things about this derivative (similar to prior derivative):

  • Plays in adobe flash plugin
  • Plays on all versions of iphone and ipad
  • Starts quickly, nearly instant seeking even to unbuffered areas of the video

Here’s a sample of how we do it with just 3 simple commands.  (We do/you should adjust “-r” argument appropriately to your video’s frames-per-second.  We also adjust the “640” in the “-vf scale” argument to be appropriate for the video’s *actual* aspect ratio, etc.  So for example, the 640 might become 852 for 16:9 widescreen video.  Although for our .mp4 specific derivative and playback ability on iPhone (1st gen and thus all versions), we would actually downrez that to 640×360).

ffmpeg -deinterlace -y -i 'camels.avi' -vcodec libx264 -fpre libx264-IA.ffpreset -vf scale=640:480 -r 20 -threads 2 -map_meta_data -1:0 -pass 1 -an tmp.mp4


ffmpeg -deinterlace -y -i 'camels.avi' -vcodec libx264 -fpre libx264-IA.ffpreset -vf scale=640:480 -r 20 -threads 2 -map_meta_data -1:0 -pass 2 -acodec aac -strict experimental -ab 128k -ac 2 -ar 44100 -metadata title='Camels at a Zoo - http://www.archive.org/details/camels' -metadata year='2004' -metadata comment=license:'http://creativecommons.org/licenses/by-nc/3.0/' tmp.mp4

qt-faststart tmp.mp4 'camels.mp4'

our preset file:
http://www.archive.org/~tracey/downloads/libx264-IA.ffpreset

 

For the adventurous out there, you can create this same setup by building ffmpeg on mac, linux, or windows.  Linux is easy, but personally, I’m a mac gal.  So here’s some ffmpeg build tips on the mac.

Happy viewing!

 

better mp4 (h.264) derivatives at archive.org!

Posted on by

Late last week, we pushed live a new video deriving technique, as well as in the process updated our audio/video file reader, ffmpeg.

New items will benefit from this newer method, and prior items can be re-derived by users if they desire (probably by the end of the year, we will rederive all our movies automatically).

The video will have significantly less “noise”, a higher PSNR (Peak Signal-to-Noise Ratio), and less”blocking” — all at similar or faster deriving speed to build and the same bitrate and filesize!

example new derivative frame

example new derivative frame

example old derivative frame

example old derivative frame

We now open the source video file up with ffmpeg, resize and convert it to raw video, and pipe it to the most recent build of “x264” tool (opting for baseline profile for iPhone, etc. compatibility).
For the very curious (and the very geeky 😉 here is a how we make our h.264 MPEG4 video files now:

• ffmpeg -i camels.avi -vn -acodec libfaac -ab 64k -ac 2 temp.aac
• ffmpeg -an -deinterlace -i camels.avi -s 320x240 -r 20 -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - 2>/dev/null | ffmpeg -an -f rawvideo -s 320x240 -r 20 -i - -f yuv4mpegpipe - 2>/dev/null | x264 --bitrate 512 --vbv-maxrate 768 --vbv-bufsize 1024 --profile baseline --pass 1 /dev/stdin --demuxer y4m -o temp.h264
• ffmpeg -an -deinterlace -i camels.avi -s 320x240 -r 20 -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - 2>/dev/null | ffmpeg -an -f rawvideo -s 320x240 -r 20 -i - -f yuv4mpegpipe - 2>/dev/null | x264 --bitrate 512 --vbv-maxrate 768 --vbv-bufsize 1024 --profile baseline --pass 2 /dev/stdin --demuxer y4m -o temp.h264
• mp4creator -c temp.h264 -r 20 t2.mp4
• mp4creator -c temp.aac -interleave t2.mp4
• ffmpeg -i t2.mp4 -acodec copy -vcodec copy -metadata title="Camels at a Zoo - http://www.archive.org/details/camels" -metadata year="2004" -metadata comment="license:http://creativecommons.org/licenses/by-nc/3.0/" camels_512kb.mp4
• mp4creator -optimize camels_512kb.mp4

–Tracey Jaquith

archive.org supports the new [video] HTML tag!

Posted on by

We now support the new [video] HTML tag when viewing movies from our site.

You’ll need Firefox v3.5+ (full or beta release) or Safari v4+ for us to show you a “try the new [video] tag” section under the “click to play” video area.

You can even persist a choice to always use it on our site (by us setting a cookie for you) (and you can change your mind and go back to the normal flash plugin based option).

-Tracey Jaquith


Bookmark and Share

Archive supports subtitles now!

Posted on by

Our flash-based player supports “SubRip” files (files with “.srt” extension). About a month ago, we updated our video pages to automatically support subtitles.

One simply needs to upload a file with a “.srt” extension (in the SubRip format) along with the video file to get started. If the item includes multiple video files/tracks, you can make multiple .srt files, example:
cow1.avi
cow1.srt
cow2.mov
cow2.srt

We support multilingual subtitling as well. Our suggested naming of .srt files for language-based tracks for the best display on our site is like:
cow.mov
cow.en.srt (english)
cow.fr.srt (french)
cow.hu.srt (hungarian)
and our site will show a selector for the three different languages
subtitle: [ en | fr | hu ]
next to the video track in our player.

Example short video with subtitling.

Enjoy!
–Tracey Jaquith

Bookmark and Share

Audio and Video improvements

Posted on by

Hi Patrons,

Yesterday we made live a large update to the way we create the audio and video displayed on our site.  Most folks might not notice the changes, so here’s a rundown:

For Audio:

  • We can now read and make mp3s from 24-bit Flac files. This has been requested for many years and we are thrilled to get it working.
  • The Ogg audio files that we create from audio files will now be using an updated “libvorbis” library.  (The library we were using before today was from 2001! 😎 )
  • We are no longer making 64kb MP3s (or zips or m3u playlists of those files).  This was a judgement call — given how poor the sound quality is for these files and the fact that most people are getting more and more bandwidth to their devices and computers.
  • Simplified back-end system, relying more and more on “ffmpeg” for format conversion.
  • We will now (try to) make derivatives from “.aac” (Advanced Audio Coding) files and “.ra”/”.rm” (Real Audio) files.
  • General ability to read more kinds of audio files more reliably.

For Movies:

  • The Ogg Video files that we create from movies files will now be using an updated “libvorbis” library for their audio.  (Previously we were using the “non reference” library ogg encoder.  Now we are using the much asked for and newer “libvorbis” library).
  • Updated ffmpeg to v0.5. This allows for a much wider range of source audio/video containers and codecs.  We will be able to derive HD-quality video formats like DV-50 and DV-100.   (For those interested in ffmpeg, changelog).
  • Better detection of widescreen movies (so less of our movies on our site will incorrectly appear “squooshed”).
  • General ability to read more kinds of video files more reliably.
  • Noting the prior point, we were able to get streaming videos for about 170 TV archive items that we could not process previously.

Enjoy!

–Tracey Jaquith

Bookmark and Share

New strategy for Internet Archive movies!

Posted on by

We have rebuilt all of our nearly 200,000 videos at the archive!

[We finished this Dec 1, 2008]

Related cross-blog with OLPC.

Here is a table-based chart of which video formats will be “derived” into which formats (we are creating 4 formats per video now):

http://www.archive.org/help/derivatives.php

Improvements and Changes from our prior movies techniques:

  • We will make a new Ogg Theora (with Vorbis audio) opensource/free-based video derivative. This derivative will play natively in Firefox 3.1 release (v3.1 is due around the end of 2008).
  • We are re/making h.264 MPEG-4 derivatives. We have updated the format to work with lighttpd “mod_h264_streaming” (which allows jumping into a movie at a specified time) but in the process will be losing the ability to serve/stream this file with RTSP.  This derivative also plays in the Adobe Flash plugin and plays on iPods/iPhones.
  • We are removing older 64kb and 256kb MPEG-4 derivatives.  With “progressive download” support becoming ubiquitous, even modems and phones are doing much better with downloading larger files.
  • We are removing older .flv “Flash Video” derivatives.  Since the much better quality h.264 derivative plays in recent flash plugins (as well as many other devices and browsers), the flash video alternative is seen as less ideal.
  • We are removing older .mpg MPEG-1 derivatives.  Their usefulness has declined in recent years, especially compared to h.264 alternatives.
  • We are remaking our animated GIFs. They attempt to make 30 thumbnails from each uploaded video.  We now evenly space them across the entire video.
  • We are remaking our Thumbnails. Similar to the GIF, we are spreading them across the videos better, and making less Thumbnails for items with *many* videos.  Additionally, we are renaming the thumbnails to indicate the second position in the video they were created at.  This will allow for the next bullet item…
  • We have developed the ability to jump into videos by clicking on the thumbnail image (to go to that scene!) We are finalizing the URL / permalinks for these “jump into video” URLs and will release this live to the public as soon as we can.

-tracey jaquith

Rederiving our movies to Ogg Theora and more!

Posted on by

[reposted and edited with generous consent from John Gilmore]

The Internet Archive has a collection of about 185,000 moving images,
including many cartoons and full-length movies that have fallen into
the public domain.  They offer full downloads in the best format they
have, as well as “re-derived” versions in other (typically smaller)
formats.  They also added a Flash-based video player in the last year
or two.   The “One Laptop Per Child”, or OLPC, software supports the Ogg Theora video
codec, but few movies had been uploaded in Ogg Theora, and none had
previously been re-derived into it.

The Archive actively supports the free software ecosystem, and is now
busy re-deriving copies of all their videos into both Ogg Theora and
H.264 (mp4) codecs.  So far they have more than 40% of the videos
converted, and hope to have the rest done by December 2008.
This makes each of these videos easily accessible on the OLPC XO, by
looking in the left margin for the download/stream link for the Ogg
Video version.  As each is converted, it immediately becomes
accessible at www.archive.org/details/movies.

The Archive is also noticing that the “OLPC” browser
is connecting, and replaces the Flash player with a direct link to the
.ogv Ogg Theora file.  This allows stock XO’s to play videos by
clicking on the big Click To Play image.  For example, try:

http://www.archive.org/details/merry_melodies_falling_hare

For the kids, they’ve already converted all 84 cartoons in this collection:

http://www.archive.org/details/classic_cartoons

You can also search their moving images collection for
format:”Ogg Video”
to restrict your search to movies that have a copy available in Ogg (Theora).

–tracey jaquith

Fast and reliable way to encode Theora Ogg videos using ffmpeg, libtheora, and liboggz

Posted on by


archive.org has started to make theora derivatives for movie files, where we create an Ogg Theora video format output for each movie file. after trying a bunch of tools over a good corpus of wide-ranging videos, i found a neat way to make the Archive derivatives.

High Level:

  • use ffmpeg to turn any video to “rawvideo”.
  • pipe its output to *another* ffmpeg to turn the video to “yuv4mpegpipe”.
  • pipe its output to the libtheora tool.
  • for videos with audio, ffmpeg create a vorbis audio .ogg file.
  • add tasty metadata (with liboggz utils).
  • combine the video and audio ogg files to an .ogv output!

Detailed example:

  • ffmpeg -an -deinterlace -s 400×300 -r 20.00 -i CapeCodMarsh.avi -vcodec rawvideo -pix_fmt yuv420p -f rawvideo – |  ffmpeg -an -f rawvideo -s 400×300 -r 20.00 -i – -f yuv4mpegpipe – |  libtheora-1.0/lt-encoder_example –video-rate-target 512k – -o tmp.ogv
  • ffmpeg -y -i CapeCodMarsh.avi -vn -acodec libvorbis -ac 2 -ab 128k -ar 44100 audio.ogg
  • oggz-comment audio.ogg -o audio2.ogg TITLE=”Cape Cod Marsh” ARTIST=”Tracey Jaquith” LICENSE=”http://creativecommons.org/licenses/publicdomain/” DATE=”2004″ ORGANIZATION=”Dumb Bunny Productions” LOCATION=http://www.archive.org/details/CapeCodMarsh
  • oggzmerge tmp.ogv audio2.ogg -o CapeCodMarsh.ogv

WTFs:

  • Why the double pipe above? Some videos could not go directly to yuv4mpegpipe format such that libtheora (or ffmpeg2theora) would work all the time.
  • We do the vorbis audio outside of libtheora (or ffmpeg2theora) to avoid any issues with Audio/Video sync.
  • We convert to yuv420p in the rawvideo step because ffmpeg2theora has (i think) some known issues of not handling all yuv422 video inputs (i found at least a few videos that did this).
  • We add the metadata to the audio vorbis ogg because adding it to the video ogv file wound up making the first video frame not a keyframe (!)

So this will end up working in Firefox 3.1 and greater — the new HTML “video” tag:

<video controls=”true” autoplay=”true” src=”http://www.archive.org/download/commute/commute.ogv”> for firefox betans </video>

This technique above worked nicely across a wide range of source and “trashy” 46 videos that I use for QA before making live a new way to derive our videos at archive.org.

-tracey jaquith