Tag Archives: h.264

new mp4 (h.264) derivative technique — simpler and easy!

Posted on by

Greetings video geeks!  😎

We’ve updated the process and way we create our .mp4 files that are shown on video pages on archive.org

It’s a much cleaner/clearer process, namely:

  • We opted to ditch ffpreset files in favor of command-line argument 100% equivalents.  It seems a bit easier for someone reading the task log of their item, trying to see what we did.
  • We no longer need qt-faststart step and dropped it.  we use the cmd-line modern ffmpeg “-movflags faststart”
  • Entire processing is now done 100% with ffmpeg, in the standard “2-pass” mode
  • As before, this derivative plays in modern html5 video tag compatible browsers, plays in flash plugin within browsers, and works on all iOS devices.   it also makes sure the “moov atom” is at the front of the file, so browsers can playback before downloading the entire file, etc.)
Here is an example (you would tailor especially the “scale=640:480” depending on source aspect ratio and desired output size;  change or drop altogether the “-r 20” option (the source was 20 fps, so we make the dest 20 fps);  tailor the bitrate args to taste):
  • ffmpeg -y -i stairs.avi -vcodec libx264 -pix_fmt yuv420p -vf yadif,scale=640:480 -profile:v baseline -x264opts cabac=0:bframes=0:ref=1:weightp=0:level=30:bitrate=700:vbv_maxrate=768:vbv_bufsize=1400 -movflags faststart -ac 2 -b:a 128k -ar 44100 -r 20 -threads 2 -map_metadata -1,g:0,g -pass 1 -map 0:0 -map 0:1 -acodec aac -strict experimental stairs.mp4;
  • ffmpeg -y -i stairs.avi -vcodec libx264 -pix_fmt yuv420p -vf yadif,scale=640:480 -profile:v baseline -x264opts cabac=0:bframes=0:ref=1:weightp=0:level=30:bitrate=700:vbv_maxrate=768:vbv_bufsize=1400 -movflags faststart -ac 2 -b:a 128k -ar 44100 -r 20 -threads 2 -map_metadata -1,g:0,g -pass 2 -map 0:0 -map 0:1 -acodec aac -strict experimental -metadata title='”Stairs where i work” – lame test item, bear with us – http://archive.org/details/stairs’ -metadata year=’2004′ -metadata comment=license:’http://creativecommons.org/licenses/publicdomain/’ stairs.mp4;

Happy hacking and creating!

PS: here is the way we compile ffmpeg (we use ubuntu linux, but works on macosx, too).

new video and audio player — video multiple qualities, related videos, and more!

Posted on by

Many of you have already noticed that since the New Year, we have migrated our new “beta” player to be the primary/default player, then to be the only player.

We are excited about this new player!
It features the very latest release of jwplayer from longtailvideo.com.

Here’s some new features/improvements worth mentioning:

  • html5 is now the default — flash is a fallback option.  a final fallback option for most items is a “file download” link from the “click to play” image
  • videos have a nice new “Related Videos” pane that shows at the end of playback
  • should be much more reliable — I had previously hacked up a lot of the JS and flash from the jwplayer release version to accommodate our various wants and looks — now we use mostly the stock player with minimal JS alterations/customizations around the player.
  • better HD video and other quality options — uploaders can now offer multiple video size and bitrate qualities.  If you know how to code web playable (see my next post!) h.264 mp4 videos especially, you can upload different qualities of our source video and the viewer will have to option to pick any of them (see more on that below).
  • more consistent UI and look and feel.  The longtailvideo team *really* cleaned up and improved their UI, giving everything a clean, consistent, and aesthetically pleasing look.  Their default “skin” is also greatly improved, so we can use that now directly too
  • lots of cleaned up performance and more likely to play in more mobile, browsers, and and OS combinations under the hood.

Please give it a try!

-tracey

 

For those of you interested in trying multiple qualities, here’s a sample video showing it:

http://archive.org/details/kittehs

To make that work, I made sure that my original/source file was:

  • h.264 video
  • AAC audio
  • had the “moov atom” at the front (to allow instant playback without waiting to download entire file first) (search web for “qt-faststart” or ffmpeg’s “-movflags faststart” option, or see my next post for how we make our .mp4 here at archive.org)
  • has a > 480P style HD width/height
  • has filename ending with one of:   .HD.mov   .HD.mp4   .HD.mpeg4    .HD.m4v

When all of those are true, our system will automatically take:

  • filename.HD.mov

and create:

  • filename.mp4

that is our normal ~1000 kb/sec “derivative” video, as well as “filename.ogv”

The /details/ page will then see two playable mpeg-4 h.264 videos, and offer them both with the [HD] toggle button (seen once video is playing) allowing users to pick between the two quality levels.

If you wanted to offer a *third* quality, you could do that with another ending like above but with otherwise the same requirements.  So you could upload:

  • filename.HD.mp4       (as, say, a 960 x 540 resolution video)
  • filename.HD.mpeg4   (as, say, a 1920 x 1080 resolution video)

and the toggle would show the three options:   1080P, 540P, 480P

You can update existing items if you like, and re-derive your items, to get multiple qualities present.

Happy hacking!

 

 

 

improved h.264 derivatives!

Posted on by

We have thoroughly tested a newer and simpler way to create h.264 derivatives!

Changes you’ll notice:

  • More pixels!  previously 320 x 240    goes to 640 x 480 pixels
  • Slightly higher video bitrate — from about 512kb/s   to   about  700kb/s bitrate
  • Switching from mp4creator container maker to ffmpeg container + qt-faststart
  • Less back-end commands to make high-quality derivative

Nice things about this derivative (similar to prior derivative):

  • Plays in adobe flash plugin
  • Plays on all versions of iphone and ipad
  • Starts quickly, nearly instant seeking even to unbuffered areas of the video

Here’s a sample of how we do it with just 3 simple commands.  (We do/you should adjust “-r” argument appropriately to your video’s frames-per-second.  We also adjust the “640” in the “-vf scale” argument to be appropriate for the video’s *actual* aspect ratio, etc.  So for example, the 640 might become 852 for 16:9 widescreen video.  Although for our .mp4 specific derivative and playback ability on iPhone (1st gen and thus all versions), we would actually downrez that to 640×360).

ffmpeg -deinterlace -y -i 'camels.avi' -vcodec libx264 -fpre libx264-IA.ffpreset -vf scale=640:480 -r 20 -threads 2 -map_meta_data -1:0 -pass 1 -an tmp.mp4


ffmpeg -deinterlace -y -i 'camels.avi' -vcodec libx264 -fpre libx264-IA.ffpreset -vf scale=640:480 -r 20 -threads 2 -map_meta_data -1:0 -pass 2 -acodec aac -strict experimental -ab 128k -ac 2 -ar 44100 -metadata title='Camels at a Zoo - http://www.archive.org/details/camels' -metadata year='2004' -metadata comment=license:'http://creativecommons.org/licenses/by-nc/3.0/' tmp.mp4

qt-faststart tmp.mp4 'camels.mp4'

our preset file:
http://www.archive.org/~tracey/downloads/libx264-IA.ffpreset

 

For the adventurous out there, you can create this same setup by building ffmpeg on mac, linux, or windows.  Linux is easy, but personally, I’m a mac gal.  So here’s some ffmpeg build tips on the mac.

Happy viewing!

 

better mp4 (h.264) derivatives at archive.org!

Posted on by

Late last week, we pushed live a new video deriving technique, as well as in the process updated our audio/video file reader, ffmpeg.

New items will benefit from this newer method, and prior items can be re-derived by users if they desire (probably by the end of the year, we will rederive all our movies automatically).

The video will have significantly less “noise”, a higher PSNR (Peak Signal-to-Noise Ratio), and less”blocking” — all at similar or faster deriving speed to build and the same bitrate and filesize!

example new derivative frame

example new derivative frame

example old derivative frame

example old derivative frame

We now open the source video file up with ffmpeg, resize and convert it to raw video, and pipe it to the most recent build of “x264” tool (opting for baseline profile for iPhone, etc. compatibility).
For the very curious (and the very geeky 😉 here is a how we make our h.264 MPEG4 video files now:

• ffmpeg -i camels.avi -vn -acodec libfaac -ab 64k -ac 2 temp.aac
• ffmpeg -an -deinterlace -i camels.avi -s 320x240 -r 20 -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - 2>/dev/null | ffmpeg -an -f rawvideo -s 320x240 -r 20 -i - -f yuv4mpegpipe - 2>/dev/null | x264 --bitrate 512 --vbv-maxrate 768 --vbv-bufsize 1024 --profile baseline --pass 1 /dev/stdin --demuxer y4m -o temp.h264
• ffmpeg -an -deinterlace -i camels.avi -s 320x240 -r 20 -vcodec rawvideo -pix_fmt yuv420p -f rawvideo - 2>/dev/null | ffmpeg -an -f rawvideo -s 320x240 -r 20 -i - -f yuv4mpegpipe - 2>/dev/null | x264 --bitrate 512 --vbv-maxrate 768 --vbv-bufsize 1024 --profile baseline --pass 2 /dev/stdin --demuxer y4m -o temp.h264
• mp4creator -c temp.h264 -r 20 t2.mp4
• mp4creator -c temp.aac -interleave t2.mp4
• ffmpeg -i t2.mp4 -acodec copy -vcodec copy -metadata title="Camels at a Zoo - http://www.archive.org/details/camels" -metadata year="2004" -metadata comment="license:http://creativecommons.org/licenses/by-nc/3.0/" camels_512kb.mp4
• mp4creator -optimize camels_512kb.mp4

–Tracey Jaquith

New strategy for Internet Archive movies!

Posted on by

We have rebuilt all of our nearly 200,000 videos at the archive!

[We finished this Dec 1, 2008]

Related cross-blog with OLPC.

Here is a table-based chart of which video formats will be “derived” into which formats (we are creating 4 formats per video now):

http://www.archive.org/help/derivatives.php

Improvements and Changes from our prior movies techniques:

  • We will make a new Ogg Theora (with Vorbis audio) opensource/free-based video derivative. This derivative will play natively in Firefox 3.1 release (v3.1 is due around the end of 2008).
  • We are re/making h.264 MPEG-4 derivatives. We have updated the format to work with lighttpd “mod_h264_streaming” (which allows jumping into a movie at a specified time) but in the process will be losing the ability to serve/stream this file with RTSP.  This derivative also plays in the Adobe Flash plugin and plays on iPods/iPhones.
  • We are removing older 64kb and 256kb MPEG-4 derivatives.  With “progressive download” support becoming ubiquitous, even modems and phones are doing much better with downloading larger files.
  • We are removing older .flv “Flash Video” derivatives.  Since the much better quality h.264 derivative plays in recent flash plugins (as well as many other devices and browsers), the flash video alternative is seen as less ideal.
  • We are removing older .mpg MPEG-1 derivatives.  Their usefulness has declined in recent years, especially compared to h.264 alternatives.
  • We are remaking our animated GIFs. They attempt to make 30 thumbnails from each uploaded video.  We now evenly space them across the entire video.
  • We are remaking our Thumbnails. Similar to the GIF, we are spreading them across the videos better, and making less Thumbnails for items with *many* videos.  Additionally, we are renaming the thumbnails to indicate the second position in the video they were created at.  This will allow for the next bullet item…
  • We have developed the ability to jump into videos by clicking on the thumbnail image (to go to that scene!) We are finalizing the URL / permalinks for these “jump into video” URLs and will release this live to the public as soon as we can.

-tracey jaquith