Audio and Video improvements

Posted on by

Hi Patrons,

Yesterday we made live a large update to the way we create the audio and video displayed on our site.  Most folks might not notice the changes, so here’s a rundown:

For Audio:

  • We can now read and make mp3s from 24-bit Flac files. This has been requested for many years and we are thrilled to get it working.
  • The Ogg audio files that we create from audio files will now be using an updated “libvorbis” library.  (The library we were using before today was from 2001! 😎 )
  • We are no longer making 64kb MP3s (or zips or m3u playlists of those files).  This was a judgement call — given how poor the sound quality is for these files and the fact that most people are getting more and more bandwidth to their devices and computers.
  • Simplified back-end system, relying more and more on “ffmpeg” for format conversion.
  • We will now (try to) make derivatives from “.aac” (Advanced Audio Coding) files and “.ra”/”.rm” (Real Audio) files.
  • General ability to read more kinds of audio files more reliably.

For Movies:

  • The Ogg Video files that we create from movies files will now be using an updated “libvorbis” library for their audio.  (Previously we were using the “non reference” library ogg encoder.  Now we are using the much asked for and newer “libvorbis” library).
  • Updated ffmpeg to v0.5. This allows for a much wider range of source audio/video containers and codecs.  We will be able to derive HD-quality video formats like DV-50 and DV-100.   (For those interested in ffmpeg, changelog).
  • Better detection of widescreen movies (so less of our movies on our site will incorrectly appear “squooshed”).
  • General ability to read more kinds of video files more reliably.
  • Noting the prior point, we were able to get streaming videos for about 170 TV archive items that we could not process previously.

Enjoy!

–Tracey Jaquith

Bookmark and Share

49 thoughts on “Audio and Video improvements”

  1. Mike

    Always good to see things moving forward!

    Will you be deleting the old, derived 64kb MP3s? FWIW, I hope you don’t–I have directly linked to hundreds of them in various blog posts, and I’d hate to see those links break. (Or to try to track them all down.) I’d bet others are in the same position.

    In any case, thanks for all the good work.

  2. Michael

    Great to see the new derivatives up and running 🙂 Big props to Tracey 😉

    To respond to Mike’s comment (you should always link to the virtual url) … maybe archive.org could promote thous virtual urls better 🙂

  3. internetarchive

    Mike:
    We will keep existing 64kb mp3 derivatives (and zips and m3us) around — sorry that I forgot to mention it. In the unlikely event we ever do remove them much later on down the line, we’d probably adopt a “dynamic 404 handler” which would automatically serve the higher quality VBR mp3 if a 64kb derivative was requested and didn’t exist. However, we have no plans as of now to remove them from existing items! 😎

    Michael:
    (wave! 😎 )
    Yes we should try to get a better documented set of pages that explain our permalinks and show our broadening JSON API (that will hopefully become the predominant API into the archive for folks wanting metadata and/or the ability to “reskin” our site or sets of items, etc.)

    -Tracey Jaquith

  5. Andrew

    Nice news, just as I’m planning to sort audio uploads for videogame related material…

    Better video stuff is great too, I have to go back and retest some entries I’ve uploaded which failed to derive now 🙂

    Documentation for me would be a plus. Somewhere it should say what versions of what input/output are used (this is great! but not really on the IA site and I am sure it’ll be updated again at some point 🙂 ), since an archive really needs to show it’s open in how the backend works – and allows uploaders to make sure at least one file they upload derives correctly for the various derivatives that are pretty necessary. 🙂

    Now I can start to do some local testing – not with the same script or OS, but at least the right input/output versions to test to see what files work, yay! 😀

  6. NameCast

    I have about 100 audio podcast posted on my website.
    There is no Player at all.Do i have to change/update each of those post again ?help1

  9. woo

    Good to know about the 64k mp3’s going away. FYI, the next update to my app (ooTunes Radio) supports playing live concerts (mp3 only for now) from Archive.org either by clicking on the .m3u playlist or with a cool new browser “play” button which pulls out the mp3 links and as much metadata as it can and creates an on the fly playlist with the audio. I wish the metadata was more accessible than it is with some sort of standard format (or at least including the duration and track title in the .m3u file) but it’s already a big hit with some of my beta testers. Is there a place I can let Archive.org users know about this (it isn’t a free app unfortunately, but it’s a swiss army knife of streaming for $4)
    Let me know either by emailing me directly or posting here
    Thanks for all the work you guys do! My kids will thank you as well!

  11. internetarchive

    @woo:
    try getting the /details/ page data as JSON! eg:
    http://www.archive.org/details.php?identifier=GrowingYounger&output=json&callback=something

    @George:
    oh, i guess our anti-malicious description cleaning must be stripping out the new [video] tag (i gather you have tried it already on an item of yours at archive.org and it was removed?) that’s a good idea — i’ll add it to the “to do” list…

    @NameCast:
    if your existing podcasts were using archive.org and working, there should be no change. however *new* items added to archive.org won’t make the 64kb mp3s. so if you have some kind of automated thing expecting the 64kb mp3s, you would want to adjust it. if that’s a big problem and you really want 64kb mp3s, reply here and we can see if there’s anything we can do.

    -Tracey Jaquith

  12. ishaq abdulwhab

    dear manager
    how are you to day are well
    my comments is the new mmetho. and the beautiful way of learning. so I wanat to learning from you.
    my is sudan
    the english is very difficult.so I need from you to allowed to me audio, vedio, and books
    thank-you from ishaq abdulwhab
    my e-mail ishaq744@yahoo.com

  13. C.Leander

    A couple of questions about recent changes:

    1) I used to be able to access streaming Quicktime clips via RTSP, which was faster than progressive download.
    This appears to have been changed. Is there another new way to view via RTSP

    2) Also, there used to be a Flash “.flv” version of some of the Quicktime video clips, that made it easy to embed on a blog or webpage. However some links have been inactivated. Is there a way to access FLV clips? Is there another way to embed video clips?

    Thanks for continuing to provide this service.

  14. Drew

    I’ve been recording live shows in CAF format (via Apple’s Logic software) and have conversions to aif format – both produce rather large files. Are either of these formats uploadable to Achive?

  16. kim

    I’ve been uploading mp3’s and I’m wondering how to begin a collection on IA so that the mp3’s related to my podcasts are all together. Thanks in advance.

  19. sinuose

    Hi
    i have a problem with my own archives, but with the last archives. The problem is that until one month ago i could listened and downloaded my own archives, but now i can’t. I can download archives of other people, but not mines. The other people can download the same archives that i can’t, why? please help me i don’t understand why.
    thanks

  21. MF Foster

    I cannot upload audio-the button just produces a spinning circle and will not function-ideas? Thank you-MF Foster

    1. Internet Archive staff

      Hi,

      We’re unable to reproduce the problem, trying with both Mac and Windows machines, using IE, Safari, or Firefox browsers.

      It may be that only the first part of the process is getting actuated. Please try clicking the share button again, even through it may feel like you’re clicking it twice.

      If you’re still having problems, please provide a description with as much detail as possible, including what exactly is happening, when in the process it’s happening, and details of your computer operating system and browser versions tried.

      If there’s any URLs from attempted uploads, please include that also, or any other information about the attempted upload. The more detail you can provide, the better chance we have for troubleshooting.

      Thank you!

  22. computersystemdownfoundjackdoemumnnniffjibbhibbtibbstacyfacenaceygacelacehauu22ui2ugeaniejimjinquinn10timbbn

    jimmie lem mama lennie known know matt may gay tay lay frettie fray tay lay may gay tay happy kay lame nane nammie tammie gampt kampt gampnppt ommomnn16 8020girl deirrr2uitttommmuitiminnnobbbyouuu16emmetttepppr got thif this wiff flo florine fling wit this mam man named jz so zoe frit frtiz so low log hoe low mean meen teen quees babbiet cabbiet harriet soak hoke broke tippier sippier solo solar wippier tippie stiffier ne no my myra mya gap hap nap gettia min dell doe toe hoe bro so no goauutauutauuafredgrrauutipierr2 banked bankers camammiepptieppterr gimmie jimmie geanie gee hee shee free tee weempie frit towe toe ling grow grauuauuauauuatiptayeyyelleauu pick pickie picked ray stay lay grey day dippie day kay away fly thems grims dimmers dims lion cowlers cows a whipped stipped stickie lip cinne noe mim men no gone go hole holy un holy warrei222kiyyykiptippie cat eye row row row row ling ting bing bing ding doe ling around if u found jims james around aroof a toof a down 4th 5th 6th 7th yards downs stepping up nows

  31. Event Technology Guy

    I really appreciate you writing this article. When you go back and look, it is truly amazing how far we have come in terms of technology. Audio and Video improvements alike have gone through tremendous change over the many years. Great read!

    1. internetarchive

      hi John,
      i just uploaded several mp3s with no problems. if you continue to have problems please send more specific information such as the collection you are trying to upload to.
      thanks. -jeff kaplan

  33. Byronious

    The new audio player is 640 pixels wide, which is to wide for my Blog post – I tried editing the embed code however my edits didn’t work..

    Any idea on a work around?

    Muchos grasias,

    Byronious

  34. iraqchooseslife

    Nowadays this
    Must be the picture and sound stronger than the previous
    Because of new technologies that make every day in this area
    I am a big fan of voice net and free of errors
    Greetings to all

  36. Master Erace

    Yeah I remember doing the conversion of a few hundred speaker cassette tapes into the only converter I could get at the time, which was that DREADFUL Real Media PRODUCER, that wasn’t bad in the quality of encoding, it was just almost NOTHING in the way of portable audio players, used the format.

    But after doing all the conversions to digital, I had disposed of all the tapes…. Bummer.

    AND it was at this point that I became aware of the EVILNESS of the REAL MEDIA formats and their attitude toward consumers, “NO I don’t want to install, this, that, these things, those things, and a number of other things”.

    And they rigged their software to go right ahead and install everything I specifically cited, that I did not want installed, and they effectively overran my PC with their software.

    BAD moves.

    Real Media – were focused on proprietary lock in and using underhanded tactics, at my expense, to achieve it. Kind of like getting a simple savings account and then finding it linked to a super high interest rate credit card – and getting ripped off when withdrawing YOUR OWN money….. that’s right – not their money, your own money, at their credit card interest rate.

    That kind of an underhanded ethical scam…

    A few years down the track along came some FREE (?) software that enabled me to DECONVERT from the RM, RAM, RA – Real Media format/s, and into MP3 format.

    I had to make an awful lot of data fit onto small drives etc., so I understand everything to do with high compression ratios – but there comes a point where the compression is so high, and the audio quality is so low, that some basic principles have to be implemented.

    Stupidity, very high compression and really poor quality audio – is to no ones advantage. It’s like scumming on ink so badly, by thinning it out so much and then spraying down so little – that no one can read the “ghost font”.

    That is where the true advantages of compression is lost.

    File compression (in audio) has it’s origins in the days of floppy disk drives, very small hard drives, slow motherboards, low strength processors, very expensive low spec RAM and low capacity internet speeds – meets the coming tidal wave of digitisation of audio and video.

    Now for many people there is NO advantage for minimalist quality audio and video, in fact it’s much better to have reasonably good compression and very good sound quality.

    So while I have done very good combinations of 56kpbs and sampling rate audio and come up with high clarity and just a fractional tinniness…… I am not sure that it’s a brilliant move, but because real sound quality from high compression – needs to be done on the basis of sound quality first and file size second.

    This becomes an issue when one has hundreds of thousands of audio files….. but rather than abandon the high compression levels, I’d be looking at the quality of the encoding rates.

    If you can achieve near identical sound quality for the entire 50 years of the hour long weekly Roy Rogers Cowboy Radio Show, at about 30% of the file size, as compared to just slack 128Kbps with a low sampling rate, I’d got the 56Kbps…with a higher sampling rate.

    High quality compression isn’t just the compression – it’s the settings of the compression engine that make a huge amount of difference in the quality of the finished product.

    Before you write off the high compression formats – experiment with the settings of the high compression levels, such as the sampling rate.

    I think I used 56Kbps audio quality compression level – at a 44 or a 64 or 96Khz sampling rate. Been a long time since I did it….

    I got sanely small file sizes and very good quality audio – especially good on speaker tapes…. and radio quality broadcasting.

    Then it was dor

  37. John Poyner

    This is the Archive news I suppose. The Archive Forum is on fire with streaming player complaints. The item webpage source sheets say the java-scripting in my InternetExplorer browser is not enabled, though I know it is. It may not be correct in WindowsMediaPlayer though(recent update screwup). WMP did stream the items coincidentally. GoogleChrome is my 2nd browser now, not the default though. GoogleChrome accesses the Archive and streams the IAplayer pretty good.

  40. Ror

    when I scroll to the share button it turns into the spinning circle but never allows me to upload any files. I’ve enabled javascript, but still no luck. Any suggestions?

  41. Max Reiner

    “Night Beat” cuts out. Please fix the sources and re-load! Thanks! It is a nice program. But it cuts out every few seconds.

  42. Aaron Kornblum

    How come it is no longer possible to download batch audio files that work? I now have to download ever song in a group individually instead of getting a zip file that contains multiple songs? And when I do get a batch/zip file to download, most of the time they do not have all of the songs/tracks listed. That is, it will download 4 songs but not the ten or more that I see are listed. Many thanks for all that you do!

    1. Hank Bromley

      @Aaron
      We fixed a bug just yesterday that had caused the audio-zipping feature to have been broken for about a week. If you’re still having the problem you describe, please post a specific URL here that fails for you, and I’ll look into it. Thanks.

      Hank Bromley
      Internet Archive

Comments are closed.