Author Archives: Jason Scott

The Internet Archive Telethon Pt. 2

See also Parts 1 and 3.

The Internet Archive Telethon 2015 began at noon on December 19th, and almost immediately we had things go pretty crazy. The co-hosts (Jason Scott and Michelle Krasowski) and the opening staff of the Cortex got used to the timing of the whole thing, while tests and fixes kept happening to get the infrastrucure functioning.

But to get things off to a rousing start, Diva Marisa Lenhardt sang an aria from The Fifth Element and talked about how the Internet Archive’s Wayback machine saved a copy of a site she’d lost due to a disagreement with her hosting provider.

From there, we ran into issues with audio synchronization, to the strangeness of the cameras, and ultimately, a network outage. Due to the efforts of multiple employees coming onsite and online on that Saturday, the network connection returned, stronger than ever, and by 4pm, we had our act together.

Pasted image at 2015_12_21 04_31 PM

So, it turns out that a telethon is a very stressful, very weird, very involved project, and a 24 hour marathon is all of that times 10!

Luckily, the calls and e-mails resulted in some pretty amazing appearances through the day and into the night (and back into the morning). Volunteers have edited together some of the highlights (although there were many more) and they are now available in the Archive’s collections. They include:

A pretty historic segment was Brewster Kahle interviewing John Perry Barlow and John Gilmore, co-founders of the EFF and with accomplishments on all sides, doing a combination interview and conversation among the statues.

20151219_180837

Internet Archive employees played a huge part in the events and the acts onstage, including Alexis Rossi and Dwalu Khasu singing christmas carols and Jeff Kaplan performing on guitar.

pasted_image_at_2015_12_21_04_32_pm20151219_163802pasted_image_at_2015_12_21_04_30_pm

It was a time for interviews and conversations about the nature of the Archive that rarely are heard outside its walls – employees sitting back and chatting about what brought them there, kept their interest, and what inspired them.

pasted_image_at_2015_12_21_04_31_pm

Among these were:

Michelle’s late late show ran from Midnight to 8 AM Pacific Standard Time.  Along with hosting engaging interviews, she was inspired to dig deep into the Archive’s collection to feature some of our fascinating oddities, which included the horrors of candy eating, promotion of violence by Santa, and testing of toys in zero gravity:

Michelle also pulled out the 35mm filmstrip projector and slide carousel to bring us back to our days of compulsory education in the dark, and we learned about Cities in Space and The Poetry of Rock.

In the Internet Archive 2015 collection, you can see both edited sequences as well as extended unbroken clips, some of them going for hours, of the Telethon as it happened.

There were moments of great excitement, of improvisation, and of having to just make do with who or what was onstage at that exact second. Though it all, hundreds of viewers weighed in with tweets, suggestions, questions and demands.

So, in basically one day, we generated a couple dozen hours of content and media, a good portion of it unique and amazing and some of it beyond classification.

But how did the Telethon actually do with regards to its goal, to raise awareness and funding for the Archive?

That’s in Part 3.

The Internet Archive Telethon Pt. 1

See also Parts 2 and 3.

The thinking was rather simple: The Internet Archive holds a year-end fundraising drive to encourage donations and recurring donations to provide the great service it does to the world. It does this in recent times through a combination of banner ads and grants, all meant to help cover the annual $12 million budget.

But the Archive is also made up of people, and is a resource for millions not just for web pages and historical documents, but for movies, music, software and a range of other data that have been contributed by institutions and individuals over the years. It also has its headquarters in San Francisco, which has quite a diverse amount of artists, musicians and figures.

So, the logic went, why not do a telethon?

20151217_190510

The beginning of the setup – machines in the “cortex” and lighting up the stage.

The word “telethon“, a combination of “television” and “marathon”, is the idea that you do something, whatever it is, broadcast it as widely as possible, and keep doing it while asking for whatever it is you’re asking for.  Traditionally, this has been charity or non-profit organizations or causes. It’s probably most popularly heard when you flip past a public radio station and you hear a couple hosts talking about all the things that the station does across the year.

And the Archive happened to have an advantage for putting on a show that many places might not – a wonderfully inspiring Great Room, which was part of the church the headquarters building used to be, which has hosted a number of fun events in the recent past.

prettypews

So, starting in the later part of 2015, a group of people started organizing the idea of an Internet Archive Telethon – contacting possible guests, figuring out what would sit where, and what would make sense for a final date.

The in-process photos of what would become “the cortex”, the central video switching and social media tracking, show the range of equipment:

IMG_20151215_103937

The arrangement of computers, video/audio cables, encoders and switchers across the bench of a pipe organ was eventually cleaned up into something a little easier to understand. One or two people could run the three cameras (although they’d have to move them) and then a large monitor would allow the social media tracking (to answer questions and see what people were saying) as well as driving the on-screen graphics (to show donation totals, list who was talking, and send out “more to come” messages).

In all, it was very complicated, very improvised, and of course we had no idea how well it would work for its very first test, a 24-hour show.

20151217_190527

The final setup, with a much cleaner arrangement (less obvious wires!) and a big screen.

In 2014, after a power outage, a 5 hour “test telethon” happened, where we sat in the great room and talked to people over a webcam. It was just a big conversation, talking about why the Archive does what it does, and dozens of people weighed in over the course of the event before we called it a day. It proved several things: We could stream live video for hours from the Archive, we could interact with people online doing it, and we needed some class acts to perform during the show.

20151217_190559

We got all of that. We’ll talk about it in Part 2.

The Internet Archive Telethon: December 19th-20th! Tickets Available!

To spice up our end-of-year fundraising drive, a number of employees of the Internet Archive are going to be hosting a 24-hour Telethon at our 300 Funston Location!

This will not only be a livestreamed event, but in a grand experiment, a simultaneous live event happening for a lucky audience, who can attend up to the full length of the telethon and have an overnight experience in a truly unique place.

The telethon location is the Great Room, Internet Archive’s legendary meeting space and stage and home to our ceramic archivists and multiple petabytes of our content.  The fun begins at noon on Saturday, December 19th, and goes through non-stop to noon on September 20th.

prettypews

RP_few_151003_0865

Your hosts are Michelle Krasowski and Jason Scott, who will be sharing duties and shifts throughout the 24-hour marathon, introducing and interviewing guests, and answering questions and requests from the on-site and on-line audiences.

We have been hard at work arranging appearances and performances from a wide variety of folks, including musical acts like Conspiracy of Beards and Marisa Lenhardt, longtime friends of the Archive including Nuala Creed and Megan Prelinger, and surprises, strangeness and dips into the deep stacks of the Archives the whole way through!

More will be added, so be sure to check both the Eventbrite page or our pop-up site at telethon.archive.org for who else is making appearances.

A Treasure Trove of Adventure and Uploading

The Internet Archive opens its doors and drives towards a goal of Universal Access to Knowledge, and encourages contributions and uploads from our audience on a global scale. In some cases, like our Netlabels section, the contributors are often the creators as well, publishing open-licensed music at the Archive. Others, however, are doing their part to make sure near-forgotten works find a new life online.

RM_Ballantyne_The_Pirate_City_0006 Shining among these contributing curators is one Nick Hodson of Athelstane e-books, who over the course of years uploaded hundreds (more that 400, in fact) of late 19th century stories of adventure. These books, created by a wide range of authors and publishing houses, contain all manner of plot, drama and intrigue, as well as some top-quality illustration work of the time.

With titles like Twice Lost (1876), The Island Home or The Adventures of Six Young Crusoes (1851), The Voyages of the Ranger and Crusader and What Befell Their Passengers and Crews (1872) and Cutlass and Cudgel (1890), these writings range from epic stories to tales for young readers, a gamut worth traveling in itself.

Additionally, historical work has been done to give context to the books, including biographies of the authors and publishing houses. Explanations of the process of converting the items have been included as well, revealing that Hodson and his colleagues would set up voice synthesizers reading the text, to catch any errors not found other ways.

RM_Ballantyne_The_Pirate_City_0010

It’s good enough to bring to the spotlight this collection of literature, and to point out that as a library, it’s a handsome snapshot of a writing style of the past. But beyond that is the fact that the Archive’s digital storehouses are most welcoming to contributors who accompany the files with the descriptions and reasons behind their being added to the collections.

When a reader finds these stories, now over a century and a half in the past, they do so looking for the descriptions, titles and other information attached to them by the uploader. The reward comes in the view counts and reviews from a happy audience stumbling on a trove of writing that references lives long past.

The Internet Archive’s upload facility awaits quality contributions of culture and history from throughout the world, and continual improvements in search, browsing and on-line reading mean the reward for your efforts will be an ever-growing and grateful audience.

Here’s to the adventures on the high seas, and the adventures of contributors to the Internet Archive, past and future!

Experiment with One Million Album Covers

coversAs might be expected, the Internet Archive has lots of data in its virtual stacks. Besides the books, movies and stored webpages, there are datasets provided from the Internet at large or from individual contributors.

But datasets are just big clumps of data unless someone does something with them. Obviously we’re keeping these around no matter what (our current goal is “forever”), but without folks tinkering, experimenting and using the data sets, they’re just piles clogging up hard drives.

So, in the name of experimentation, we’ve put together one million album cover images from a variety of sources, and put them into this item. The total size is 148 gigabytes (!) of .JPG, .GIF and .PNG images. (There is a torrent on the item, allowing you a more flexible way to download that amount of imagery.)

The albums are somewhat-arbitrarily split according to filename, with .TAR (tape archive) files for the letter a, b, c, etc.  The goal here is experimentation – these have not been curated, overly quality checked, or any differently-sized doubles removed. If you’re writing programs or doing analysis, these are the sorts of oddness or strangeness you should be aware of.

(If you just want to play around a bit, there’s a link to a set of a mere 1200 album covers, for a total of 200 megabytes.)

We’ve included some suggestions for using the data, and some projects that might be interesting to get into, either as a hacking project or just because you’re learning computer science.

Let us know how it works for you!

 

Making Your DOS Programs Live Again at the Internet Archive

MSDOSSince the beginning of the year, the Internet Archive has been making a large amount of DOS-based games and programs run in the browser, much like our Console Living Room and Internet Arcade collections. Many thousands of people have stopped by and tried out these programs, enjoying such classics as Llamatron 2112 or Dangerous Dave. With countless examples of DOS programs going back spanning 30 years, there’s lots of great software to try out and experiment with. Here’s a great place to start.

If you want to just try out the software, we’re done here. Go into our stacks and have a great time!

However, some people have asked about adding DOS software they created or which they have which isn’t part of our collections, and especially how to make these programs boot in a window like our currently available programs do.

This is a quick guide to getting your DOS programs up and emulating in the browser. If any of these instructions are unclear to you, please contact the Software Curator at jscott@archive.org.

Please note: these instructions are for DOS programs, not Windows programs.

First, you should register for your Internet Archive library card if you haven’t already.

getcardNext, you should upload your DOS software as a .ZIP file. It is important that your program and any support files be inside a single .ZIP file and not uploaded separately.

uploadWhen you upload, you’ll be asked to fill out all sorts of information about your program. Be sure to be as complete as possible, including the description, date of creation, who the author or authors were, and so on. You’re the curator of this software – help the world understand why they should look at it!

Set the “Collection” to Community Software.

Finally, at the bottom of this upload screen, there is an add additional metadata option.

metadataAdd these two metadata pairs:

  • Set “emulator” to “dosbox”.
  • Set “emulator_ext” to “zip”.

Finally, and this is very important … inside the .ZIP file you uploaded is the program that starts the program running. It might be an .EXE, .BAT or .COM file.  For example, if your ZIP file has a single file in it, called LEMON.EXE, then that’s the program that “starts” your program.

  • Set “emulator_start” to this program.

After double-checking your work, click on “Upload and Create your Item” and the system will upload your program to the Archive, and if all goes well, your program will be emulated in our pages after a few minutes.

Again, if you have any questions or experience any issues, contact Jason Scott, the software curator at the Archive, at jscott@archive.org.

Let’s bring the DOS prompt back! And let a thousand programs bloom!

 

Mirroring the Stone Oakvalley Music Collection

soamc_logo

The Internet Archive has begun mirroring a fantastic collection of music called the “Stone Oakvalley Music Collection”. When you visit one of their websites, the archive.org mirror is one of the choices for download. Going forward, the Archive will offer a full backup of the entire site (over a terabyte) for permanent storage.

Why the Stone Oakvalley Collection is important

Manufactured from the early 1980s to the mid 1990s, the Commodore 64 computer was a revolutionary piece of hardware and a critical introduction to programming for generations. It also had, within its design, a very well-regarded sound chip: the 6581/8580 SID (Sound Interface Device), whose unique properties in wave generation and effects gave a special sound in the hands of the right developers and musicians.

MOS_Technologies_6581

 

This successful piece of hardware was manufactured in the millions across the life of the C64, and in the late 1980s, the introduction of the Commodore Amiga computer brought to life an improved chipset for generating sound; the 8364, or PAULA. With a range of improvements to what sounds and music could come out of this chip, the Amiga soared with capabilities that took years to match in other machines.

paula8364The Archive hosts many examples of music generated by these chips: our C64 Games Archive has videos in the hundreds of games played on a Commodore 64, and searching for terms like “Amiga Music”, “Chiptunes” and “C64 Music” will yield a good amount of sound to enjoy.

But nothing comes close to the Stone Oakvalley Collection in terms of breadth, dedication, and craft in ensuring the unique sound of these chips can be enjoyed in the future.

setup01

The process, which is documented here, involved setting up a large amount of Commodore hardware connected to servers which would reboot the machines, over and over, playing thousands of pieces of music in different configurations, and automatically cataloging and saving the resulting waveforms. Considerations for modifications of the chipset over the years, of stereo versus mono recordings, and verification of the resulting 400,000 files have provided the highest quality of snapshots of this period.

Browsing the Collection

Currently, there are two websites for Stone Oakvalley’s collection – one based around the C64, and the other based around the Amiga.  Impeccable work has been done to catalog the music, so if there are songs or games you remember, they are likely to be saved on the site (and powered from Archive.org’s servers). Otherwise, browse the stacks of the sites and enjoy a soundscape of computer history.

The Internet Archive strives to provide universal access to the world’s knowledge. Through mirroring, hosting and gathering of data, our mission allows millions to gain ad-free, fast access to information and materials. Be sure to check our many collections on our main site.

Inviting the Internet Over to Play

B062ql8CMAErlkJ

At our Annual Event last week, the Archive announced a variety of new projects and plans, including our new beta interface, our compact book scanner, and our progress in tracking political ads on television. The event (full video is here) went very well, with lots of activities and social gathering before and afterwards, and included the first public unveiling of our newest project, the Internet Arcade.

Photo by Kyle Way

Photo by Kyle Way

It was obvious we were on to something – the smallish room with the two stations set up to play emulated arcade games from the collection was constantly packed. Players young and old tried out classic video games, including parents showing their children games they’d played in their own teenage years. All of it was running off the Archive’s own web pages through standard web browsers, with no special plug-ins – and it held up well. We even tracked high scores.

B1F2VfECYAAHKr3

The party, of course, was just the beginning – over the weekend, we quietly announced that the Internet Arcade was available through the main site. With over 900 arcade machines in the collection, most every major machine released between 1976 and 1988 was included. (The emulation system we use, JSMESS, is a Javascript port of a long-running emulation project called MESS/MAME, which has had hundreds of contributors over the years – we salute them.)

After an initial tweet or two, the Arcade’s existence went from a mention by Waxy and Laughing Squid, to sites like Hacker News and Mashable, and from there it hit larger and larger audiences. Within a few hours news had spread to a whole range of sites, including Joystiq, The Verge, Engadget, CNN, PC World, Gizmodo, Ars Technica… and, well, let’s just say a very large amount of sites were reporting on this story.

And that’s when the world showed up.

We’re still counting, but we know hundreds of thousands of people came, many of them all at once, to play.

And as these thousands of curious visitors and first-time callers came to the Archive to try out our collection, minor inefficiencies became showstoppers and the site was temporarily crushed. Our brave administration team persevered, repairs were made, and the site settled in for the new reality:

That's a lot of new visitors!

Everything’s fine and normal… then we crash and fix things… and WOW that’s a lot of new visitors!

This crush of new visitors are coming to the Internet Archive, possibly for the first time ever, and we welcome them with open arms. After all, that’s what we were founded for –  our stated purpose is to function as the Internet’s Library, with stored websites, digitized texts, music, movies and software.  It’s our mission as a non-profit library: make as much of culture and information available to as many people as possible. You can lose a workday or a whole winter in our virtual stacks, and our users often do.

Meanwhile, the story continues to have legs, appearing in newspapers, on radio shows, video podcasts, and message boards around the world.

And then we made it to TV news:

B1hYwH-CQAAb5_f

So now that we have (apparently) the world’s attention… ahem ahem..

Even we don’t know where this story is going to lead. But one thing is sure – video games and software are as important a part of history and culture as books, movies and music have been in the past.  And we’re dedicated to bringing all of this to you, the Internet. Sure, it can be a bit surprising when the entire internet comes over to play, but we wouldn’t have put out the welcome mat if we didn’t want you to visit.

As a non-profit, we depend heavily on user donations to stay afloat – we even take Bitcoin and subscriptions. Keeping 20 petabytes of information flowing, fast and free, is what we’re working on day and night and the positive messages and feedback we’ve gotten this past week (and over the years) tell us we’re doing the right thing.

The JSMESS emulation project is one of many open-source projects the Internet Archive is involved with, and while a lot of it is fun and games we’ve got a serious side too, gathering up disappearing web resources and important historical events into our archives to preserve for next generations. We hope that after you relive your childhood or live out a second new one, you’ll stick around and see what else we have here. It’s quite a place.

Game on!

 

 

 

 

 

Free the Screenshots!

As the Archive moves more widely into the archiving of software, it quickly becomes apparent that there’s going to be an awful lot of programs online without much indication of what they are. With many thousands of programs or program collections to choose from, determining what might be inside becomes a pretty involved task.

In the case of movies, images and texts, there are previews that help show what is contained in the files in a given item. These are extremely helpful, as they not only show the quality or style of the works, but give all sorts of information that might not be reflected in the metadata.

Starting now, the same will be true for many types of software.

screenshot_01

The Atari 800 graphical masterpiece Astro Chase.

Using a combination of the JSMESS emulator and screen capturing software, the Archive has begun automatic “playing out” of sets of programs, snagging shots of what the software does, and then providing it as a guidepost of what is to come with that program.

For example, work has just been completed on the playable Sega Genesis Library,  where the directory view of the items in the collection show helpful screenshots, and individual games show animated playthroughs of the beginning of the cartridge.

00_coverscreenshotThe process is still evolving – currently it requires real-time capture (that is, capturing the first five minutes of a program takes an actual five minutes), but with multiple machines moving through collections, screenshots will be available for huge amounts of programs in coming weeks and months.

Along with the obvious graphical prettiness comes an even greater cultural benefit: the freeing of screenshots.

As these shots have often been done manually or have been gathered by hand, there has risen a tendency to put watermarks or credits with the images to indicate who did the work. While it’s an understandable urge to want some kudos for the effort, it meant that the very work being lauded (the graphics of the program) was being vandalized to ensure credit where credit was due.

None of the screenshots we are generating will have watermarks, and can be used freely for other purposes as you see fit.

To celebrate this, we’ve created a compilation of all the Sega Genesis screenshots generated by the project so far. The compilation is here. Be warned – it’s 4.3 gigabytes of 16,900 screenshots of 573 cartridges! (There’s a way to browse it at this link.)

Many screenshots are simply informative, but many more are truly works of art, as artists and programmers strained the edges of these underpowered machines to create the most evocative images possible. With this screenshotting effort underway, that work will hopefully get a new life and respect on the web.

Free the Screenshots!

screenshot_36

 

 

The Console Living Room Expands

Back on the day after Christmas, we announced the Console Living Room, a collection of console games dating from the 1970s through the 1980s that could be played right in your browser, with no plugins or installation necessary. With over 800 cartridges emulated from five game consoles, the chance to relive old memories, make new ones, and experience video game history were huge.

In the months since, there have been college courses assigned to study the old games in the Living Room, reviews written by players trying these games for the first time, and a crowd of tens of thousands of players checking the whole thing out.

So, it’s time to make the Living Room a little bigger.

As of today, the Console Living Room now supports 2,300 cartridges for 21 consoles.

Sega Game GearAmstrad GX4000Sega GenesisEpoch Super Cassette VisionAPF-MP1000Channel FSuper A'Can

It is 100% guaranteed that you have never heard of all of these consoles, even if you were playing video games at the time they were around. Some flamed out spectacularly, only creating a handful of cartridges. Others were the victim of bad timing or needless delays, making their technology significantly out of date upon release. One of them had a short lifespan and was only ever released in Taiwan.

All of them are a part of history.

These consoles (now spanning from the 1970s through to the 1990s) represent the uniqueness of the video game revolution, as living rooms were transformed from watch-only television shrines into places of activity and competition, often against the machines themselves. Only a small percentage of companies now release home consoles, investing many millions and huge armies of support/development staff to do so – these online exhibits harken back to when a comparatively small number of people could pull off what now takes many hundreds to do.

The relative stability of the market now is a quiet meadow compared to the intense battles that came before.

As for the games themselves, there’s an untold mass of creativity, triumphs, tears and near-misses throughout the thousands of cartridges. Games that should have been big but weren’t, games that make you wonder what they were thinking, and games that have taken on the patina of warm regard but… just aren’t as good as they say.

Some other notes about this expanded collection:

Adding 2,300 cartridges to the collection at once means that a significant number lack the documentation or information they deserve. A team of volunteers has been working to shore up descriptions, cover images, and screenshots for these many programs, but the work is ongoing. In a notable amount of cases, there exists very little information about the game cartridge at all. Games that sold well or had a notable brand tend to have more information available, while short-selling products fell between the cracks. If you would like to volunteer to help backfill some of these items, please contact Jason Scott, software curator, at jscott@archive.org.

The JSMESS Emulator that drives these playable games continues to be improved and the use of JavaScript means there are no plugins needed. We find the Firefox and Chrome browsers are the most responsive. Some consoles, especially later model ones, require a hefty machine to run the games at the proper speed. If you find a game is unresponsive or slow, it is very likely not your fault at all. Let us know if you find games or platforms that are acting unusually badly or are crashing.

Lastly, we continue to have no sound available on these emulations. We have experimentally proven sound works, and we are now working with multiple teams of people who are involved in emulation, the browser audio standards, and other aspects to get this dealt with. When it’s ready, we’ll announce it.