Author Archives: Roger Macdonald

71,716 video tapes in 12,094 days

On November 4, 1979 Marion Stokes began systematically video taping television news and continued for more than 33 years, until the day she died. The Internet Archive is now home to the unique 71k+ video cassette collection and is endeavoring to help make sure it is digitized and made available online to everyone, forever, for free.

Ms. Stokes was a fiercely private African American social justice champion, librarian, political radical, TV producer, feminist, Apple Computer super-fan and collector like few others. Her life and idiosyncratic passions are sensitively explored in the exceedingly well reviewed new documentary, Recorder: The Marion Stokes Project, by Matt Wolf. Having premiered last month at the 2019 Tribeca Film Festival, the film is on tour and will be featured at San Francisco’s Indefest, June 8th & 10th. For those in the Bay Area, please consider joining Internet Archive staff and leadership at the 7:00pm June 10th screening. Advance tickets are available now, seating is limited.

Long before many questioned the media’s motivations and recognized the insidious intentional spread of disinformation, Ms. Stokes was alarmed. In a private herculean effort, she took on the challenge of independently preserving the news record of her times in its most pervasive and persuasive form – TV.

Background Materials, Resources & Reviews

Input
Marion Stokes and her future husband John Stokes appear in and helped produce Input, a weekly panel discussion series on the CBS affiliate in Philadelphia that ran from 1968 through early 1971. It addressed a remarkable range of timely social topics, some far ahead of their time.  Panelists included diverse thoughtful scholars, activists, clergy and others.  Some had already made recognized accomplishments. And some would only later make their profound contributions to civil rights and social justice.

Pete Seeger was already a well known political folk singer when he appeared in February 1970 on a panel with a prison warden and recently released inmates discussing the nature of incarceration and criminal justice reform. Here he is sharing his song “Walking Down Death Row” on the program.

John Fryer was a Philadelphia psychiatrist. Here he is on Input in January 1968 discussing contradictory social norms. Five years later, Dr. Fryer would give a speech, in disguise, at the American Psychiatric Association annual convention. Introduced as Dr. Anonymous, he announced “I am a homosexual. I am a psychiatrist. I am a member of the APA,”  He went on to decry the prejudice directed toward gay people by the Association and social institutions.  Dr. Fryer’s brave and bold call for reform is credited as galvanizing his peers in 1973 to remove homosexuality from the Diagnostic and Statistical Manual of Mental Disorders.

William Davidon was professor of Physics at Haverford College. Here he is in a December 1968 Input episode, discussing the nature of television as a means of manipulating an uninformed public. 27 months later he would take an action of great social consequence and his role would remain secret for the next 43 years. In 2014, the book The Burglary posthumously revealed Dr. Davidon as the leader of a group that in 1971 broke into the FBI field office in Media, PA. They were never caught. The 8-member team stole, and released to the press, an enormous trove of documents that revealed COINTELPRO. It was the FBI’s then 15-year long covert, and often illegal, domestic surveillance program to disrupt, discredit and destroy American civil rights, anti-war and other social activist organizations and leaders.  Included in the documents was evidence of the FBI’s attempt to induce, via blackmail, Dr. Martin Luther King Jr. to commit suicide. The release of the papers lead to significant additional revelations by journalists and Congressional investigations, which prompted substantial reform.

Personal Journals
Ms. Stokes was a committed diarist, note taker and list maker. Under the leadership of archivist Jackie Jay, The Internet Archive has been digitizing the contents of 55 bankers boxes of her papers that include her personal journals, magazines, newspapers, civic organization pamphlets, leaflets and handbills. Some of her earliest (1960 & 1961) hand-written journal entries are now publicly available and can be viewed here. More will be added as they are scanned and QC’d.

Recorder: The Marion Stokes Project” Documentary Reviews & Press

Matt Wolf’s remarkable Recorder uses Stokes’ recording obsession as a way to explore both Stokes herself and the world she literally committed to video tape. The results are fascinating, weird, and often quite moving.” – Indiewire

Intriguing from first minute to last… Relating this stranger-than-fiction tale with the narrative twists and turns of a well-paced thriller, Recorder will make news junkies feel a lot better about themselves.” – Hollywood Reporter

One outstanding offering in this year’s Tribeca Film Festival is Recorder, which reveals the secret greatness of a reclusive activist… An information revolutionary, Stokes, despite her decades of isolation, touched the nerve center of the times.” – The New Yorker

Recorder is more than just a portrait of a woman’s complicated relationships and obsessions… Recorder quietly seeds damning observations about the ways media narratives are formed, and how the shapers of these narratives distort the truth and our worldview.” – Flixist

Stokes’s archival work is unprecedented; a time machine back to the advent of the 24-hour news cycle covering historical and cultural events that otherwise would have been overlooked” – The Outline

But Recorder: The Marion Stokes Project is not just — or even predominantly — an essay film about the media. What makes the documentary so fascinating is the parallel it draws between restoring an archive and retrieving a life.” – Filmmaker Magazine

…rarely do we experience the passion and purpose of a methodical collector, who really made a difference. Matt Wolf’s masterful documentary, Recorder: The Marion Stokes Project takes us into the visionary psychic and cluttered physical worlds of a woman who turned her acquiring fury into a unique archive of contemporary history.” – Helen Highly

But maybe the real value of the Marion Stokes Project is that starting close to 20 years before the digital age, it reveals how the news was going to evolve into an addiction, one that had the power to displace whatever subject it was ostensibly about. For even if you’re obsessed with the inaccuracy of TV news, it has still entrapped you, like a two-way mirror that won’t let you see the other side.” – Variety

The story of Marion Stokes inspires and challenges us to consider our world and the legacy we can create through dedication to our own ideals and principals.” – 2019 Maryland Film Festival

Recorder: The Marion Stokes Projectmanages to capsulize Stokes’ efforts and present them as a springboard for a greater conversation on the societal effects of the media, and what we can accomplish given the right resources and individual determination.” – Film Threat

Data, it is said, is the new oil. A woman named Marion Stokes knew this early on and believed that freedom was inextricably linked to data and facts because with it one can make informed decisions. So she took what is seen as a curious and radical approach to feverishly create massive archives of what used to be the prime source of such data, television news, in a time when no one else would.“ – Forbes

This story is beautifully told, inspiring, and is a constant reminder that everything we hear is not always the whole truth.” – Irish Film Critic

Much like Stokes’ archives, Recorder: The Marion Stokes Project is a cautionary reminder that, now more than ever, we need to be scrutinizing who is shaping the breaking news we consume.Cinema Axis

Marion had fought a quixotic but worthy battle against the tyranny of transience.” – New Statesman

The first ( November 2013) press article on the Marion Stokes TV Archive [note: early estimates of collection size were off by a factor of 2]
The Incredible Story Of Marion Stokes, Who Single-Handedly Taped 35 Years Of TV News – Sarah Kessler


2016 Political TV Ad Tracker: with Analysis & Fact-checking Citizens Can Trust

KNC project illustrationThe Internet Archive is honored to receive today a Knight News Challenge grant to support our collaborative efforts to help citizens make sound decisions in the 2016 U.S. elections; for the best interests of themselves, their communities and future generations.

Experts are predicting 2016 election spending will be double, or more, that of 2012. Much of that money will be spent on TV advertising. Local stations across the country will be raking in enormous sums to air these ads. But how well will the stations educate us on the issues; and offer critical analysis?  If not they, then who?

FingersCrossed
To help citizens navigate their way towards informed choices amidst the flood of political messaging, we will be building on journalism partnerships to present digital library reference pages for political ads.  Our journalism launch partners include Politifact, FactCheck.org and the Center for Public Integrity.

We will be capturing all TV programming in select 2016 primary election locales, front-loaded to reflect early-state candidate winnowing. We hope to apply lessons learned during the primaries, to key general election battleground states in the fall.  In addition to our regular TV news research library interface, we’ll be creating an online reference page for each unique-content political ad.  These pages will present journalist fact-checking and other analysis.   Accompanying these assessments will be information about ad sponsors, campaign financial transparency data as well as dynamically updated tracking on each ad’s plays, including frequency, locale, etc.  

Our 2016 Political Ad Tracker project is informed by extensive collaborative experiments conducted during the 2014 general elections in the Philadelphia-region where there were a number of hotly contested Congressional and state elections.  For more on these pilot collaborations, see Philly Political Media Watch Project and Political Ads Win Over News 45 to 1 in Philly TV News 2014.

We are continuing to refine our approaches to facilitating advanced analysis of regional inventories of television political ads.  To get a sense of the degree of their granularity, explore this interactive search visualization, created by Kalev Leetaru, derived from last year’s experiments: Philly 2014 Political Ad Trends Viewer.

Ad_FingerprintingAnother outgrowth of our political ad experiments last year was applying audio fingerprinting to algorithmically find all other instances of an ad, once a single one had been identified.  We used the audfprint tool developed by Dan Ellis at the Laboratory for the Recognition and Organization of Speech and Audio at Columbia University.

The Internet Archive and Kalev Leetaru recently took ad-finding a step further and prototyped a new way of tracking “memes” on television.  For example, everyone can now chart how the President’s 2015 State of the Union address was excerpted and discussed across U.S. and select international television over the following two weeks.  You could think of it as a TV news seismometer, tracking the propagation of key news sound bites throughout complex TV news media ecosystems, including the context in which they were presented.  We expect to apply this approach to 2016 election debates, speeches, etc.

We are humbled by the challenge of getting the word out about how our Political Ad Tracker information resources can be used.  As librarians, archivists, and technologists….market outreach is not our strength.  We’d like your help.

We are incredibly excited with the prospects of working in concert with diverse journalists, scholars and civic organizations.  Together, we hope to help balance the forces of Big Money with reason & insight, resting on sound data.  To inform and engage citizens better than ever before!

We are deeply appreciative of the Knight Foundation, Rita Allen Foundation, Democracy Fund and Hewlett Foundation for their support!

Political Ads Win Over News 45 to 1 in Philly TV News 2014

[press: Columbia Journalism Review, USA Today, BloombergPolitics, Washington Post]

Study finds 842 minutes of political Ads compared to 18.7 minutes of political news stories in large sample of Philadelphia TV news programs archived by the Internet Archive in a joint project.

In the closing eight weeks of the 2014 campaign, political candidates and outside groups bombarded viewers of Philadelphia’s major TV stations with nearly 12,000 ads designed to sway voters in the Nov. 4 elections. But the stations that benefited from political advertisers’ $14 million spending spree also appear to have devoted little time to political journalism. A study of a representative sampling of newscasts on those stations put the ratio of time devoted to political advertising and spent on substantive political news stories at 45:1.

Political Ads & Local TV News – Philly 2014, by Danilo Yanich

These are the findings of a University of Delaware team lead by Associate Professor Danilo Yanich. The university’s Center for Community Research and Service researchers collaborated with the Internet Archive, The Sunlight Foundation, and the Committee of Seventy – the 100+ year-old Philadelphia-based political watchdog organization.

Our joint pilot project, Philly Political Media Watch, worked to open a library of all television news from stations based in and around Philadelphia and index the political ads presented in their newscasts. The ads were joined with information on who paid how much for them.  The Sunlight Foundation was able to unearth those financial data from being buried in PDF disclosures every TV stations is required to submit to the Federal Communications Commission. The experimental project was supported by individual contributors and grants from the Democracy Fund and the Rita Allen Foundation.

Philly TV Market AreaThe Philadelphia television market was chosen as a 2014 laboratory to experiment how the interaction between news media and political money; to learn lessons that could be taken to scale across the nation in 2016. The Philadelphia region is the nation’s 4th largest TV market, 19% African American, and includes parts of three states. In 2014, important contests in the region included races for: Pennsylvania governor, a Delaware U.S. Senate seat, two open congressional seats in New Jersey and an open state Senate seat in suburban Philadelphia.

The six major Philadelphia metro TV stations carried 8,003 political ads in their news broadcasts between September 8 and Election Day. As Yanich’s report notes, political strategists have long acknowledged that they try to place ads during or near news programming because it attracts the highest proportion of likely voters.

Here is a sample program from the Delaware study.  This 60-minute WCAU, a NBC affiliate, program aired at 5:00pm the day before the elections.  It offered two substantive political stories.  One about election day poll hours and the other about the leading candidates for governor commenting on their attack ads.  Good set up.  Questions of incumbent elicit an unequivocal assessment of opponent’s assertions.   Followed by other candidate asked if his ads are negative.  Seemingly timely and germane.  Quiz: Can you find WCAU’s mistake followed sometime later by an unacknowledged correction?

Although WCAU clearly addressed important election issues, that same 60 minute program was also stuffed with 24 political ads.  Here is one, below.  Quiz: Can you spot the word “EBOLA”?  And for extra credit: which is more toxic to our Republic, this kind of ad or the disease?

Although local TV station marketing directors are more than happy to accommodate the needs of political ad buyers, the  local news directors appear to take a less supportive view of their audience’s interest in politics. Yanich and his research team looked at a representative sample of the news programs (390 of 1,256) and found politics taking a back seat to other types of stories in terms both of time and placement in the broadcast. The Delaware researchers found that many of the political stories aired were blandly informational, describing candidate schedules or appearances. Isolating political stories that focused on substantive political issues, Yanich’s team found that during the broadcasts they analyzed, there 18.7 minutes of those stories, compared to 842 minutes of political ads, a ratio of 45:1.

Next Steps

With so much heat, where will citizens find the light they need to navigate through this onslaught of political messaging?

Internet_Archive 2016 Political Ad TrackerThe Internet Archive has begun to welcome new collaborators to join us in tackling the challenge of creating timely information resources for the 2016 U.S. election cycles. Data individuals and civic organizations can trust when considering how to participate in some of their community’s most important decision making. Reliable information they can use to hold television stations accountable for the choices they make in balancing obligations to serve the information needs of their communities and the allure of one of their biggest revenues sources: political advertising.

How might we better inform voters and increase civic participation before, during and after elections?

 

 

Locking the Web Open, a Call for a Distributed Web

Presentation by Brewster Kahle, Internet Archive Digital Librarian at Ford Foundation NetGain gathering, — a call from 5 top foundations to think big about prospects for our digital future.  (More detailed version)


Hi, I’m Brewster Kahle, Founder of the Internet Archive. For 25 years we’ve been building this fabulous thing—the Web. I want to talk to you today about how can we Lock the Web Open.


Code=LawOne of my heroes, Larry Lessig, famously said that “Code is Law.” The way we code the Web will determine the way we live online. So we need to bake our values into our code.

Freedom of expression needs to be baked into our code. Privacy should be baked into our code. Universal access to all knowledge. But right now, those values are not embedded in the Web.


IA_serversIt turns out that the World Wide Web is very fragile. But it is huge. At the Internet Archive we collect 1 billion pages a week. We now know that Web pages only last about 100 days on average before they change or disappear. They blink on and off in their servers.


map_China_RussiaAnd the Web is massively accessible, unless you live in China. The Chinese government has blocked the Internet Archive, the New York Times, and other sites from its citizens. And so do other countries every once in a while.


Censorship_flic.kr_p_gZZRQvSo the Web is not reliableAnd the Web isn’t private. People, corporations, countries can spy on what you are reading. And they do. We now know that Wikileaks readers were targeted by the NSA and the UK’s equivalent. We, in the library world, know the value of reader privacy.


It is FunBut the Web is fun. We got one of the three things right. So we need a Web that is Reliable, Private but is still Fun. I believe it is time to take that next step. And It’s within our reach.

Imagine “Distributed Web” sites that are as functional as Word Press blogs, Wikimedia sites, or even Facebook. But How?


Tubes_flic_kr_p_89HvvdContrast the current Web to the internet—the network of pipes that the World Wide Web sits on top of. The internet was designed so that if any one piece goes out, it will still function. The internet is a truly distributed system. What we need is a Next Generation Web; a truly distributed Web.


Peer2PeerHere’s a way of thinking about it: Take the Amazon Cloud. The Amazon Cloud works by distributing your data. Moving it from computer to computer—shifting machines in case things go down, getting it closer to users, and replicating it as it is used more. That’s a great idea. What if we could make the Next Generation Web work that, but across the entire internet, like an enormous Amazon Cloud?

In part, it would be based on Peer-to-peer technology—systems that aren’t dependent on a central host or the policies of one particular country. In peer-to-peer models, those who are using the distributed Web are also providing some of the bandwidth and storage to run it.

Instead of one web server per website we would have many. The more people or organizations that are involved in the distributed Web, the safer and faster it will become. The next generation Web also needs a distributed authentication system without centralized log-in and passwords. That’s where encryption comes in.


PrivateAnd it also needs to be Private—so no one knows what you are reading. The bits will be distributed—across the Net—so no one can track you from a central portal.


 MemoryAnd this time the Web should have a memory. We’d build in a form of versioning, so the Web is archived thru time. The Web would no longer exist in a land of the perpetual present.

Plus it still needs to be Fun—malleable enough spur the imaginations of a millions of inventors. How do we know that it can work? There have been many advances since the birth of the Web in 1992.


Blockchain_JavaWe have computers that are 1000 times faster. We have JAVAScript that allows us to run sophisticated code in the browser. So now readers of the distributed web could help build it. Public key encryption is now legal, so we can use it for authentication and privacy. And we have Block Chain technology that enables the Bitcoin community to have a global database with no central point of control.


NewWebI’ve seen each of these pieces work independently, but never pulled together into a new Web. That is what I am challenging us to do.

Funders, and leaders, and visionaries– This can be a Big Deal. And it’s not being done yet! By understanding where we are headed, we can pave the path.


DistributedWebLarry Lessig’s equation was Code = Law. We could bake the First Amendment into the code of a next generation Web.

We can lock the web open.
Making openness irrevocable.
We can build this.
We can do it together.


Delivered February 11, 2015 at the Ford Foundation-hosted gathering: NetGain, Working Together for a Stronger Digital Society

Media, Money & Elections: 2014 Philly Political Media Ad Watch

Philadelphia-region Political Media Ad Watch is a pilot project that allows citizens and journalists to go online to search every political message in the Philly television market, compare all the ads from a single sponsor (sample: Tom Wolf for Governor) —positive and negative—and trace back who is paying for those ads.

She’s Dishonest!
He’s in Bed with an Accused Mobster!
This is what television audiences in Pennsylvania and Southern New Jersey are hearing a lot of this season. And it’s not Judge Judy or the Jerry Springer Show. Nope. It’s the deeply disturbing reality television show of our nation’s mid-term elections.

Dark accusations run back-to-back with heartwarming assurances of compassion.  All financed by increasingly unfettered flows of cash from ever more veiled donors.

Voters have a right to know who’s paying for these messages. And this flood of commercials begs a few critical questions for our democracy:

  • With so much heat, where can citizens find the light they need to make thoughtful choices?
  • Are the local media, many of whom make big bucks on election advertising, doing a good job giving voters the information and context they need to make sound decisions on Election Day?
  • Can we establish a baseline of metrics to evaluate the performance of local media during elections?

The project is a collaboration between the Internet ArchiveSunlight Foundation, Philadelphia’s Committee of Seventy (a non-partisan government watchdog), University of Delaware’s Center for Community Research & Service and the Linguistic Data Consortium at the University of Pennsylvania. It immediately enables local media to do a better job sifting between fact and fiction in political messaging and revealing financial sources of political influence.

In the coming year, University of Delaware researchers will sift project data to answer some basic questions about how local media is serving the public:

  • To what extent, if any, do local television news broadcasts examine the claims that are made in the political ads that appear on the newscasts?
  • Do the broadcasts cover the same issues that are the subject of political ads? If so, which issues are covered, which issues are not covered?
  • How much time is devoted to that coverage? Where does that coverage appear in the newscasts?

And in the long term, our pioneering work in the Philadelphia-region will help us create an affordable and technically scalable model to answer these questions in local markets nationwide leading up to the 2016 elections.

One of the exciting features of this project is that it brings cutting edge technology together with campaign finance expertise and grassroots good-government advocates in Philadelphia to potentially provide vastly greater understanding on who funds our political system and how they influence campaigns on the ground. Each of these organizations by themselves have a strong potential impact—together, we have the ability to amplify the rich, revealing information that can move voters and sway debate toward better outcomes.

What We’re Doing

The Internet Archive is recording, indexing for search and presenting online Philadelphia TV Market Area television news—which includes 22 counties in Pennsylvania and southern New Jersey; indexing for search all political ads therein; creating an interface for trained volunteers to identify and tag political advertising; joining indexed ads with sponsor information databases; making news and ads searchable, quotable and embeddable; capturing and presenting, in a full-text searchable database, much of the region’s Web media ecosystem..

The Sunlight Foundation is training volunteer political ad sponsorship coders, creating adaptations of the Influence Explorer interface and database to include real time Pennsylvania state campaign data; developing specialized optical character recognition algorithms for extracting Public Inspection File disclosures on sponsorship for TV political ad buys on its Political Ad Sleuth database; conducting outreach to journalists and others for their collaboration and use of resources for stories; integrating ad sponsor data into related Sunlight Foundation data tools and API’s; working with the Internet Archive to sync up sponsorship data with the actual ads in the same interface.

The Committee of Seventy is organizing a team of volunteers; acting as liaison with Philadelphia-region civic organizations; conducting outreach to area press; and providing guidance on issues and political candidates to track.

The University of Delaware’s Center for Community Research & Service at the School of Public Policy & Administration will conduct an analysis of the broadcast news programs in the Philadelphia television market, aired September 1 through Election Day, November 4.  After Election Day, the University team will conduct content analysis to address the research questions above and publish findings next year.

The Linguistic Data Consortium at the University of Pennsylvania is providing technical support and advice regarding the Internet Archive’s broadcast monitoring in the Philadelphia area.

Project Resources

View all identified political TV ads
• Watch video tour guide to using Philly-region TV news search
Search just Philadelphia content from the TV News Archive
Philadelphia stations’ political ad sponsor reports to FCC
Archived Philadelphia web media ecosystem sites (key word searchable)

Project Advisors

Kathleen Hall Jamieson, the Elizabeth Ware Packard Professor of Communication at the Annenberg School for Communication; and Walter and Leonore Annenberg Director of the Annenberg Public Policy Center at the University of Pennsylvania.

Travis N. Ridout, the Thomas S. Foley Distinguished Professor of Government and Public Policy and Associate Professor in the school of Politics, Philosophy and Public at Washington State University; and co-director of the Wesleyan Media Project.

David Westin, former president of ABC News, Founding CEO of NewsRight, a digital start-up spun off from the AP; and now Principal of Witherbee Holdings, LLC

Supported in part by grants and other contributions from:

David Glassco
Democracy Fund
Rita Allen Foundation
Hawthorn Family Fund
Buck Foundation (NYC)
Kahle/Austin Foundation
John S. and James L. Knight Foundation
Philadelphia Foundation, from an anonymous contributor to their donor-advised funds

Project Collaborator Contacts

Internet Archive – Roger Macdonald  roger@archive.org
Sunlight Foundation – Kathy Kiely  kkiely@sunlightfoundation.com
University of Delaware – Danilo Yanich  dyanich@udel.edu
Committee of Seventy – Ellen Kaplan  ekaplan@seventy.org
Linguistic Data Consortium – Denise DiPersio  dipersio@ldc.upenn.edu

 

 

 

 

1 Year Later: NSA Revelations, Debate and Dire Prospects

Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. Benjamin Franklin, November 15, 1755

A year ago today, Glenn Greenwald published the first article on the extent of NSA surveillance, based on documents provided by whistleblower Edward Snowden.

Prior warnings by members of the US Congress, whistleblowers and others had gone un-headed. Effective Congressional oversight was circumvented by secret Executive Branch interpretations of relevant laws.

After the June 5th revelation last year, most members of Congress were shocked, even the author of the Patriot Act.

Members of the U.S. House of Representatives recently attempted to end mass government surveillance of Americans via the USA FREEDOM Act. Some initial supporters, like House Judiciary Committee member, Zoe Lofgren, withdrew their support for the Act after amendments modified its effectiveness. The Electronic Frontier Foundation called the Act “gutted”.

The USA FREEDOM Act moves to the Senate for debate and likely further modification. The public debate over the relationship between freedom and safety in our increasingly digital world continues to deepen.

Edward Snowden frames the choices between Liberty and Security confronting Americans today as dire as Ben Franklin did in 1755.

NSA-issues TV News Quote Library

Our experimental library presents more than 1,100 chronologically ordered television citations drawn from the Internet Archive’s television news research library. TV quotes can be browsed by rolling over clip thumbnails, queried via transcripts and sorted for specific speakers. Citations, context, source broadcasters, and options to share, quote or borrow can be explored by following links on each thumbnail.

Thanks to the exceptional curatorial efforts of Robin Chin, a media researcher for the Internet Archive, you can use this library to reflect upon one of the great issues of our times.

Peering at a Dark Time in Tiananmen Square Through Stokes Archive Glass

Twenty five years ago Tiananmen Square was splashed with the blood of protestors. For seven weeks they had banded together to advocate a democratic future for the Peoples Republic of China. Their voices were stilled June 4, 1989 by the guns of China’s army. The protestors’ optimistic vision of reform was reflected later that year in the fall of the Berlin Wall and has persisted in China to this day.


In respectful remembrance of the terrible sacrifices exacted in Tiananmen Square, and to inform thoughtful reflection, we offer a few glimpses gleaned from the Marion Stokes Archive of how U.S. media told the story.

Marion Stokes, an African American librarian and social justice advocate, dedicated the last thirty five years of her life to recording television news so that we might consider the past through the lens of contemporaneous media. Her devotion resulted in an extraordinary collection of 40,000 video cassettes. We are in the very early stages of beginning to index the collection and experiment with digitizing it.

Let Our Video Go

MetMuseunScroll_DT11631UI / UX Advances in Freeing Information Enslaved by an Ancient Egyptian Model  Or… Why Video Scrolling is so Last Millenniums

In creating an open digital research library of television news, we have been challenged by being unable to reference a current user experience model for searching video. Conventional video search requires users to start at the beginning of video and proceed at the pace and sequencing dictated by content creators. Our service has vaulted over the confines of the linear video storytelling framework by helping users jump into content at points directly pertaining to their search.  But by doing so, we have left some of our prospective users adrift, without a conceptual template to rely on.  That is until this April, with the release of a new user interface.

GutenbergPress_LoC

Treating video as infinitely addressable data is enabling us to do an increasingly better job at getting researchers right to their points of interest. While revolutionary in its application to television news at the scale we are doing it, it does have an antecedent in a prior media revolution — the transition from the age of scrolls to printed books. Gutenberg used movable type to print identical bibles in the mid-1400’s. It took a hundred more years before detailed indexes started appearing at the end of books. The repurposing of closed captioning to facilitate deep search of video is, in some ways, as significant for television as the evolution from parchment and papyrus rolls to page numbered and indexed books.

The value of most major innovations can only be realized when people adapt their conceptual models to understand and use them. Our interface design challenge included helping users make a perceptual leap from a video experience akin to ancient Egyptians unfurling scrolls to that of library-literate modern readers, or the even more recent experience of being able to find specific Web “pages” via search engines.

SprocketScroll

Our latest interface version helps users cross the cognitive bridge from video “scrolling” through television programs to accessing them instead as digitally indexed “books” with each page comprised of 60-second video segments. We convey this visually by joining the video segments with filmstrip sprocket border graphics. Linear, like film, but also “paginated” for leaping from one search-related segment to another.

ScreenShot3

When searching inside individual broadcasts, the new interface reinforces that metaphor of content hopping by truncating presentation of interleaving media irrelevant to the search query. We present the search-relevant video segments, while still conveying the relative “distance” between each jump — again referencing the less efficient linear “scroll” experience that most still find more familiar.

ScreenShot4

The new UI has another revolutionary aspect that also hearkens back to one of the great byproducts of the library index model: serendipitous discovery of adjacent knowledge. Dan Cohen, founding Executive Director of the Digital Public Library of America recently recounted, “I know a professor who was hit on the head by a book falling off a shelf as he reached for a different one; that book ended up being a key part of his future work.”

Segments

When using the new “search within” a single program feature, the browser dynamically refines the results with each character typed. As typing proceeds towards the final search term, unexpected 60-second segments and phrases arise, providing serendipitous, yet systematic choices, even while options narrow towards the intended results. These surprising occurrences suggest the diverse opportunities for inquiry afforded by the unique research library and encourage some playful exploration.

Carter_10The Internet Archive is still in the early stages of helping guide online television out of its imprisonment in ancient conceptual frameworks. A bright future awaits knowledge seekers and content creators alike when digital video is optimized for systematic discovery of even short segments. New audiences and new use-cases will be joined with media that has been languishing for too long in digital tombs, mostly unseen and unheard.

At its heart, the Internet Archive is an invitation to explore and collaborate. Please, join us in evolving digital opportunities to open knowledge for the benefit of all.

Start by giving our service a whirl, find something important and quote it.  I just did – https://twitter.com/r_macdonald/status/463492832867516416

Mapping 400,000 Hours of U.S. TV News

TVnewMap2
We are excited to unveil a couple experimental data-driven visualizations that literally map 400,000 hours of U.S. television news. One of our collaborating scholars, Kalev Leetaru, applied “fulltext geocoding” software to our entire television news research service collection. These algorithms scan the closed captioning of each broadcast looking for any mention of a location anywhere in the world, disambiguate them using the surrounding discussion (Springfield, Illinois vs Springfield, Massachusetts), and ultimately map each location. The resulting CartoDB visualizations provide what we believe is one of the first large-scale glimpses of the geography of American television news, beginning to reveal which areas receive outsized attention and which are neglected.

Watch4-year

 

Watch TV news mentions of places throughout the world for each day.

 

Compare-Contrast

 

Select a TV station and time window to view their representations of places.

 

Keep in mind that as you explore, zoom-in and click the locations in these pilot maps, you are going to find a lot of errors. Those range from errors in the underlying closed captioning (“two Paris of shoes”) to locations that are paired with onscreen information (a mention of “Springfield” while displaying a map of Massachusetts on the screen). Thus, as you click around, you’re going to find that some locations work great, while others have a lot more error, especially small towns with common names.

What you see here represents our very first experiment with revealing the geography of television news and required bringing together a bunch of cutting-edge technologies that are still very much active areas of research. While there is still lots of work to be done, we think this represents a tremendously exciting prototype for new ways of interacting with the world’s information by organizing it geographically and putting it on a map where it belongs!

Virtual Machines: Unlocking Media for Research

In addition to our public web-based research service, we are facilitating scholars, like Kalev, and other researchers in applying advanced data treatments to our entire collection, at a speed and scale beyond any individual’s capacity. As responsible custodians of an enormous collection of television news content created by others, we endeavor to secure their work within the context of our library. Therefore, rather than lending out copies of large portions of the collection for study, researchers instead work in our “virtual reading room” where they may run their computer algorithms on our servers within the physical confines of the Archive. We hope our evolving demonstrations of this data queries in — results out — process may help forge a new model for how exceptional public interest value can be derived from media without challenging their value and integrity to their creators.

The Knight Foundation and other insightful donors are providing critical support in our ongoing efforts to open television news and join with others in re-visioning how digital libraries can respectfully address the educational potential of other diverse media. We hope you will consider lending your support.

The Atlantic

A Dream to Preserve TV News, on the Road to Realization… with Your Help

On The Media’s TLDR
CNN
Huffington Post
Philly.com
Fast Company

We are about to receive a remarkable private collection of video taped U.S. television news that spans 35 years.  We welcome contributions of TV news recorded before the year 2001 to help broaden our research library.

M_Stokes4Marion Marguerite Stokes, a librarian, social justice advocate and TV interview program host, believed that it was vital to preserve television news.

Mrs. Stokes started recording news at home in 1977 — and never stopped. Before her death in December 2012 she recorded 140,000 video cassettes. Her family searched for a home for her unique collection and found us in June.

It is a unique collection of local news from Boston (1977-1986) and Philadelphia (1986-2012), as well as all the national news. The Boston era is particularly notable for the busing/desegregation strife that raged throughout.

Marion Stokes’ amazing commitment to preserve television news, a passion that few at the time entirely understood, shaped the daily lives of her children growing up and, later, visits of her grandchildren. Her dream of using this collection for the public good can now be fulfilled.

In just a few days, four large shipping containers on trucks will be winding their way across the country to our Richmond, California physical archive. The digitization of such a huge collection will take a number of years and funding we have yet to raise.

Join us in helping to realize Marion Stokes’ gift to the future and make it available to all, forever, for free.  Please consider making a contribution, right now!