Tag Archives: Joostware

TV News Record: Recognizing Trump’s voice on TV, NYT & Axios coverage, + Ryan fact-check

A round up on what’s happening at the TV News Archive by Katie Dahl and Nancy Watzman.

This week we explore cutting edge work by Joostware that moves us closer to solving the challenge of searching vast archives of video by speaker, note the use of TV News Archive data by The New York Times and Axios, and share a fact-checked interview by exiting House Speaker Paul Ryan about his legacy.

Joostware trained model to recognize Trump’s voice

What if you wanted to search the TV News Archive to find every instance where President Donald Trump is talking?

That’s the research question that the San Francisco-based firm Joostware concentrated on for its Who Said What project, which won a $50,000 prototype grant from the John S. and James L. Knight Foundation. Last week Joostware’s founder, Delip Rao, presented the project’s progress at a gathering in Austin, Texas. (The Internet Archive’s own Dan Schultz, in his Bad Idea Factory incarnation, also presented on Contextubot, which we recently profiled here.)

Audio and video today is viewed as an opaque object and it’s meant for linear consumption,” Rao said in his presentation. “But truly any audio and video especially in the context of news has a lot of structure to it. There are speakers of interest, and these speakers take turns, and then within each turn something was communicated. So our goal is to identify these speakers who are of interest and also the content that was spoken in that turn and indexing that.

Anyone can search the TV News Archive already via closed captions at the Internet Archive or via Television Explorer. Our experiments with facial detection and chyron extraction are another way to find and analyze news clips. But searching a video archive by “speaker id” – finding all the video where a person is actually talking – is a tough technical challenge. Our Trump Archive and congressional, executive branch, and administration archives are all manually curated video collections designed to demonstrate what it would be like to have automated speaker id search.

Joostware researchers have made progress toward this goal. They took material from the Trump Archive, and used it to train a model that recognizes the president’s voice, by using properties of the voice signal. They created a prototype search software that is more than 95% accurate on a human annotated dataset in returning video clips where Trump is actually speaking.

What’s next? With more resources, Joostware hopes to give this technology back to the Internet Archive to improve search within the TV News Archive. And Rao and others continue to work within the larger community of researchers working to crack the code of video to help fact-checkers and journalists hold power accountable.

No one is talking about tax law on cable TV news

Jim Tankersley and Karl Russell, reporters for The New York Times, used TV News Archive captions via GDELT’s Television Explorer to demonstrate how little coverage there is on cable TV news for the newly minted $2.5 trillion tax overhaul:

“Consider one of Mr. Trump’s preferred yardsticks: cable news coverage. Throughout the fall, as Republicans rushed their tax bill through Congress in two breakneck months, CNN, Fox News and MSNBC routinely devoted 10 percent of their daily coverage to tax issues, according to data from the Gdelt Project. Interest spiked as Mr. Trump signed the bill in late December, and then it fell precipitously.”

“Stormy Daniels wins TV war: overshadows taxes, health care”

For Axios, Caitlin Owens used TV New Archive data with GDELT’s Television Explorer to shed light on whether the TV networks are paying attention the priorities of the political parties: “Tax cuts and the Affordable Care Act are supposed to be big issues in the midterm elections, but both have faded from the attention of the cable news networks now that they’re no longer front and center in Congress.” Owens thinks it matters because “Democrats are campaigning hard on the GOP’s unpopular attempt to repeal and replace the ACA, and Republicans are pushing the financial benefits of their tax law.”


Fact-Check: Corporate tax revenues are rising (misleading)

House Speaker Paul Ryan, R., Wisc., announced last week he would not be seeking reelection, prompting television interviews that reflected on his legacy. In a “Meet the Press” interview Sunday on NBC, host Chuck Todd asked Ryan to respond to a statement by Sen. Bob Corker, R., Tenn.:

“’This Congress and this administration likely will go down as one of the most fiscally irresponsible administrations and Congresses that we ever had.’ And he’s referring to the fact that this tax bill spiked the deficit. It’s higher than even what was projected.” Ryan responded “That was going to happen. The baby boomers’ retiring was going to do that. These deficit trillion-dollar projections have been out there for a long, long time. Why? Because of mandatory spending, which we call entitlements. Discretionary spending under the CBO baseline is going up about $300 billion over the next 10 years. Tax revenues are still rising. Income tax revenues are still rising. Corporate income tax revenues. Corporate rate got dropped 40 percent, still rising.”

Eugene Kiely reported for FactCheck.org that “Ryan is right that $1 trillion deficit projections ‘have been out there for a long, long time…But corporate tax revenues are down for the first six months of the fiscal year, and they are projected to be less over the next 10 years than they otherwise would have been because of the law.”

Salvador Rizzo and Meg Kelly reported for The Washington Post’s Fact Checker, “The baby-boom generation is retiring, and Congress at best has taken only modest steps to rein in spending on old-age programs, largely because any serious effort is met with hostility and often-misleading attack ads…But the revenue side of the picture cannot be ignored.” “Congress has not been able to grapple with the spending — and  keeps taking steps to undermine the revenue flow as well.”

Follow us @tvnewsarchive, and subscribe to our biweekly newsletter here.

Knight Prototype Fund winners use TV News Archive to fight misinformation

We are delighted to report that two of  20 winners recently announced Knight Prototype Fund’s $1 million challenge to combat misinformation will directly draw on the TV News Archive.

One of these is the Bad Idea Factory’s “Glorious ContextuBot.” Let’s say you come across a tweet that is brandishing a TV news clip to bolster a strong statement on a controversial issue – but you have no idea where or when or whether that clip was aired, by whom, and whether or not it’s legitimate.

The ContextuBot will take the Duplitron 5000, an audio fingerprinting tool developed to track political ads for the Political TV Ad Archive, and build upon it to help users find any relevant TV news coverage of that video snippet. If the video was aired, users will be able to see what came before and after the report.

The team, led by the Dan Schultz, senior creative technologist of the TV News Archive, will bring in veteran media innovators Mark Boas and Laurian Gridnoc of Hyperaudio and Trint. Together they will not only build the ContextuBot, but will also work to improve the speed and accuracy of the Duplitron.

From sticky notes to Glorious Contextubot: on right, Dan Schultz, TV News Archive senior creative technologist, plots prototype plans

Second, Joostware‘s “Who Said What” project will use deep learning algorithms to annotate TV news clips to identify speakers and what they are talking about. Developing this capability will help fact checkers sort through and identify claims by public officials, pundits, and others that bear examination.

We are delighted to work with Joostware as part of our ongoing goal to collaborate with researchers, companies, and others who are exploring how to use Artificial Intelligence tools to draw on the Internet Archive’s collections to enhance journalism and research.

The Knight Prototype Fund will award $50,000 apiece to each of the 20 winners, who are now charged with developing their ideas over the next nine months. The John S. and James L. Knight Foundation, the Democracy Fund, and the Rita Allen Foundation all support the effort. Winners attended a human-centered design training workshop last week in Phoenix, Arizona, as part of the support offered by the foundations.