Tag Archives: Duke Reporters’ Lab

TV News Record: State of the Union, past and future

A round up on what’s happening at the TV News Archive by Katie Dahl and Nancy Watzman. Additional research by Robin Chin.

When President Donald Trump takes the podium to deliver his first official State of the Union address to a joint session of Congress on Tuesday, January 30,  he’ll be following in the footsteps of the nation’s very first president, George Washington, long before there was cable TV or radio.

In Washington’s time, the speech was not yet known as the State of the Union, but the annual message, and according to Donald Ritchie, former U.S. Senate historian, seen here in a clip from C-Span, the practice was “to [physically] cut the State of the Union message up into paragraphs and create committees to address each one of the issues the president suggested.” There were no standing committees in Congress at that time. Now it’s fact-checkers who examine the speech, line by line, and since 2017, we’ve been annotating our TV news programs with fact-checks of Trump, top administration officials, and the four top congressional leaders, Democrat and Republican.

Make history by being a beta tester of FactStream, a new free app for iPhone or iPad, which will deliver live fact-checks of Trump’s State of the Union address from national fact-checking organizations. The app is a product of Duke Reporters Lab Tech & Check collective, of which the Internet Archive’s TV News Archive is a member. We’ll be adding the fact-checks to the TV News Archive, too.

At the TV News Archive, we’ve got historical footage of some past State of the Union addresses, listed below. Last year we annotated Trump’s address to Congress – not officially a State of the Union, since he was newly inaugurated – with fact-checks from our fact-checking partners, FactCheck.org, PolitiFact, and The Washington Post’s Fact Checker. Fact-checks are noted with a red check mark on the TV News Archive filmstrip screen.

For example, the above segment of Trump’s 2017 speech, marked with a red check mark, was fact-checked by both PolitiFact and The Washington Post’s Fact Checker. Trump said, “According to data provided by the Department of Justice, the vast majority of individuals convicted of terrorism and terrorism related offenses since 9/11 came here from outside of our country.”

PolitiFact’s Miriam Valverde rated this claim as “mostly false”: “Trump’s statement contains an element of truth but ignores critical facts that would give a different impression. We rate it Mostly False.” Michelle Ye Hee Lee, writing for The Washington Post’s Fact Checker gave the claim “four Pinocchios,” stating it relied on “a grossly exaggerated misuse of federal data.”

Past State of the Union addresses

2016: Barack Obama

2015: Barack Obama

2014: Barack Obama

2013: Barack Obama

2012: Barack Obama

2011: Barack Obama

2010: Barack Obama

1995: Bill Clinton

1988: Ronald Reagan (no closed captioning)

1980: Jimmy Carter  (no closed captioning)

1975:  Gerald Ford

1969: Lyndon Johnson

1965: Lyndon Johnson

1963: John F. Kennedy (no closed captioning)

1961: John F. Kennedy (no closed captioning)

1942: Franklin D. Roosevelt (no closed captioning)

History is happening, and we’re not just watching

  1. Which recent hurricane got the least amount of attention from TV news broadcasters?
    1. Irma
    2. Maria
    3. Harvey
  2. Thomas Jefferson said, “Government that governs least governs best.”
    1. True
    2. False
  3. Mitch McConnell shows up most on which cable TV news channel?
    1. CNN
    2. Fox News
    3. MSNBC

Answers at end of post.

The Internet Archive’s TV News Archive, our constantly growing online, free library of TV news broadcasts, contains 1.4 million shows, some dating back to 2009, searchable by closed captioning. History is happening, and we preserve how broadcast news filters it to us, the audience, whether it’s through CNN’s Jake Tapper, Fox’s Bill O’Reilly, MSNBC’s Rachel Maddow or others. This archive becomes a rich resource for journalists, academics, and the general public to explore the biases embedded in news coverage and to hold public officials accountable.

Last October we wrote how the Internet Archive’s TV News Archive was “hacking the election,” then 13 days away. In the year since, we’ve been applying our experience using machine learning to track political ads and TV news coverage in the 2016 elections to experiment with new collaborations and tools to create more ways to analyze the news.

Helping fact-checkers

Since we launched our Trump Archive in January 2017, and followed in August with the four congressional leaders, Democrat and Republican, as well as key executive branch figures, we’ve collected some 4,534 hours of curated programming and more than 1,300 fact-checks of material on subjects ranging from immigration to the environment to elections.

 

The 1,340 fact-checks–and counting–represent a subset of the work of partners FactCheck.orgPolitiFact and The Washington Post’s Fact Checker, as we link only to fact-checks that correspond to statements that appear on TV news. Most of the fact-checks–524–come from PolitiFact; 492 are by FactCheck.org, and 324 from The Washington Post’s Fact Checker.

We’re also proud to be part of the Duke Reporter’s Lab’s new Tech & Check collaborative, where we’re working with journalists and computer scientists to develop ways to automate parts of the fact-checking process.  For example, we’re creating processes to help identify important factual claims within TV news broadcasts to help guide fact-checkers where to concentrate their efforts. The initiative received $1.2 million from the John S. and James L. Knight Foundation, the Facebook Journalism Project and the Craig Newmark Foundation.

See the TrumpUS Congress, and executive branch archives and collected fact-checks.

TV News Kitchen

We’re collaborating with data scientists, private companies and nonprofit organizations, journalists, and others to cook up new experiments available in our TV News Kitchen, providing new ways to analyze TV news content and understand ourselves.

Dan Schultz, our senior creative technologist, worked with the start-up Matroid to develop Face-o-Matic, which tracks faces of selected high level elected officials on major TV cable news channels: CNN, Fox News, MSNBC, and BBC News. The underlying data are available for download here. Unlike caption-based searches, Face-o-Matic uses facial recognition algorithms to recognize individuals on TV news screens. It is sensitive enough to catch this tiny, dark image of House Minority Leader Nancy Pelosi, D., Calif., within a graphic, and this quick flash of Senate Minority Leader Chuck Schumer, D., N.Y., and Senate Majority Leader Mitch McConnell, R., Ky.

The work of TV Architect Tracey Jaquith, our Third Eye project scans the lower thirds of TV screens, using OCR, or optical character recognition, to turn these fleeting missives into downloadable data ripe for analysis. Launched in September 2017, Third Eye tracks BBC News, CNN, Fox News, and MSNBC, and collected more than four million chyrons captured in just over two weeks, and counting.

Download Third Eye data. API and TSV options available.

Follow Third Eye on Twitter.

Vox news reporter Alvin Chang used the Third Eye chyron data to report how Fox News paid less attention to Hurricane Maria’s destruction in Puerto Rico than it did to Hurricanes Irma and Harvey, which battered Florida and Texas. Chang’s work followed a similar piece by Dhrumil Mehta for FiveThirtyEight, which used Television Explorer, a tool developed by data scientist Kalev Leetaru to search and visualize closed captioning on the TV News Archive.

 

FiveThirtyEight used TV News Archive captions to create this look at how cable networks covered recent hurricanes.

CNN’s Brian Stelter followed up with a similar analysis on “Reliable Sources” October 1.

We’re also working with academics who are using our tools to unlock new insights. For example, Schultz and Jaquith are working with Bryce Dietrich at the University of Iowa to apply the Duplitron, the audiofingerprinting tool that fueled our political ad airing data, to analyze floor speeches of members of Congress. The study identifies which floor speeches were aired on cable news programs and explores the reasons why those particular clips were selected for airing. A draft of the paper was presented in the 2017 Polinfomatics Workshop in Seattle and will begin review for publication in the coming months.

What’s next? Our plans include making more than a million hours of TV news available to researchers from both private and public institutions via a digital public library branch of the Internet Archive’s TV News Archive. These branches would be housed in computing environments, where networked computers provide the processing power needed to analyze large amounts of data. Researchers will be able to conduct their own experiments using machine learning to extract metadata from TV news. Such metadata could include, for example, speaker identification–a way to identify not just when a speaker appears on a screen, but when she or he is talking. Metadata generated through these experiments would then be used to enrich the TV News Archive, so that any member of the public could do increasingly sophisticated searches.

Going global

We live in an interdependent world, but we often lack understanding about how other cultures perceive us. Collecting global TV could open a new window for journalists and researchers seeking to understand how political and policy messages are reported and spread across the globe. The same tools we’ve developed to track political ads, faces, chyrons, and captions can help us put news coverage from around the globe into perspective.

We’re beginning work to expand our TV collection to include more channels from around the globe. We’ve added the BBC and recently began collecting Deutsche Welle from Germany and the English-language Al Jazeera. We’re talking to potential partners and developing strategy about where it’s important to collect TV and how we can do so efficiently.

History is happening, but we’re not just watching. We’re collecting, making it accessible, and working with others to find new ways to understand it. Stay tuned. Email us at tvnews@archive.org. Follow us @tvnewsarchive, and subscribe to our weekly newsletter here.

Answer Key

  1. b. (See: “The Media Really Has Neglected Puerto Rico,” FiveThirtyEight.
  2. b. False. (See: Vice President Mike Pence statement and linked PolitiFact fact-check.)
  3. c. MSNBC. (See: Face-O-Matic blog post.)

Members of the TV News Archive team: Roger Macdonald, director; Robin Chin, Katie Dahl, Tracey Jaquith, Dan Schultz, and Nancy Watzman.

Three takeaways after logging 1,032 political ads in the primaries

The Political TV Ad Archive launched on January 22, 2016, with the goal of archiving airings of political ads across 20 local broadcast markets in nine key primary states and embedding fact checks and source checks of those ads by our journalism partners. We’re now wrapping up this first phase of the project, and are preparing for the second, where we’ll fundraise so we can apply the same approach to political ads in key 2016 general election battleground states.

But first: here are some takeaways from our collection after logging 1,032 ads. Of those ads, we captured 263 airing at least 100 times apiece, for a total all together of more than 145,000 airings.

1. Only a small number of ads earned “Pants on Fire!” or “Four Pinocchio” fact checking ratings. Just four ads received the worst ratings possible from our fact-checking partners.

Donald Trump’s campaign won the only “Pants on Fire” rating awarded by fact checking partner PolitiFact for a campaign ad: “Trump’s television ad purports to show Mexicans swarming over ‘our southern border.’ However, the footage used to support this point actually shows African migrants streaming over a border fence between Morocco and the Spanish enclave of Melilla, more than 5,000 miles away,” wrote PolitiFact reporters C. Eugene Emery Jr. and Louis Jacobson in early January, when Trump released the ad, his very first paid ad of the campaign. The ad aired more than 1,800 times, most heavily in the early primary states of Iowa and New Hampshire.

Trump also won a “four Pinocchio” rating from the Washington Post’s Fact Checker for this ad which charges John Kasich of helping “Wall Street predator Lehman Brothers destroy the world economy.” “[I]t’s preposterous and simply not credible to say Kasich, as one managing director out of 700, in a firm of 25,000, “helped” the firm “destroy the world economy,” wrote reporter Michelle Ye Hee Lee.

Two other ads received the “four Pinocchio” rating from the Washington Post’s Fact Checker. This one, from Ted Cruz’s campaign, claims that Marco Rubio supported an immigration plan that would have given President Obama the authority to admit Syrian refugees, including ISIS terrorists. “[T]his statement is simply bizarre,” wrote Glenn Kessler. “With or without the Senate immigration bill, Obama had the authority to admit refugees, from any country, under the Refugee Act of 1980, as long as they are refugees and are admissible….What does ISIS have to do with it? Nothing. Terrorists are not admissible under the laws of the United States.”

This one, from Conservative Solutions PAC, the super PAC supporting Rubio, claims that there was only one “Republican helpful” who had “actually done something” to dismantle the Affordable Care Act, by inserting a provision preventing protection for insurance companies from losses if they didn’t do accurate estimates on the premiums in first three years of the law. “Rubio goes way too far in claiming credit here,” wrote Kessler. “He raised initial concerns about the risk-corridor provision, but the winning legislative strategy was executed by other lawmakers.”

Overall, our fact-checking and journalism partners—the Center for Responsive Politics, the Center for Public IntegrityFactCheck.org, PolitiFact, and the Washington Post’s Fact Checker—wrote 57 fact- and source-checks of 50 ads sponsored by presidential campaigns and outside groups. (The American Press Institute and Duke Reporters’ Lab, also partners, provided training and tools for journalists fact checking ads.)

Of the 25 fact checks done by PolitiFact, 60 percent of the ads earned “Half True,” “Mostly True,” and “True” ratings, with the remainder earning “Mostly False,” “False,” and “Pants on Fire” ratings. The Washington Post’s Fact Checker, the other fact-checking group that uses ratings, fact-checked 11 ads. Of these, seven earned ratings of three or four Pinocchios. A series of ads featuring former employees and students denouncing Trump University, from a “dark money” group that doesn’t disclose its donors, earned the coveted “Geppetto Checkmark” for accuracy. Those ads aired widely in Florida and Ohio leading up to the primaries there.

The ad that produced the most fact checks and source checks was this one from the very same group, the American Future Fund, for an attack ad on John Kasich. Robert Farley of FactCheck.org wrote, “An ad from a conservative group attacks Ohio Gov. John Kasich as an ‘Obama Republican,’ and misleadingly claims his budget ‘raised taxes by billions, hitting businesses hard and the middle class even harder.'” PolitiFact Ohio reporter Nadia Pflaum gave the ad a “False” rating; Michelle Ye Hee Lee of the Washington Post’s Fact Checker awarded it “Three Pinocchios.” The Center for Public Integrity described the American Future Fund as “a conservative nonprofit linked to the billionaire brothers Charles and David Koch that since 2010 has inundated federal and state races with tens of millions of dollars.”

This ad from Donald Trump’s campaign earned a “Pants on Fire” rating from PolitiFact.

2. Super Campaign Dodger, and other creative ways to experience and analyze political ads. Journalists did some serious digging into the downloadable metadata the Political TV Ad Archive provides here to analyze trends in presidential ad campaigns.

The Economist mashed up data about airings in Iowa and New Hampshire with polling data and asked the question: Does political advertising work? The answer—”a bit of MEH” (or, “minimal-effects hypothesis”)—in other words, voters are persuaded, but just the littlest bit.

Farai Chideya of FiveThirtyEight and Kate Stohr of Fusion delved into data on anti-Trump ads airing ahead of the Florida primary—which Trump went on to win handily, despite the onslaught.

Nick Niedzwiadek plumbed the collection when writing about political ad gaffes for The Wall Street Journal. Nadja Popovich of The Guardian graphed Bernie Sanders’s surge in ad airings in Nevada, ahead of the contest there.

William La Jeunesse of Fox News reported on negative ads here. Philip Bump of The Washington Post used gifs to illustrate just how painful it was to be a TV-watching voter in South Carolina in the lead up to the primary there.

And in what was the most interactive use of the project’s metadata, Andrew McGill, a senior associate editor for The Atlantic, created an old-style video game, where the viewer uses the space key on a computer keyboard to try to dodge all the ads that aired on Iowa airwaves ahead of the caucuses there. For links to other journalists’ uses of the Political TV Ad Archive, click here.

via GIPHY
3. Candidates’ campaigns dominated; super PACs favored candidates who failed. In our collection, candidates’ official campaigns sponsored the most ad airings—63 percent. Super PACs accounted for another 27 percent, and nonprofit groups, often called “dark money” groups because they do not disclose their donors, accounted for nine percent of ad airings.

Bernie Sanders‘ and Hillary Clinton‘s campaigns had the most ad airings—29,347 and 26,891 respectively. Of the GOP candidates, who faced a more divided competition, it was Marco Rubio’s campaign that had the most airings—11,798—and Donald Trump was second, with 9,590. However, in the Republican field, super PACs played a much bigger role, particularly those advocating for candidates who have since pulled out of the race. Conservative Solutions PAC, the super PAC that supported Marco Rubio in his candidacy, showed 12,851 airings; Right to Rise, which supported Jeb Bush, had 12,543.

This pair of issue ads sponsored by the AARP (aka the American Association of Retired People), aired at least 9,653 times; the ads focus on social security and have been broadcast across the markets monitored by the Political TV Ad Archive.

The biggest non-news shows that featured political ads were “Jeopardy!,” “Live With Kelly and Michael,” and “Wheel of Fortune.” Fusion did an analysis that showed that the most popular entertainment shows targeted by presidential candidates and mashed it up with Nielsen data about viewership. For example, Bernie Sanders’ campaign favored “Jimmy Kimmel Live,” while Hillary Clinton’s campaign likes “The Ellen Degeneres Show.”

 

Screenshot 2016-03-04 13.50.08

The Political TV Ad Archive–which is a project of the Internet Archive’s TV News Archive–is now conducting a thorough review of this project, which was funded by a grant from the Knight News Challenge, an initiative of the John S. and James L. Knight Foundation. The Challenge is a joint effort of the Rita Allen Foundation, the Democracy Fund, and the Hewlett Foundation.

Stay tuned for news of the Political TV Ad Archive’s plans for covering future primaries in California, New York, and Pennsylvania, and beyond, our fundraising for the second phase of this project: fundraising to track ads in key battleground states in the general elections.

This post is cross posted at the Political TV Ad Archive.