Tag Archives: Paul Ryan

TV News Record: Six takeaways from adding Hillary Clinton, Barack Obama & more to Face-o-Matic facial detection

A round up on what’s happening at the TV News Archive by Katie Dahl and Nancy Watzman.

This week we release new data generated by our Face-o-Matic tool, developed in collaboration with Matroid, adding to our list of public figures detected by facial-recognition on major cable news stations on the  TV News Archive.

In addition to President Donald Trump and the four congressional leaders, the expanded list now includes most former living presidents and recent major party presidential contenders, including Hillary Clinton and Barack Obama. (For the full list of public officials tracked, as well as methodical notes, see bottom of the post.)

Detecting faces on TV news and turning them into data provides a new quantitative path for journalists and researchers to explore how news is presented to the public and compare and contrast editorial choices that individual networks make. This new measure shows us the duration that politicians’ faces are actually shown on screen, whether it’s a clip of that person speaking, muted footage, or a still photo shown in the background to illustrate a point.

Adding to the Television Explorer, fueled by closed captions and our Third Eye chyron reading tool, a wealth of information is now available to analyze. (See the TV News Archive home page for examples of visualizations created by journalists and researchers using TV News Archive data.)

Here are six quick takeaways using Face-o-Matic for an analysis covering roughly six months, from November 2017 through May 2018, looking at four cable TV news networks: BBC News, CNN, Fox News, and MSNBC.

Download Face-o-Matic data to explore your own research questions.

1. Trump trumps every other political figure in face-time on cable TV news, all the time, every day, in every way, on every network and program.

As we’ve seen in past analyses with Face-o-Matic data, President Donald Trump is the major political star on cable TV news as compared to other top political figures examined. To put this in perspective: over a six month period stretching from November 2017 to May 2018, the president’s face appeared on TV cable news the equivalent of a full 13.5 days, counting every second of face-time. The next closest political figure we analyzed was House Speaker Paul Ryan, R., Wis., whose visage appeared the equivalent of one day.


  1. After Trump, GOP leaders in Congress are the most popular faces on TV cable news.

The two GOP leaders in Congress, Ryan and Senate Majority Leader Mitch McConnell, R., Ky. are the next most popular faces on TV news cable news networks. Between the two, Ryan ranks first on the TV news cable networks we examined: BBC News, CNN, Fox News, and MSNBC.  McConnell is the next most shown face on these networks, with the exception of BBC News.

Link to interactive version of above chart, where view can be changed to exclude specific politicians.

  1. Hillary Clinton and Barack Obama figure prominently on Fox News.

Fox News airs proportionately more images of failed presidential candidate 2016 Hillary Clinton and former president Barack Obama than other cable TV news networks. Fox News showed Clinton’s face 7.6 times more than CNN did, and Obama’s 3.6 times more. Fox News also showed Clinton 3.6 times more than MSNBC, and Obama, 2.3 times more.


  1. Hannity shows more Hillary Clinton face-time than any other top-rated Fox News show.

Not only does the Fox News “Hannity” program air more images of Hillary Clinton proportionately than any other top rated Fox News show, with just one exception, it is the Fox News show that shows her face more than current congressional leaders–Ryan, McConnell, Schumer or Pelosi. “Hannity” also shows more images of Obama than other top rated Fox News shows.

Link to interactive version of above chart, where view can be changed to exclude specific politicians.

  1. Ryan face-time spikes on news shows aired during morning hours.

All three U.S. cable news networks examined showed high rates of face-time for Ryan on shows airing during morning hours, ranging from 9 am to 11 am. This may be linked to his leadership role in Congress and that morning hours are prime for large announcements. For example, on Fox News’ “America’s Newsroom” and “Happening Now” show spikes of face-time for Ryan. On MSNBC, “Live with Hallie Jackson” and “Live with Velshi and Ruhle” show high rates of images for Ryan. And on CNN, “At This Hour with Kate Bolduan” shows high rates of Ryan as well. 

Links to interactive charts for top-rated news shows; view can be adjusted to exclude specific politicians. The source for top-rated shows is shows with 2017 top viewership by Nielsen.

Top-rated Fox News shows.

Top-rated MSNBC news shows.

Top-rated CNN shows.

  1. BBC News just isn’t that into us.

BBC News provides a window into how news is presented to a major foreign audience. Like U.S. cable news networks, BBC News features more face-time for Trump than other political figures examined. Ryan ranks a distant second. Overall, BBC News, however, shows much lower rates of images of U.S. political figures than U.S. cable news shows do.

Link to interactive version of above chart, where view can be changed to exclude specific politicians.

Methodological notes

The Face-o-Matic data set, available for download on the Internet Archive, uses facial recognition to track the faces of prominent public officials as they appear on major cable TV news networks: BBC News, CNN, Fox News, and MSNBC. The list of public officials tracked, along with the date that detection began, is here:

President & current congressional leaders

President Donald Trump, 7/13/17

Speaker Paul Ryan, R., Wis., 7/13/17

House Minority Leader Nancy Pelosi, D., Calif., 7/13/17

Senate Majority Leader Mitch McConnell, R., Ky., 7/13/17

Senate Minority Leader, Chuck Schumer, D., N.Y., 7/13/17

Former living presidents and recent major party presidential candidates*

George H.W. Bush, 10/5/17

George W. Bush, 11/1/17

Jimmy Carter, 10/21/17

Bill Clinton, 9/12/17

Hillary Clinton, 9/12/17

Barack Obama, 7/13/17

Mitt Romney, 10/4/17

*Note: Our data set does not include Sen. John McCain, R., Ariz., who ran for president opposite Obama in 2008. Sample testing of facial detection for the senator revealed a somewhat frequent rate of false positives  – instances where the identified face was not the senator’s, but rather one of a number of lookalikes. While we make no claim that all of the detections in the Face-o-matic data set are error free, we did test faces to minimize these. Please be sure to notify us if you find errors in the data.

TV News Record: Recognizing Trump’s voice on TV, NYT & Axios coverage, + Ryan fact-check

A round up on what’s happening at the TV News Archive by Katie Dahl and Nancy Watzman.

This week we explore cutting edge work by Joostware that moves us closer to solving the challenge of searching vast archives of video by speaker, note the use of TV News Archive data by The New York Times and Axios, and share a fact-checked interview by exiting House Speaker Paul Ryan about his legacy.

Joostware trained model to recognize Trump’s voice

What if you wanted to search the TV News Archive to find every instance where President Donald Trump is talking?

That’s the research question that the San Francisco-based firm Joostware concentrated on for its Who Said What project, which won a $50,000 prototype grant from the John S. and James L. Knight Foundation. Last week Joostware’s founder, Delip Rao, presented the project’s progress at a gathering in Austin, Texas. (The Internet Archive’s own Dan Schultz, in his Bad Idea Factory incarnation, also presented on Contextubot, which we recently profiled here.)

Audio and video today is viewed as an opaque object and it’s meant for linear consumption,” Rao said in his presentation. “But truly any audio and video especially in the context of news has a lot of structure to it. There are speakers of interest, and these speakers take turns, and then within each turn something was communicated. So our goal is to identify these speakers who are of interest and also the content that was spoken in that turn and indexing that.

Anyone can search the TV News Archive already via closed captions at the Internet Archive or via Television Explorer. Our experiments with facial detection and chyron extraction are another way to find and analyze news clips. But searching a video archive by “speaker id” – finding all the video where a person is actually talking – is a tough technical challenge. Our Trump Archive and congressional, executive branch, and administration archives are all manually curated video collections designed to demonstrate what it would be like to have automated speaker id search.

Joostware researchers have made progress toward this goal. They took material from the Trump Archive, and used it to train a model that recognizes the president’s voice, by using properties of the voice signal. They created a prototype search software that is more than 95% accurate on a human annotated dataset in returning video clips where Trump is actually speaking.

What’s next? With more resources, Joostware hopes to give this technology back to the Internet Archive to improve search within the TV News Archive. And Rao and others continue to work within the larger community of researchers working to crack the code of video to help fact-checkers and journalists hold power accountable.

No one is talking about tax law on cable TV news

Jim Tankersley and Karl Russell, reporters for The New York Times, used TV News Archive captions via GDELT’s Television Explorer to demonstrate how little coverage there is on cable TV news for the newly minted $2.5 trillion tax overhaul:

“Consider one of Mr. Trump’s preferred yardsticks: cable news coverage. Throughout the fall, as Republicans rushed their tax bill through Congress in two breakneck months, CNN, Fox News and MSNBC routinely devoted 10 percent of their daily coverage to tax issues, according to data from the Gdelt Project. Interest spiked as Mr. Trump signed the bill in late December, and then it fell precipitously.”

“Stormy Daniels wins TV war: overshadows taxes, health care”

For Axios, Caitlin Owens used TV New Archive data with GDELT’s Television Explorer to shed light on whether the TV networks are paying attention the priorities of the political parties: “Tax cuts and the Affordable Care Act are supposed to be big issues in the midterm elections, but both have faded from the attention of the cable news networks now that they’re no longer front and center in Congress.” Owens thinks it matters because “Democrats are campaigning hard on the GOP’s unpopular attempt to repeal and replace the ACA, and Republicans are pushing the financial benefits of their tax law.”


Fact-Check: Corporate tax revenues are rising (misleading)

House Speaker Paul Ryan, R., Wisc., announced last week he would not be seeking reelection, prompting television interviews that reflected on his legacy. In a “Meet the Press” interview Sunday on NBC, host Chuck Todd asked Ryan to respond to a statement by Sen. Bob Corker, R., Tenn.:

“’This Congress and this administration likely will go down as one of the most fiscally irresponsible administrations and Congresses that we ever had.’ And he’s referring to the fact that this tax bill spiked the deficit. It’s higher than even what was projected.” Ryan responded “That was going to happen. The baby boomers’ retiring was going to do that. These deficit trillion-dollar projections have been out there for a long, long time. Why? Because of mandatory spending, which we call entitlements. Discretionary spending under the CBO baseline is going up about $300 billion over the next 10 years. Tax revenues are still rising. Income tax revenues are still rising. Corporate income tax revenues. Corporate rate got dropped 40 percent, still rising.”

Eugene Kiely reported for FactCheck.org that “Ryan is right that $1 trillion deficit projections ‘have been out there for a long, long time…But corporate tax revenues are down for the first six months of the fiscal year, and they are projected to be less over the next 10 years than they otherwise would have been because of the law.”

Salvador Rizzo and Meg Kelly reported for The Washington Post’s Fact Checker, “The baby-boom generation is retiring, and Congress at best has taken only modest steps to rein in spending on old-age programs, largely because any serious effort is met with hostility and often-misleading attack ads…But the revenue side of the picture cannot be ignored.” “Congress has not been able to grapple with the spending — and  keeps taking steps to undermine the revenue flow as well.”

Follow us @tvnewsarchive, and subscribe to our biweekly newsletter here.

TV News Record: Whoops, they said it again (on taxes)

A biweekly round up on what’s happening at the TV News Archive by Katie Dahl and Nancy Watzman

This week, we demonstrate GOP and Democrat talking points on taxes; display a case of mistaken facial identity; and present fact checks on the GOP tax proposal.

Whoops, they said it again

Was that House Minority Leader Nancy Pelosi, D., Calif., who said “tax cuts for the rich”? Or was that Senate Minority Leader Chuck Schumer, D., N.Y.? Wait: they both said it. Often.

Meanwhile, House Speaker Paul Ryan, R., Wis., keeps talking about tax reform being a “once in a generation opportunity,” and, coincidence!, so does Sen. Majority Leader Mitch McConnell, R., Ky. It’s a recurring theme.

These types of repeated phrases, often vetted via communication staff, are known as “talking points,” and it’s the way politicians, lobbyists, and other denizens of the nation’s capital sell policy. The TV News Archive is working toward the goal of applying artificial intelligence (AI) to our free, online library of TV news to help ferret out talking points so we can better understand how political messages are crafted and disseminated.

For now, we don’t have an automated way to identify such repeated phrases from the thousands of hours of television news coverage. However, searching within our curated archives of top political leaders can provide a quick way to check for a phrase you think you’re hearing often. Visit archive.org/tv to find our Trump archive, executive branch archive, and congressional archives, click into an archive, then search for the phrase within that archive.

Sample search results in the congressional archive

Funny, you look familiar

Wait, is this former President George W. Bush trying out a new look?

No, it’s not. This is Bob Massi, a legal analyst for Fox Business News and host of “Bob Massi is the Property Man.”  In a test run of new faces for our Face-o-Matic facial detection tool, Massi’s uncanny resemblance (minus the hair) to the former president earned him a “false positive” – the algorithm identified this appearance as Bush incorrectly.

This doesn’t get us too worried, as we still include human testers and editors in our secret sauce: we’ll retrain our algorithm to disregard photos of Massi in the TV news stream. It does point toward why we want to be very careful, particularly with facial recognition, where a private individual may be tracked inadvertently or a public official misrepresented. Our concern about developing ethical practices with facial recognition is why, for the present, we are restricting our face-finding to elected officials. We invite discussion with the greater community about ethical practices in applying AI to the TV News Archive at tvnews@archive.org.

In our current Face-o-Matic set we track the faces of President Donald Trump and the four congressional leaders in their TV news appearances. After receiving feedback from journalists and researchers, our next set will include living ex-presidents and recent major presidential party nominees: Jimmy Carter, Bill Clinton, George H.W. Bush, George W. Bush,  Barack Obama, Hillary Clinton, John McCain, and Mitt Romney. Stay tuned, while we fine tune our model.

Fact-check: everyone will get a tax cut (false)

In an interview on November 7, on Fox News’s new “The Ingraham Angle,” House Speaker Paul Ryan, R., Wis., says: “Everyone enjoys a tax cut all across the board.”

Pulling in information from the Tax Policy Center and a tax model created by the American Enterprise Institute, The Washington Post’s Fact Checker Glenn Kessler counters Ryan’s claim: “In the case of married families with children — whom Republicans are assiduously wooing as beneficiaries of their plan — about 40 percent are estimated to receive tax hikes by 2027, even if the provisions are retained.”

Ryan changed his language, according to Kessler, following an inquiry on November 8 from the Fact Checker. Now he is saying, “the average taxpayer in all income levels gets a tax cut.”

Fact-check: tax bill not being scored by CBO as is tradition (false)

In an interview on November 12 on CNN’s “State of the Union,” Senate Minority Whip Dick Durbin, D., Ill., claimed that the GOP tax plan is “not being scored by the Congressional Budget Office, as it is traditionally. It’s because it doesn’t add up.”

“Under the most obvious interpretation of that statement, Durbin is incorrect. The nonpartisan analysis for tax bills is actually a task handled by the Joint Committee on Taxation, and the committee has been actively analyzing the Republican tax bills,” reported Louis Jacobson of PolitiFact.

Follow us @tvnewsarchive, and subscribe to our biweekly newsletter here.

TV News Record: 1,340 fact checks collected and counting

A weekly round up on what’s happening and what we’re seeing at the TV News Archive by Katie Dahl and Nancy Watzman. Additional research by Robin Chin.

In an era when social media algorithms skew what people see online, the Internet Archive TV News Archive’s collections of on-the-record statements by top political figures serves  as a powerful model for how preservation can provide a deep resource for who really said what, when, and where.

Since we launched our Trump Archive in January 2017, and followed in August with the four congressional leaders, Democrat and Republican, as well as key executive branch figures, we’ve collected some 4,534 hours of curated programming and more than 1,300 fact-checks of material on subjects ranging from immigration to the environment to elections.

The 1,340 fact-checks–and counting–represent a subset of the work of partners FactCheck.org, PolitiFact and The Washington Post’s Fact Checker, as we link only to fact-checks that correspond to statements that appear on TV news. Most of the fact-checks–524–come from PolitiFact; 492 are by FactCheck.org, and 324 from The Washington Post’s Fact Checker.

As a library, we’re dedicated to providing a record – sometimes literally, as in the case of 78s! – that can help researchers, journalists, and the public find trustworthy sources for our collective history. These clip collections, along with fact-checks, now largely hand-curated, provide a quick way to find public statements made by elected officials.

See the Trump, US Congress, and executive branch archives and collected fact-checks.

The big picture

Given his position at the helm of the government, it is not surprising that Trump garners most of the fact-checking attention.  Three out of four, or 1008 of the fact-checks, focus on Trump’s statements. Another 192 relate to the four congressional leaders: Senate Majority Leader Mitch McConnell, R., Ky.; Senate Minority Leader Chuck Schumer, D., N.Y.; House Speaker Paul Ryan, R., Wis.; and House Minority Leader Nancy Pelosi, D., Calif. We’ve also logged 140 fact-checks related to key administration figures such as Sean Spicer, Jeff Sessions, and Mike Pence.

pie chart

The topics

The topics covered by fact-checkers run the gamut of national and global policy issues, history, and everything in between. For example, the debate on tax reform is grounded with fact-checks of the historical and global context posited by the president. Fact-checkers have also examined his aides’ claims on the impact of the current reform proposal on the wealthy and on the deficit. They’ve also followed the claims made by House Speaker Paul Ryan, R., Wis., the leading GOP policy voice on tax reform.

Another large set of fact-checks cover health care, going back as far as this claim made in 2010 by Pelosi about job creation under healthcare reform (PolitiFact rated it “Half True.”) The most recent example is the Graham-Cassidy bill that aimed to repeal much of Obamacare. One of the most sharply contested debates about that legislation was whether or not it would require coverage of people with pre-existing conditions. Fact-checkers parsed the he-said he-said debate as it unfolded on TV news, for example examining dueling claims by Schumer and Trump.

Browse or download  fact-checked TV clips by topic

The old stuff

The collection of Trump fact checks include a few dating back to 2011, long before his successful presidential campaign. Here he is at the CPAC conference that year claiming no one remembered now-former President Barack Obama from school, part of his campaign to question Obama’s citizenship. (PolitiFact rated: “Pants on Fire!”) And here he is with what FactCheck.org called a “100 percent wrong” claim about the Egyptian people voting to overturn a treaty with Israel.

This fact-check of McConnell dates back to 2009, when PolitiFact rated “false” his claim of how much federal spending occurred under Obama’s watch: “In just one month, the Democrats have spent more than President Bush spent in seven years on the war in Iraq, the war in Afghanistan and Hurricane Katrina combined.”

Meanwhile, this 2010 statement by Schumer, rated “mostly false” by PolitiFact, asserted that the U.S. Supreme Court “decided to overrule the 100-year-old ban on corporate expenditures.” The ban on giving directly to candidates is still in place; however,  corporations are free to spend unlimited funds on elections providing they do so separate from a candidate’s official campaign.

The repetition

Twenty-four million people will be forced off their health insurance, young farmers have to sell the farm to pay estate tax, NATO members owe the United States money, millions of women turn to Planned Parenthood for mammograms, and sanctuary cities lead to higher crime. These are all examples of claims found to be inaccurate or misleading, but that continued or continue to be repeated by public officials.

The unexpected

Whether you lean one political direction or another, there are always surprises from the fact-checkers that can keep all our assumptions in check. For example, if you’re opposed to building a wall on the southern border to keep people from crossing into the U.S., you might guess Trump’s claim that people use catapults to toss drugs over current walls is an exaggeration. In fact, that statement was rated “mostly true” by PolitiFact. Or if you’re conservative, you might be surprised to learn an often repeated quote ascribed to Thomas Jefferson, in this case by Vice President Mike Pence, is in fact falsely attributed to him.

How to find

If you’re looking for the most recent TV news statements with fact-checks, you can see the latest offerings on the TV Archive’s homepage by scrolling down.

screen grab of place on tv homepageYou can review whole speeches, scanning for just the fact-checked claims by looking for the fact-check icon  on a program timeline. For example, starting in the Trump Archive, you can choose a speech or interview and see if and how many of the statements were checked by reporters.

screen grab of timeline w icons

You can also find the fact-checks in the growing table, also available to download, which includes details on the official making the claim, the topic(s) covered, the url for the corresponding TV news clip, and the link to the fact-checking article.

image of fact-checks table

To receive the TV News Archive’s email newsletter, subscribe here.

###

TV News Record: Debt ceiling, hurricane funding, GDP

A weekly round up on what’s happening and what we’re seeing at the TV News Archive by Katie Dahl and Nancy Watzman. Additional research by Robin Chin.

In this week’s TV News Archive roundup, we examine the latest Face-o-Matic data (you can too!); and present our partner’s fact-checks on Sen. Ted Cruz’s claims that Hurricane Sandy emergency funding was filled with “unrelated pork” and President Donald Trump’s claims about other country’s GDPs.

What got political leaders sustained face-time on TV news last week?

What got Trump, McConnell, Schumer, Ryan, and Pelosi the longest clips on TV cable news screens this past week? Thanks to our new trove of Face-O-Matic data developed with the start-up Matroid’s facial recognition algorithms, reporters and researchers can get quick answers to questions like these.

House Minority Leader Nancy Pelosi, D., Calif., got almost six minutes – an unusually large amount of sustained face-time for the Democrat from California – from “MSNBC Live” on September 7 covering her press conference following President Donald Trump’s surprise deal with congressional Democrats on the debt ceiling.

Senate Majority Leader Mitch McConnell, R., Ky., also enjoyed his longest sustained face-time segment last week on the debt ceiling, clocking in at 34 seconds on MSNBC’s “Morning Joe.”

House Speaker Paul Ryan, R., Wis., got 11 minutes on September 7 on Fox News’ “Happening Now,” for his weekly press conference, where he was shown discussing a variety of topics, including Hurricane Harvey, tax reform, and also debt relief. For Sen. Majority Leader Chuck Schumer, D., N.Y., the topic that got him the most sustained time–21 seconds–was also his unexpected deal with the president on the debt ceiling.

For President Donald Trump, however, who never lacks for TV news face-time, his longest sustained appearance on TV news this past week was his speech at this week’s 9/11 memorial at the Pentagon.

Fact-check: Hurricane Sandy relief was 2/3 filled with pork and unrelated spending (false)

In the aftermath of Hurricane Harvey, Sen. Ted Cruz, R., Texas, came under criticism by supporting federal funding for Harvey victims while having opposed such funding for victims of Hurricane Sandy in 2013.  Cruz defended himself by saying, “The problem with that particular bill is it became a $50 billion bill that was filled with unrelated pork. Two-thirds of that bill had nothing to do with Sandy.”

But Lori Robertson of FactCheck.org labeled this claim as “false,” noting that a Congressional Research Service study pegged at least 69 percent of that bill’s funding as related to Sandy, and that even more of the money could be attributed to hurricane relief funding: “Cruz could have said he thought the Sandy relief legislation included too many non-emergency items. That’s fair enough, and his opinion. But he was wrong to specifically say two-thirds of the bill “had nothing to do with Sandy,” or “little or nothing to do with Hurricane Sandy.”

Fact-check: Trump spoke with world leader unhappy with nine percent GDP growth rate (three Pinocchios)

At a recent press conference on his tax reform plan, President Donald Trump remarked that other foreign leaders are unhappy with higher rates of growth in gross domestic product (GDP) than the U.S. has. “I spoke to a leader of a major, major country recently. Big, big country. They say ‘our country is very big, it’s hard to grow.’ Well believe me this country is very big. How are you doing, I said. ‘Cause I have very good relationships believe it or not with the leaders of these countries. I said, how are you doing? He said ‘not good, not good at all. Our GDP is 7 percent.’ I say 7 percent? Then I speak to another one. ‘Not good. Not good. Our GDP is only 9 percent.’”

Nicole Lewis of The Washington Post’s Fact Checker gave this claim “three Pinocchios”: “Of the 58 heads of state he’s met or spoke with since taking office, not one can claim 9 percent GDP growth. Perhaps Trump misheard. Or perhaps the other leader was fibbing. Or maybe Trump just thought the pitch for a tax cut sounded better if he could quote two leaders….In any case, Trump is making a major economic error in comparing the GDP of a developed country to a developing one. For his half-truths, and for comparing apples to oranges, Trump receives Three Pinocchios.”

To receive the TV News Archive’s email newsletter, subscribe here.

Face-o-Matic data show Trump dominates – Fox focuses on Pelosi; MSNBC features McConnell

For every ten minutes that TV cable news shows featured President Donald Trump’s face on the screen this past summer, the four congressional leaders’ visages were presented  for one minute, according an analysis of Face-o-Matic downloadable, free data fueled by the Internet Archive’s TV News Archive and made available to the public today.

Face-o-Matic is an experimental service, developed in collaboration with the start-up Matroid, that tracks the faces of selected high level elected officials on major TV cable news channels: CNN, Fox News, MSNBC, and the BBC. First launched as a Slack app in July, the TV News Archive, after receiving feedback from journalists, is now making the underlying data available to the media, researchers, and the public. It will be updated daily here.

Unlike caption-based searches, Face-o-Matic uses facial recognition algorithms to recognize individuals on TV news screens. Face-o-Matic finds images of people when TV news shows use clips of the lawmakers speaking; frequently, however, the lawmakers’ faces also register if their photos or clips are being used to illustrate a story, or they appear as part of a montage as the news anchor talks.  Alongside closed caption research, these data provide an additional metric to analyze how TV news cable networks present public officials to their millions of viewers.

Our concentration on public officials and our bipartisan tracking is purposeful; in experimenting with this technology, we strive to respect individual privacy and extract only information for which there is a compelling public interest, such as the role the public sees our elected officials playing through the filter of TV news. The TV News Archive is committed to doing this right by adhering to these Artificial Intelligence principles for ethical research developed by leading artificial intelligence researchers, ethicists, and others at a January 2017 conference organized by the Future of Life Institute. As we go forward with our experiments, we will continue to explore these questions in conversations with experts and the public.

Download Face-o-Matic data here.

We want to hear from you:

What other faces would you like us to track? For example, should we start by adding the faces of foreign leaders, such as Russia’s Vladimir Putin and South Korea’s Kim Jong-un? Should we add former President Barack Obama and contender Hillary Clinton? Members of the White House staff? Other members of Congress?

Do you have any technical feedback? If so, please let us know what they are by contacting tvnews@archive.org or participating in the GitHub Face-o-Matic page.

Trump dominates, Pelosi gets little face-time

Overall, between July 13 through September 5, analysis of Face-o-Matic data show:

  • All together, we found 7,930 minutes, or some 132 hours, of face-time for President Donald Trump and the four congressional leaders. Of that amount, Trump dominated with 90 percent of the face-time. Collectively, the four congressional leaders garnered 15 hours of face-time.
  • House Minority leader Nancy Pelosi, D., Calif., got the least amount of time on the screen: just 1.4 hours over the whole period.
  • Of the congressional leaders, Senate Majority Leader Mitch McConnell’s face was found most often: 7.6 hours, compared to 3.8 hours for House Speaker Paul Ryan, R., Wis.; 1.7 hours for Senate Minority Leader Chuck Schumer, D., N.Y., and 1.4 hours for Pelosi.
  • The congressional leaders got bumps in coverage when they were at the center of legislative fights, such as in this clip of McConnell aired by CNN, in which the senator is shown speaking on July 25 about the upcoming health care reform vote. Schumer got coverage on the same date from the network in this clip of him talking about the Russia investigation. Ryan got a huge boost on CNN when the cable network aired his town hall on August 21.

Fox shows most face-time for Pelosi; MSNBC, most Trump and McConnell

The liberal cable network MSNBC gave Trump more face-time than any other network. Ditto for McConnell. A number of these stories highlight tensions between the senate majority leader and the president. For example, here, on August 25, the network uses a photo of McConnell, and then a clip of both McConnell and Ryan, to illustrate a report on Trump “trying to distance himself” from GOP leaders. In this excerpt, from an August 21 broadcast, a clip of McConnell speaking is shown in the background to illustrate his comments that “most news is not fake,” which is interpreted as “seem[ing] to take a shot at the president.”

MSNBC uses photos of both Trump and McConnell in August 12 story on “feud” between the two.

While Pelosi does not get much face-time on any of the cable news networks examined, Fox News shows her face more than any other. In this commentary report on August 20, Jesse Waters criticizes Pelosi for favoring the removal of confederate statues placed in the Capitol building. “Miss Pelosi has been in Congress for 30 years. Now she speaks up?” On August 8, “Special Report With Bret Baier” uses a clip of Pelosi talking in favor of women having a right to choose the size and timing of her family as an “acid test for party base.”

Example of Fox News using a photo of House Minority Leader Nancy Pelosi to illustrate a story, in this case about a canceled San Francisco rally.

While the BBC gives some Trump face-time, it gives scant attention to the congressional leaders. Proportionately, however, the BBC gives Trump less face-time than any of the U.S. networks.

On July 13 the BBC’s “Outside Source” ran a clip of Trump talking about his son, Donald Trump, Jr.’s, meeting with a Russian lobbyist.

For details about the data available, please visit the Face-O-Matic page. The TV News Archive is an online, searchable, public archive of 1.4 million TV news programs aired from 2009 to the present.  This service allows researchers and the public to use television as a citable and sharable reference. Face-O-Matic is part of ongoing experiments in generating metadata for reporters and researchers, enabling analysis of the messages that bombard us daily in public discourse.

 

McConnell, Schumer, Ryan, Pelosi fact-checked clips featured in new TV News Archive collections

Today the Internet Archive’s TV News Archive unveils growing TV news collections focused on congressional leadership and top Trump administration officials, expanding our experimental Trump Archive to other newsworthy government officials. Together, all of the collections include links to more than 1,200 fact-checked clips–and counting–by our national fact-checking partners, FactCheck.org, PolitiFact, and The Washington Post‘s Fact Checker.

These experimental video clip collections, which contain more than 3,500 hours of video, include archives focused on Senate Majority Leader Mitch McConnell, R., Ky.; Sen. Minority Leader Charles (“Chuck”) Schumer, D., N.Y.; House Speaker Paul Ryan, R., Wis.; and House Minority Leader, Nancy Pelosi, D., Calif., as well as top Trump officials past and present such as Secretary of State Rex Tillerson and former White House Press Secretary Sean Spicer.

Download a csv of fact-checked video statements or see all the fact-checked clips.

Visit the U.S. Congress archive.

Visit the Executive Branch archive.

Visit the Trump Archive.

We created these largely hand-curated collections as part of our experimentation in demonstrating how Artificial Intelligence (AI) algorithms could be harnessed to create useful, ethical, public resources for journalists and researchers in the months and years ahead. Other experiments include:

  • the Political TV Ad Archive, which tracked airings of political ads in the 2016 elections by using the Duplitron, an open source audio fingerprinting tool;
  • the Trump Archive, launched in January;
  • Face-O-Matic, an experimental Slack app created in partnership with Matroid that uses facial detection to find congressional leaders’ faces on TV news. Face-O-Matic has quickly proved its mettle by helping our researchers find clips suitable for inclusion in the U.S. Congress Archive; future plans include making data available in CSV and JSON formats.
  • in the works: TV Architect Tracey Jaquith is experimenting with detection of text in the chyrons that run on the bottom third of cable TV news channels. Stay tuned.

Red check mark shows there’s a fact-check in this footage featuring House Minority Leader Nancy Pelosi, D., Calif. Follow the link below the clip to see the fact-check, in this case by The Washington Post’s Fact Checker.

At present, our vast collection of TV news –1.4 million shows collected since 2009–is searchable via closed-captioning. But closed captions, while helpful, can’t help a user find clips of a particular person speaking; instead, when searching a name such as “Charles Schumer” it returns a mix of news stories about the congressman, as well as clips where he speaks at news conferences, on the Senate floor, or in other venues.

We are working towards a future in which AI enrichment of video metadata will more precisely identify for fact-checkers and researchers when a public official is actually speaking, or some other televised record of that official making an assertion of fact. This could include, for example, camera footage of tweets.

Such clips become a part of the historical record, with online links that don’t rot, a central part of the Internet Archive’s mission to preserve knowledge. And they can help fact-checkers decide where to concentrate their efforts, by finding on-the-record assertions of fact by public officials. Finally, these collections could prove useful for teachers, documentary makers, or anybody interested in exploring on-the-record statements by public officials.

For example, here are two dueling views of the minimum wage, brought to the public by McConnell and Schumer.

In this interview on Fox News in January 2014, McConnell says, “The minimum wage is mostly an entry-level wage for young people.” PolitiFact’s Steve Contorno rated this claim as “mostly true.” While government statistics do show that half of the people making the minimum wage are young, 20 percent are in their late 20s or early 30s and another 30 percent are 35 or older. Contorno also points out that it’s a stretch to call these jobs “entry-level,” but rather are “in the food or retail businesses or similar industries with little hope for career advancement.”

Schumer presents a different assertion on the minimum wage, saying on “Morning Joe” in May 2014 that with a rate of $10.10/hour “you get out of poverty.” PolitiFact’s Louis Jacobson rated this claim as “half true”: “Since the households helped by the $10.10 wage account for 46 percent of all impoverished households, Schumer is right slightly less than half the time.”

These new collections reflect the hard work of many at the Internet Archive, including Robin Chin, Katie Dahl, Tracey Jaquith, Roger MacDonald, Dan Schultz, and Nancy Watzman.

As we move forward, we would love to hear from you. Contact us with questions, ideas, and concerns at tvnews@archive.org. And to keep up-to-date with our experiments, sign up for our weekly TV News Archive newsletter.

 

TV News Record: McCain returns to vote, Spicer departs

A weekly round up on what’s happening and what we’re seeing at the TV News Archive by Katie Dahl and Nancy Watzman. Additional research by Robin Chin.

Last week, Sean Spicer left his White House post and Anthony Scaramucci, the new communications director, made his mark; Sen. John McCain, R., Ariz., returned to the Senate floor to debate–and cast a deciding vote on–health care reform; and fact-checkers examined claims about Trump’s off-the-record meeting with Russian President Vladimir Putin, and more.

McCain shows up in D.C. – and on Face-O-Matic

Last week, after we launched Face-O-Matic, an experimental Slack app that recognizes the faces of top public officials when they appear on TV news, we received a request from an Arizona-based journalism organization to track Sen. John McCain, R., Ariz.. Soon after we added the senator’s visage to Face-O-Matic, we started getting the alerts.

News anchors talked about how McCain’s possible absence because of his brain cancer diagnosis could affect upcoming debates and votes on health care.

Reporters gave background on how the Senate has dealt with absences due to illness in the past.

Pundits discussed McCain’s character, and his daughter provided a “loving portrait.” Then coverage shifted to report the senator’s return to Washington, and late last night his key no vote on the “skinny” health care repeal.



White House: Spicer out, Scaramucci in 

After Sean Spicer resigned as White House communications director, Fox News and MSNBC offered reviews of his time at the podium.

On Fox News, Howard Kurtz introduced Spicer as someone “long known to reporters as an affable spokesman; he became the president’s pit bull,” and went on to give a run-down of his controversial relationship with the press. The conclusion, “He lasted exactly, six months.”

MSNBC offered a mashup of some of Spicer’s most famous statements. These include: “This was the largest audience to ever witness an inauguration, period, both in person and around the globe,” and “But you had a – you know, someone who is as despicable as Hitler who didn’t even sink to using chemical weapons.”

Late this week, Ryan Lizza published an article in The New Yorker based on a phone call he received from the new White House communications director, Anthony Scaramucci, in which the new White House communications director used profanity to describe other members of the White House staff he accused of leaking information. That article soon became fodder for cable TV.



Schumer, Ryan weigh in on Mueller

As Special Counsel Robert Mueller widens his investigation into Russian interference in U.S. elections, speculation is running high on TV news that President Donald Trump might fire him.

Fox News ran a clip of Senate Minority Leader Chuck Schumer, D., NY., saying, “I think it would cause a cataclysm in Washington.”

MSNBC ran a radio clip from House Speaker Paul Ryan, R., Wis.:  “I don’t think many people are saying Bob Mueller is a person who is a biased partisan. We have an investigation in the House, an investigation in the Senate, and a special counsel which sort of depoliticizes this stuff and gets it out of the political theater.”



Fact-check: Transgender people in the military would lead to tremendous medical costs and disruption (lacks context)

In a series of tweets this week, President Trump wrote, “After consultation with my Generals and military experts, please be advised that the United States Government will not accept or allow… Transgender individuals to serve in any capacity in the U.S. Military. Our military must be focused on decisive and overwhelming… victory and cannot be burdened with the tremendous medical costs and disruption that transgender in the military would entail. Thank you.”

For FactCheck.org, Eugene Kiely reported, “Although Trump described the cost as ‘tremendous,’ RAND estimated that providing transition-related health care would increase the military’s health care costs for active-duty members ‘by between $2.4 million and $8.4 million annually.’ That represents an increase of no more than 0.13 percent of the $6.27 billion spent on the health of active-duty members in fiscal 2014.”



Fact Check: Nixon held meetings with heads of state without an American interpreter (true)

Speaking on “The Rachel Maddow Show,” Ian Bremmer, president of the Eurasia Group, said:  “Apparently, President Nixon used to do it because he felt, didn’t really trust the State Department, at that point, providing the translators and didn’t necessarily want information getting out, leaking, that he would want to keep private.”

“True,” wrote Joshua Gillan for PolitiFact: “Presidential historians, historical accounts and Nixon’s own memoir show this was the case. But it’s notable that even in the example most comparable to Trump’s meeting with Putin, when Nixon used only a Soviet translator during two meetings with Brezhnev, official records of the meeting exist.”



Fact-check: Allowing insurers to sell plans across state lines will mean premiums go down 60-70% (no evidence)

Not long before the Senate took up health care reform, President Donald Trump said “We’re putting it [allowing insurers to sell plans across state lines] in a popular bill, and that will come. And that will come, and your premiums will be down 60 and 70 percent.”

FactCheck.org’s Lori Robertson reported the “National Association of Insurance Commissioners — a support organization established by the country’s state insurance regulators — said the idea that cross-state sales would bring about lower premiums was a ‘myth.’”



Fact-Check: When the price for oil goes up, it goes up, and never goes down (false)

In an interview Sunday about the new Democratic Party national agenda, Senate Minority Leader Chuck Schumer, D., N.Y., said, “We have these huge companies buying up other big companies. It hurts workers and it hurts prices. The old Adam Smith idea of competition, it’s gone. So people hate it when their cable bills go up, their airline fees. They know that gas prices are sticky. You know … when the price for oil goes up on the markets, it goes right up, but it never goes down.”

For PolitiFact, Louis Jacobson reported, “This comment takes a well-known phenomenon and exaggerates it beyond recognition. While experts agree that prices tend to go up quickly after a market shock but usually come down more slowly once the shock is resolved, this phenomenon only occurs on a short-term basis – a couple of weeks in most cases.”

To receive the TV News Archive’s email newsletter, subscribe here.

TV News Record: adventures with Face-O-Matic

A weekly round up on what’s happening at the TV News Archive by Katie Dahl and Nancy Watzman

This week we bring you adventures with Face-O-Matic; fact-checks on President Donald Trump’s legislative record and on health care reform; and we follow the TV on use of the term “lies” and “lying.”

Here’s a face, there’s a face, everywhere a face…

Face-O-Matic, our new experimental Slack app that finds faces of political leaders on major national cable networks, has given us a whole new perspective on how imagery is used in news production. Sure, Face-O-Matic picks up clips of President Donald Trump, Senate Majority Leader Mitch McConnell, R., Ky., and others speaking, whether on the floor, at press conferences, or at a luncheon.

However, often these elected officials’ faces are used to illustrate a point a news anchor is making, or in footage without audio, sometimes as a floating head somewhere on the screen, or as part of a tweet. Face-O-Matic even picks up faces in a crowd.

Face-O-Matic can help find frequently re-aired clips.

Face-O-Matic can find even images that are only briefly displayed on the screen.

Face-O-Matic finds both still and video of Trump in a single clip.

Please take Face-O-Matic on a spin and share your feedback with us, tvnews@archive.org. This blog post explains how it fits into our overall plan to turn TV news into data. To install,  for now you’ll need to ask your Slack team administrator or owner to set it up. The administrator can click on the button below to get started. Visit Slack to learn how to set up or join a Slack team. Questions? Contact Dan Schultz, dan.schultz@archive.org.

Fact-check: Trump has signed more bills than any president ever (wrong)

News cameras captured Trump saying, “We’ve signed more bills — and I’m talking about through the legislature — than any president ever.” (A moment later, he commented that he doesn’t “like Pinocchios,” referring to The Washington Post’s Fact Checker rating system.)

Glenn Kessler, reporting for that same fact-checking site, explained why Trump is not, in fact, besting his predecessors in the White House when it comes to bill signing. But, he refrains from stating how many Pinocchios the president had earned: “Tempted as we are to give the president Pinocchios for his statement, he seemed to be speaking off the cuff and was operating on outdated information from his first 100 days. We don’t play gotcha here at The Fact Checker, and we appreciate that he added a caveat. He certainly appeared to pause for a moment and wonder if he was right. For Trump, that’s a step in the right direction…But he’s way off the mark and actually falling behind in legislative output.”



Fact-check: “bushel” of Pence claims on health care reform (range from “twists the facts” to “false”)

Also writing for The Washington Post’s Fact Checker, Michelle Ye Hee Lee checked a number of statements Vice President Mike Pence made about the Senate health care reform bill during an appearance at the National Governors Association.

These included, for example, the claim, “I know Governor Kasich isn’t with us, but I suspect that he’s very troubled to know that in Ohio alone, nearly 60,000 disabled citizens are stuck on waiting lists, leaving them without the care they need for months or even years.”

Lee wrote that this claim is false: “[T]here’s no evidence the wait lists are tied to Medicaid expansion. We previously gave four Pinocchios to a similar claim….The expansion and wait list populations are separate, and expansion doesn’t necessarily affect the wait list population….Whether people move off the wait list depends on many factors, such as how urgent their needs are, how long they’ll need services and whether the states have money to pay for them. Many times, a slot opens up only if someone receiving services moves out of the state or dies.”



Follow the TV

There’s been much controversy in news gathering circles about when, whether, and how to invoke the word “lie” when reporting on public officials. One of our archivists, Robin Chin, has noticed a number of prominent uses of the term by commentators in recent TV news coverage.

For example, here’s Shepard Smith on Fox News on July 14 saying, “Jared Kushner filled out his form. I think it’s an F-86 saying who he met with and what he had done… He went back and added 100 names and places. None of these people made it… Why is it lie after lie after lie? … My grandmother used to say when first we practice to — oh, what a tangled web we weave when first we practice to deceive. The deception, Chris, is mind boggling.”

And here’s Tom Brokaw on July 16 on NBC’s “Meet the Press,” saying: “Certainly there are atmospherics here that call to mind Watergate, the kind of denial of the obvious and the petty lying that is going on. But at the same time, Watergate, I like to think, was there by itself and this president is entangling himself in that kind of discussion they we’re having here today when it’s not in the interest of anyone, most of all this country, when we have so many issues before us. It’s got to get cleaned up.

On July 17, on “CNN: Tonight With Don Lemon,” here is David Gergen saying: “Other presidents succeed at this by just being straightforward about the facts. And it’s gone on for so long and so duplicitous and so much double speak that you begin to wonder, this is quite intentional. This may be quite intentional. You create a fog bank of lies and uncertainties and vagueness and create so many different details that people just sort of say, the hell with that, I don’t want to watch this… My sense is that a lot of Americans are starting to tune out…”

Search captions for terms you are interested in at the TV News Archive. For trends, try the Television Explorer, built by data scientist Kalev Leetaru, and powered by TV News Archive data, which can provide quick visualizations of terms broken down by network.

To receive the TV News Archive’s email newsletter, subscribe here.

Internet Archive TV News Lab: Introducing Face-O-Matic, experimental Slack alert system tracking Trump & congressional leaders on TV news

Working with Matroid, a California-based start up specializing in identifying people and objects in images and video, the Internet Archive’s TV News Archive today releases Face-O-Matic, an experimental public service that alerts users via a Slack app whenever the faces of President Donald Trump and congressional leaders appear on major TV news cable channels: CNN, Fox News, MSNBC, and the BBC. The alerts include hyperlinks to the actual TV news footage on the TV News Archive website, where the viewer can see the appearances in context of the entire broadcast, what comes before and what after.

The new public Slack app, which can be installed on any Slack account by the team’s administrator, marks a milestone in our experiments using machine learning to create prototypes of ways to turn our public, free, searchable library of 1.3 million+ TV news broadcasts into data that will be useful for journalists, researchers, and the public in understanding the messages that bombard all of us day-to-day and even minute-to-minute on TV news broadcasts. This information could provide a way to quantify “face time”–literally–on TV news broadcasts. Researchers could use it to show how TV material is recycled online and on social media, and how editorial decisions by networks help set the terms of public debate.

If you want Face-O-Matic to post to a channel on your team’s Slack, ask an administrator or owner to set it up. The administrator can click on the button below to get started. Visit Slack to learn how to set up or join a Slack team. Questions? Contact Dan Schultz, dan.schultz@archive.org.

Add to Slack

To begin, Dan Schultz, senior creative technologist for the TV News Archive, trained Matroid’s facial detection system to recognize the president;  Senate Majority Leader Mitch McConnell, R., Ky., and Senate Minority Leader Charles Schumer, D, NY; and House Speaker Paul Ryan, R-Wis. and House Minority Leader Nancy Pelosi, D., Calif. All are high-ranking elected officials who make news and appear often on TV screens. The alerts appear in a constantly updating stream as soon as the TV shows appear in the TV News Archive

For example, on July 15, 2017 Face-O-Matic detected all five elected officials in an airing of MSNBC Live.

As can be seen, the detections in this case last as little as a second – for example, this flash of Schumer’s and McConnell’s faces alongside each other is a match for both politicians. The moment is from a promotion for “Morning Joe,” the MSNBC show that made headlines in late June when co-hosts Mika Brzezinski and Joe Scarborough were the targets of angry tweets from the president.  

The longest detected segment in this example is 24 seconds featuring Trump, saying “we are very very close to ending this health care nightmare. We are so close. It’s a common sense approach that restores the sacred doctor-patient relationship. And you’re going to have great health care at a lower price.”

Why detect faces of public officials?

First, our concentration on public officials is purposeful; in experimenting with this technology, we strive to respect individual privacy and harvest only information for which there is a compelling public interest, such as the role of elected officials in public life. The TV News Archive is committed to these principles developed by leading artificial intelligence researchers, ethicists, and others at a January 2017 conference organized by the Future of Life Institute.

Second, developing the technology to recognize faces of public officials contained within the TV News Archive and turning it into data opens a whole new dimension for journalists and researchers to explore for patterns and trends in how news is reported.  

For example, it will eventually be possible to trace the origin of specific video clips found online; to determine how often the president’s face appears on TV networks and programs compared to other public officials; to see how often certain video clips are repeated over time; to determine the gender ratio of people appearing on TV news; and more. It will become useful not just in explaining how media messages travel, but also as a way to counter misinformation, by providing a path to verify source material that appears on TV news.

This capability adds to the toolbox we’ve already begun with the Duplitron, the open source audio fingerprinting tool developed by Schultz that the TV News Archive used to track political ads and debate coverage in the 2016 elections for the Political TV Ad Archive. The Duplitron is also the basis for The Glorious ContextuBot, which was recently awarded a Knight Prototype Fund grant.

All of these lines of exploration should help journalists and researchers who currently can only conduct such analyses by watching thousands of hours of television and hand coding it or by using an expensive private service. Because we are a public library, we make such information available free of charge.

What’s next?

The TV News Archive will continue to work with partners such as Matroid to develop methods of extracting metadata from the TV News Archive and make it available to the public. We will develop ways to deliver such experimental data in structured formats (such as JSON, csv, etc.) to augment Face-O-Matic’s Slack alert stream. Such data could help researchers conduct analyses of the different amounts of “face-time” public officials enjoy on TV news.

Schultz also hopes to develop ways to augment the facial detection data with closed captioning, with for example OpenedCaptions, another open source tool he created that provides a constant stream of data from TV for any service set up to listen. This will make it simpler to search such data sets to find a particular moment that a researcher is looking for. (Accurate captioning presents its own technological challenges: see this post on Hyper.Audio’s work.)

Beyond this experimental facial detection, we have big plans for the future.  We are planning to make more than a million hours of TV news available to researchers from both private and public institutions via a digital public library branch of the Internet Archive’s TV News Archive. These branches would be housed in computing environments, where networked computers provide the processing power needed to analyze large amounts of data.

Researchers will be able to conduct their own experiments using machine learning to extract metadata from TV news. Such metadata could include, for example, speaker identification–a way to identify not just when a speaker appears on a screen, but when she or he is talking. Researchers could create ways to do complex topic analysis, making it possible to trace how certain themes and talking points travel across the TV news universe and perhaps beyond. Metadata generated through these experiments would then be used to enrich the TV News Archive, so that any member of the public could do increasingly sophisticated searches.

Feedback! We want it 

We are eager to hear from people using the Face-O-Matic Slack app and get your feedback.

  • Is the Face-O-Matic Slack app useful? What would make it more useful?
  • Would a structured data stream delivered via JSON, csv, and/or other means be helpful? What sort of information would you like to be included in such a data set?
  • Who is it important for us to track?
  • What else?

Please reach us by email at: tvnews@archive.org, or via twitter @tvnewsarchive. Also please consider signing up for our weekly TV News Archive newsletter. Or, comment or make contributions over here, where Schultz is documenting his progress; all the code developed is open source. (One observer already provided images for a training set to track Mario, the cartoon character.)

The weeds

The TV News Archive, our collection of 1.3 million+ TV news broadcasts dating back to 2009, is already searchable through closed captions.

But captions don’t always get you everything you want. If you search, for example, on the words “Donald Trump” you get back a hodge-podge of clips in which Trump is speaking and clips where reporters are talking about Trump. His image may not appear on the screen at all. The same is true for “Barack Obama,” “Mitch McConnell,” “Chuck Schumer,” or any name.

.

Search “Barack Obama” and the result is a hodge podge of clips.

Developing the ability to search the TV News Archive by recognizing the faces of public officials requires applying algorithms such as those developed by Matroid. In the future we hope to work with a variety of firms and researchers; for example, Schultz is also working on a separate facial detection experiment with the firm Datmo.

Facial detection requires a number of related steps: first, training the system to recognize where a face appears on a TV screen; second, extracting that image so it can be analyzed; and third, comparing that face to a set known to be a particular person to discover matches.

In general, facial recognition algorithms tend to rely on the work of FaceNet, described in this 2015 paper, in which researchers describe creating a way of “mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.” In other words, it’s a way of turning a face into a pattern of data, and it’s sophisticated enough to describe faces from various vantage points – straight ahead, three-quarter view, side view, etc. To develop Face-o-Matic, TV News Archive staff collected public images of elected officials from different vantage points to use as training sets for the algorithm.

The Face-O-Matic Slack app is meant to be a demonstration project that allows the TV News Archive a way to experiment in two ways: first, by creating pipelines that run the TV News Archive video streams through Artificial Intelligence models to explore whether the resulting information is useful; second, by using a new way to distribute TV News Archive information through the popular Slack service, used widely in journalistic and academic settings.  

We know some ways it can be improved, but we also want to hear from you, the user, with your ideas. In the words of Thomas the Tank Engine, we aspire to be a “really useful engine.”

Face-O-Matic on GitHub

Follow TV News Archive progress in recognizing faces on TV on the following GitHub pages:

Tvarchive-faceomatic. The Face-o-Matic 2000 finds known faces on TV.

Tvarchive-ai_suite. A suite of tools for exploring AI research against video

This post is part of a blog series, TV News Lab, in which we demonstrate how the Internet Archive is partnering with technology, journalism, and academic organizations to experiment with and improve the TV News Archive, our free, public, online library of TV news shows.