Sarah Lamdan was working as an academic law librarian at the City University of New York in 2017 when something concerning caught her eye.
“I was really startled and confused because I didn’t understand how Lexis and Westlaw would be doing ICE surveillance,” said Lamdan, who wondered about the potential impact on the campus’ immigrant population and her role as a librarian in giving away data.
Lamdan and a colleague wrote a blog for the American Association of Law Libraries raising questions. However, within minutes, at the “advice of legal counsel,” the post was removed, Lamden said. She didn’t know why they were not allowed to raise the issue, and her quest for answers began.
“It made me really, really curious,” Lamdan said. “That started this five-year course of research to unpack what these companies really are, what they’re doing, how they can be the main legal information providers and also be building surveillance systems.”
She shares her findings in “Data Cartels: The Companies that Control and Monopolize Our Information” published in November by Stanford University Press. Lamdan talked about her book with SPARC Executive Director Heather Joseph at an online webinar November 30 sponsored by the Internet Archive and the Authors Alliance. [Recording available here]
Watch Session Recording
The U.S. Immigration and Customs Enforcement (ICE) was building an invasive data surveillance system and journalists reported that Thomson Reuters and LexisNexis were interested in participating. She quickly realized that those were the parent companies of the gold-standard legal databases, Westlaw and Lexis, that Lamdan regularly taught students to use.
The book chronicles the unregulated underworld of a few companies that operate as “data cartels,” highlighting how selling data and informational resources perpetuate social inequalities and threaten the democratic sharing of knowledge.
In her research, Lamdan, who has a law degree and master’s in library science, said she was surprised to discover the scope of the enterprises and ways they leveraged users’ personal data without consent.
“I saw Lexis and Westlaw as these little mom-and-pop legal information expert shops that gave us tote bags and helped sponsor our annual meeting,” Lamdan said. “I didn’t realize that they are actually parts of these multi-billion-dollar giant corporations that are basically like informational warehouses.”
The library community has been increasingly concerned about companies’ commoditization of research, said Joseph, and the book spells about the trend with a sense of urgency.
“We think of these companies as content providers, but they’re more than that,” Joseph said. “They have a multiplicity of companies that have different functions under the umbrella company name and what those divisions do is critically important. For example, having one company essentially, owning the legal corpus of the United States and then controlling the data of people who access that information and distributing it is unbelievable.”
Purchase from the publisher, Stanford University Press
Too often, people view legal or academic publishers as benign distributors of useful information, Joseph said, but it is big business driven by profit. Companies are increasingly seeing opportunities to expand their services and become data analytic brokers. With so much information in the hands of so few players, these companies have a stronghold over predictive platforms affecting people’s privacy, health and finances.
Information is a unique commodity, Lamdan said, because one information product cannot be replaced with another similar product. Libraries can’t merely unsubscribe to these services or journals because students and attorneys rely on the unique informational products they provide. This has created a classic monopoly problem where consumers have little choice about which products they use, which Lamdan said should be addressed.
“Together, these companies are pivoting from publishing, towards data analytics. They are changing the way our information systems work and the way their markets work,” Lamdan said in the online talk. “They are acting in a way that drives us from information access to these closed walled garden data analytics systems that exploit our personal data and limit access to certain types of information.”
Lamdan is clear that there is no one fix to address the concentration of power in these information companies. She does, however, suggest that federal antitrust laws be revisited and revised to better address digital and data problems. Regulators could intervene to say that companies should not be allowed to be in both the business of providing critically important information to the public, and the business of selling personal data products to the government simultaneously.
Joseph said the broader community can break its dependency on these companies by expanding open access and creating an infrastructure that does not rely on commercial enterprises for information. Approaching knowledge as a public good, rather than a private commodity, can also shift the framework for how information is disseminated.
To find out more about Lamdan’s book or to purchase a copy, click here.
Friends and supporters of the Internet Archive gathered October 19 at the organization’s headquarters in San Francisco to celebrate the launch of Democracy’s Library.
Plans to collect government documents from around the world and make them easily accessible online were met with enthusiasm and endorsements. Speakers at the event expressed an urgency to preserve the public record, make valuable research discoverable, and keep the citizenry informed—all potential benefits of Democracy’s Library.
“If we really succeed — and we have to succeed — then Democracy’s Library might become an inspiration for openness in areas that are becoming more and more closed,” said Internet Archive founder Brewster Kahle.
The 10-year project aims to make freely available the massive volume of government publications (from the U.S. and other democracies), including books, guides, reports, surveys, laws and academic research results, which are all funded with taxpayer money, but often difficult to find.
To kick off the project, Kahle announced the Internet Archive’s initial contributions to Democracy’s Library:
United States .gov websites collected since 2008;
Crawls of the U.S. state government websites;
Digitized microfilm and microfiche from the U.S. Government Publishing Office, NASA and other government entities;
Crawls of government domains from 200 other countries;
50 million government PDF documents made into text searchable information.
It will be a collaborative effort, said Kahle, calling upon others to join in the ambitious undertaking to contribute to the online collection.
The need for Democracy’s Library
“We need Democracy’s Library. The Internet Archive’s work leading this project represents a critical step in the evolution of democracy,” said Jamie Joyce, executive director of The Society Library and emcee of the program. “Archives and libraries, as they’ve always done in the past, will continue to change in their scope, scale, and capabilities to be of critical use to society, especially democratic societies. Tonight is about witnessing another transformation.”
Although there is more data available than ever before, Joyce said, society’s knowledge management system is badly broken. Misinformation is rampant, while high quality government data is buried and scattered across different federal, state and local agencies.
Having public material consolidated, digitized and machine readable will allow journalists, activists, and others to be better informed. It will also make democracy more transparent and accountable, as well as protect the historical documents. “We will not be able to compute in the future what we do not save today,” Joyce said.
At a time when polarized politics can put information at risk, the event highlighted the need to safeguard public data.
Gretchen Gehrke, co-founder of the Environmental Data and Governance Initiative, has been working in partnership with the Internet Archive to track changes in federal environmental websites.
“People should be able to know about environmental issues and have a say in environmental decisions,” she said. “For the last 20 years, the majority of this information has been delivered through the web, but the right to access that information through the web is not protected.”
Gehrke described how public resources and tools related to the federal Clean Power Plan, a hallmark environmental regulation of the Obama administration, were taken down from the Environmental Protection Agency’s website under President Trump’s tenure.
“There are no policies protecting federal website information from suppression or outright censorship,” Gehrke said. “This case serves as an example of why we need Democracy’s Library to preserve and provide continued access to these critical government documents.”
When statistics are being cited in policy debates, citizens need to be able to have access to sources of claims. For example, Sharon Hammond, chief operating officer of The Society Library, said documents related to the environmental impact of California’s Diablo Canyon power plant should be easily available. There are nearly 5 different government bodies that have some role in monitoring the plant’s ecological impact, but the agencies house the reports on their own websites.
“Finding governmental records about public policy matters should not be a barrier to becoming an informed participant in these collective decisions,” Hammond said. “When we connect evidence directly to the claims and make that information publicly accessible as a resource, we can improve the public discourse.”
Hammond said a searchable, machine readable repository of government documents, with active links and a register of relevant government agencies, will dramatically increase meaningful access to the public’s information.
An international vision
The effort is an international one, and Canada has stepped forward as an early partner.
Canada has contributed crawls by the Library and Archives Canada of all the country’s government websites, as well as digitized microfilm and books from the Canadian Research Knowledge Network, Canadiana, and the University of Toronto.
Leslie Weir, librarian and archivist of Canada, spoke in support of the initiative.
“We know by making our collection and work of government openly accessible, we will create a more engaged community, a community that participates in elections, school board meetings, in public consultations, and yes, even and especially in protests,” Weir said. “Access is the key to understanding. And understanding is the underpinning of democracy.”
Celebrating heroes
The festivities concluded with a tribute to Carl Malamud, recipient of the 2022 Internet Archive Hero Award. Corynne McSherry, legal director of the Electronic Frontier Foundation, presented the award. “Carl has always seen what the internet could be. He has dedicated his life to building that internet,” she said. “He is a true hero.”
Malamud said government information is more than just a good idea. “It is about the law. It is about our rulebook. It is the manual on how we, as citizens, choose to run our society. We own this manual,” he said. “We cannot honor our obligations to future generations if we cannot freely read and speak and even change that rulebook.”
Malamud urged the audience to get involved to realize the vision of Democracy’s Library and guarantee universal access to human knowledge.
“This is our moment. We must build a distributed and interoperable internet for our global village. We must make the increase in diffusion of knowledge our mutual and everlasting mission,” Malamud said. “We must seize the means of computation and share their fruits with all the people. Let us all swim together in the ocean of knowledge.”
For more on Malamud’s career and contributions, read his profile here.
Democracies need an educated citizenry to thrive. In the 21st century, that means easy access to reliable information online for all.
To meet that need, the Internet Archive is building Democracy’s Library—a free, open, online compendium of government research and publications from around the world.
“Governments have created an abundance of information and put it in the public domain, but it turns out the public can’t easily access it,” said Internet Archive founder Brewster Kahle, who is spearheading the effort to collect materials for the digital library.
By having a wealth of public documents curated and searchable through a single interface, citizens will be able to leverage useful research, learn about the workings of their government, hold officials accountable, and be more informed voters.
Too often, the best information on the internet is locked behind paywalls, said Kahle, who has helped create the world’s largest digital library.
“It’s time to turn that scarcity model upside down and build an internet based on abundance,” Kahle said. There is a need for equitable access to objective, historical information to balance the onslaught of misinformation online.
Libraries have long played a vital role in collecting and preserving materials that can educate the public. This mission continues, but the collections need to include digital items to meet the needs of patrons of the internet generation today.
Over the next decade, the Internet Archive is committing to work with libraries, universities, and agencies everywhere to bring the government’s historical information online. It is inviting citizens, libraries, colleges, companies, and the Wikipedians of the world to unlock good information and weave it back into the Internet.
Democracy’s Library will be celebrated at the October 19 event, Building Democracy’s Library, in San Francisco and online.
Watch the livestream of Building Democracy’s Library:
The project is part of Kahle’s vision to build a better Internet—one that keeps the public interest above private profit. It is based on an abundance model, in which data can be uncovered, unlocked and reused in new and different ways.
“We know there’s an information flood, but it’s not necessarily all that good,” Kahle said. “It turns out the information on the Internet is not very deep. If you know a subject well, you find that the best information is buried or not even online.”
Democracy’s Library is a move to make governments’ massive investment in research and publications open to all.
Kahle added: “Democracy’s Library is a stepping stone toward citizens who are more empowered and more engaged.“
Carl Malamud is a man with a mission: To make public information freely available to the public.
For more than three decades, Malamud has not just talked in theory about why government materials should be online—he has taken action to digitize and upload massive amounts of data himself. He is the reason many laws and judicial opinions, corporate filings and patents, Congressional hearings and government films are at the fingertips of the American people.
“Our democracy, particularly today, depends on an informed citizenry, with so much misinformation and disinformation,” said Malamud, 63, founder of the nonprofit organization Public.Resource.Org. “We have to learn how our government works, what our fundamental values are, and we have to communicate that with our fellow citizens.”
Malamud is a disrupter for the public good.
His effort to unleash government data behind paywalls has put him at odds with many trying to profit from dispensing public records. Yet in case after case, Malamud is winning and adding to the body of open knowledge freely available online.
In recognition of his relentless work on behalf of the public interest, Malamud has been honored with the 2022 Internet Archive Hero Award.
“Carl has spent his career getting public access to the public domain, bringing government information to everyone with no restrictions,” said Internet Archive founder Brewster Kahle. “He’s been unwavering in his vision, seeing how the works of governments can be leveraged by everyone using this digital technology.”
Although he’s not in the civil service, Kahle said Malamud acts as a civil servant. He sides with advancing the public interest over corporate profits, and has been a pioneer in how to operate a nonprofit in the internet space. Malamud’s tenacity and drive is at the essence of what it means to be a hero, said Kahle: “Somebody who puts themselves at risk or in harm’s way to get their vision built.”
Early work
After studying the convergence of computers and communication in college, Malamud went to Washington, D.C., to work in public policy. Malamud developed an expertise in databases, networking, and technology to broadcast audio and video over the internet. In 1993, he started the nonprofit Internet Multicasting Service and ran the first radio station on the internet out of an office in the National Press Building. (An archive of his broadcasts from 1993-95 are available here.)
One of Malamud’s early projects was putting corporate information from the U.S. Securities and Exchange Commission — the Electronic Data Gathering, Analysis, and Retrieval system (EDGAR) online. This allowed investors, journalists and citizens to download information about SEC filings for free, rather than pay a fee to a private company.
The work was funded with a grant from the National Science Foundation, and with money left over from the project, Malamud put the databases from the U.S. Patent Office online. In each instance, Malamud had to first purchase the database from.
“I got a grant from the American people, to buy the data from the American people, so I could give it back to the American people,” said Malamud, who often uses such plain language in his arguments for unlocking information into the public realm.
Demonstrating by doing
Getting the SEC data online was a seminal event, said Tim O’Reilly, founder of O’Reilly Media, noting he and others were inspired by Malamud’s fearless “hacktivism” approach. “It was the beginning of the open government data movement,” he said. “I’ve always called Carl an unsung hero ever since that, because he’s the guy who started it all in motion.”
Faced with pushback from entities that say it’s too hard or it will take too long to put information online, Malamud moves forward and demonstrates it can be done affordably—and the public will use it. It was Malamud who set up the first internet demonstration in the White House during Bill Clinton’s presidency. He advised the administration, and others that followed, on technology policy and identified opportunities to make government records available online—and demonstrated it’s possible.
“Carl has an unwavering commitment to the core principle that citizens should have access to the law and to government documents….and he’s establishing an important legal precedent,” said Tom Kalil, former White House aide to President Clinton and President Obama. “He’s not just a public intellectual writing op-eds, but actually getting things done.’
A passion for changing systems
Malamud has also been a prolific writer. He is the author of nine books, including “Exploring the Internet,” all composed in long hand on paper.
His writing caught the eye of John Podesta in the early 1990s, who was working for President Clinton and figuring out how to move from paper to digital archiving.
“Carl and I had a passion for [the idea] that public records should be public and electronic records should be preserved,” said Podesta. “Carl was both a pioneer and advocate for the power of the net as a democratic tool.”
Podesta said Malamud was a force on Capitol Hill trying to shape legislation, and when he started the Center for American Progress, in 2003, Podesta hired Malamud to be chief technology officer of the progressive think tank. “Carl is friendly and funny, but what really makes him effective is that he’s dogged and passionate. He wears that on his sleeve,” Podesta said. “He just gets right to the point, and I really admire that in him.”
From pushing for access to material from the Smithsonian Institution to the House of Representatives , Podesta said his single-handed influence is clear. “He’s really changed systems,” Podesta said. “He just won’t accept the status quo.”
Podesta said Malamud has had the most impact going right to the source of the data, trying to convince the entities to put information in the public domain.
“It’s extremely valuable in a democracy to make sure that people have not just theoretical access, but real access,” to information, Podesta said. “Oftentimes, the burdens are either bureaucracy or ridiculous charges to get public documents. No one challenges that, but Carl does.”
A battle for the ages
In 2007, Malamud started Public.Resource.Org, based in Sonoma County, California. He has 18 people on contract and numerous collaborators, and works with a dozen pro bono law firms to advance the mission of the nonprofit. The organization operates with a grant from Arcadia (a charitable trust of Lisbet Rausing and Peter Baldwin) and donations from individuals. He appeals to players across the political spectrum with a variety of tactics: writing letters, making speeches, talking to officials in person, and, when necessary, filing lawsuits to challenge claims of copyright.
Recently, Malamud had a big win with a U.S. Supreme Court case (Georgia v. Public.Resource.Org) after he posted the Official Code of Georgia and was sued for copyright violations—a decision that has had a ripple effect across the country. For nearly a decade, he’s been embroiled in a legal fight to put building, electrical and other public safety codes with the force of law online.
“I look for things that should be available and are not,” Malamud said, then simply lays out why information should be free with clear, defensible reasons. “You have to have a story that makes sense.”
Malamud has worked at this cause like no one else, determined to make sure the public realizes what’s at stake when powerful people are concealing the world of knowledge, said David Halperin, a Washington, D.C., attorney. Halperin was with the Clinton administration and has been counsel to Public.Resource.Org since 2012. “He puts it on them to have to explain why their special interests are more important than global progress and democracy,” he said.
Halperin said Malamud is effective because he is relentless and shares his infectious love of democracy. “And, he is willing to be the person who, when everyone else says, ‘Shut up and get along,’ says: ‘No, this still isn’t right. I’m not going to be cuddly here. It’s time for me to be the moral voice, to be the energy in the room that says, Okay, everyone else may now feel it’s time to be collegial. I feel like it’s time to be just.’”
Corynne McSherry, legal director at the Electronic Frontier Foundation, has represented Malamud in several cases and said he knows how to adjust his strategy to persuade others and be creative in his messaging.
“He tries to help people understand what it is he’s up to, because it’s not always clear to everybody,” McSherry said. “When you can’t see the world that the person is building towards, that person has to imagine it for you—and that’s the thing he does.”
Since Malamud was involved in the early days of the internet, he embraced the potential promise of the technology to open up knowledge, McSherry said.
“We live in a nation of rules, and we should have the ability to actually know what they are,” McSherry said, although for a long time those rules were only available to experts with special access. “That changed. Pulling our governmental structures and all our laws into the 21st century is not a small task, but that’s what Malamud took on.”
Drawing inspiration from history
To make his case in the court of public opinion, Malamud has used humor and tapped into his artistic side. He produced a video about making building and electrical codes open, “Show Me The Manual,” and a short movie about his philosophy, “Open Access Ninja.” He speaks at conferences and universities, tailoring his message to attorneys, government workers, students, or fellow open advocates to advance his cause. The Internet Archive hosts a collection of his videos, texts and other materials online, as well as FedFlix, which includes government films Malamud uploaded and curated.
Malamud has expanded his efforts internationally, working with organizations in India to scan government and cultural information. His Public Library of India collections on the Internet Archive are some of the most popular India resources on the net.He’s become an Indian food expert, of sorts, too, said McSherry, and often expresses his gratitude to her and other attorneys working on his behalf by gifting them with Indian spices.
Since he began working in this space, Malamud said he’s encouraged to see more forward thinking about open data. Still, barriers exist. Most often, he said, he’s up against money and control. While Malamud said he’s making inroads in the power struggle, he said it’s “sort of Whack-A-Mole” with every win followed by another challenge popping up.
When he needs a little inspiration himself, Malamud said he reads from his library of writings from early American feminists and civil rights leaders. Sometimes he quotes Martin Luther King Jr. (“Change only comes with continuous struggle”) or Gandhi (“A public worker has to learn to endure with fortitude.’)
A recurring lesson he’s gleaned from others in history who had fought against the establishment: “You can, in fact, change the way the world works—but you have to be patient. It takes time.”
This year’s Library Leaders Forum kicked off on October 12 with news of promising research, digitization projects and advocacy efforts designed to best shape the library of the future.
The virtual gathering also called on participants to take action in sharing resources and promoting a variety of public interest initiatives underway in the library community.
Watch session recording:
Chris Freeland, director of Open Libraries, moderated the first event of the 2022 forum with librarians, policy experts, publishers and authors. (A complete recording of the virtual session is available here) The second session will take place Oct. 19, live in San Francisco and via Zoom starting at 7 p.m. PT. (Registration is still open).
Libraries have a vital role to play in educating citizens, combating misinformation and preserving materials that the public can use to hold officials accountable. To help meet those challenges, Internet Archive Founder Brewster Kahle gave a preview of a new project: Democracy’s Library. The vision is to establish a free, open, online compendium of government research and publications from around the world.
“We have the big opportunity to help inform users of the internet and bring as good information to them as possible to help them understand their world,” said Kahle, who will launch the initiative next week and invited others to join in the effort. “We need your input and partnership.”
The virtual forum covered the latest on Controlled Digital Lending (CDL), the library practice that is growing in popularity in the wake of pandemic closures when physical collections were unavailable to the public. Freeland announced the 90th library recently joined the Open Libraries program, which embraces CDL as the digital equivalent of traditional library lending, allowing patrons to borrow one copy at a time of a title the library owns.
As librarians look for ways of safeguarding digital books, Readium LCP was highlighted as a promising, open source technology gaining popularity. Participants were encouraged in this same space to spread the word about the advocacy work of the nonprofit Library Futures, and recognize many authors who have recently offered public support for libraries, CDL and digital ownership of books.
Lila Bailey reported on an emerging coalition of nonprofits working on a policy agenda to build a better internet centered on public interest values. A forthcoming paper will outline four digital library rights that without which it would be impossible to function in the 21st century. They include the right to collect, preserve, lend and access material. This encouraging collaboration is the result of two convenings earlier this year, including one in Washington, D.C. in July.
CDL Community of Practice
A panel at the forum discussed projects within the CDL community of practice.
Nettie Lagace of the National Information Standards Organization gave an update on an initiative, funded by the Mellon Foundation, to create a consensus framework and recommendations on CDL. Working groups are focused now on considering digital objects, circulation and reserves, interlibrary loans and asset sharing. Public comments on the draft will be welcome in the coming months, with a final document likely released next summer.
Amanda Wakaruk a copyright and scholarly communications librarian at the University of Alberta, announced a new paper exploring the legal considerations of CDL for Canadian libraries. She is one of the co-authors on the research, along with others in the Canadian Federation of Library Associations. The preprint is available now and the final paper will be published soon in the journal, Partnership: The Canadian Journal of Library and Information Practice and Research.
Working with Project ReShare, the Boston Library Consortium is leveraging CDL as a mechanism for interlibrary loan. “BLC really believes that CDL is an extension of existing resource sharing practices, both in the legal sense–the same protections and opportunities afforded to interlibrary loan also apply to CDL,” said Charlie Bartow, executive director, “but, also in a services sense–that existing resource sharing systems and practices can be readily adapted to include CDL.”
Also, speaking in the session was Caltech’s Mike Hucka. He described efforts on his campus to provide students with learning materials when the pandemic hit by creating a simple model they named the Digital Borrowing System (DIBS).
In Canada, a large digitization project is underway at the University of Toronto, where 40,000 titles in the library’s government collection are being scanned and made available online for easier public access.
Take action
In the final segment, Freeland announced that Carl Malamud is the recipient of the 2022 Internet Archive Hero Award for his dedication in making government information accessible to all. Malamud will receive the Hero Award onstage at next week’s evening celebration, “Building Democracy’s Library.”
Freeland concluded the event with a final call to action: To join the #OwnBooks campaign. People are encouraged to take a photo of themselves holding a book they own that has special meaning, perhaps something that has influenced their career path or has sentimental value. As the Internet Archive fights for the right for libraries to own books, this is a chance to bring attention to the issue and build public support.
Carl Malamud, founder of Public.Resource.Org and a champion for making government information accessible to all, will receive the 2022 Internet Archive Hero Award. He will be presented the award at next week’s evening celebration, “Building Democracy’s Library.”
The Internet Archive Hero Award is an annual award that recognizes those who have exhibited leadership in making information available for digital learners all over the world. Previous recipients have included librarians Kanta Kapoor and Lisa Radha Vohra, copyright expert Michelle Wu, the Biodiversity Heritage Library, and the Grateful Dead.
This year, the Internet Archive is honoring Carl as a tireless advocate for free access to government information. Some highlights of his work include:
In the early days of the internet, Carl was a pioneer in pushing for public materials to be available online. Over three decades, he has digitized and uploaded thousands of documents from Congressional hearings, government films, and worked with the executive branch to shape public policy on information sharing.
He is to thank for EDGAR (Electronic Data Gathering, Analysis, and Retrieval system) Online, the free Securities and Exchange Commission database of corporate information and putting the database of U.S. patents on the internet.
Carl is relentless in his ongoing quest to have detailed codes for buildings, product safety, and infrastructure available to the public on the internet.
He founded Public.Resource.Org, a nonprofit based in California in 2007. Several contractors and pro-bono attorneys work with him to unleash public information from behind paywalls—sometimes landing him in court to defend his actions, all done in the name of the public good.
Carl is known as a dedicated, passionate, principled individual whose creative strategies—and, at times, dose of humor and flair—have fueled his success in opening up access to public knowledge.
Carl has been a supporter of the Internet Archive since its inception. Much of his work appears in the Internet Archive collection including his book, “Exploring the Internet,” a movie, Open Access Ninja, about his philosophy with Public Resource.org and a video, “Show Me the Manual,” about making building and electrical codes available.
A new digital rights management (DRM) technology that is open source—and embraced by publishers—is gaining traction in the library eBook world.
Readium LCP was developed five years ago to protect digital files from unauthorized distribution. Unlike proprietary platforms, the technology is open to anyone who wants to look inside the codebase and make improvements. It is a promising alternative for libraries and users wanting to avoid the limitations of traditional DRM.
“It’s important to have a decentralized, open source system for lending and vending eBooks,” said Brewster Kahle, Internet Archive founder. “LCP is a new generation of software protection that is proving popular with both libraries and publishers.”
LCP is a flexible, vendor-neutral, low-cost solution against over-sharing of content for eBooks, as well as audiobooks. The codebase is open source with the exception of an algorithm that protects the files.
“LCP was developed in conjunction with publishers to make sure it would meet their criteria to safeguard the content of their books,” said Brenton Cheng, senior engineer at the Internet Archive. “Yet, it’s an open format, and not tied to one particular company or commercial entity. In that spirit of openness, it’s available to anyone who wants to protect their content.”
A number of leading publishers, libraries and book distributors have adopted LCP, including:
HarperCollins integrated LCP into its Harlequin Plus subscription service.
Academic publisher John Libbey Eurotext has adopted LCP for its 2022 publications.
Stockholm Public Library has incorporated LCP into its Bibblix mobile app for young readers.
Numilog has deployed LCP for more than 500,000 eBooks in French & English.
BiblioVault adopted LCP in 2021, serving more than 90 scholarly presses & 40,000 books.
The Palace Project has integrated LCP into its mobile apps.
It’s a simple system that allows readers to access eBooks and audiobooks—and does not limit the selection of titles from a single source (as with Amazon or Apple).
It offers a large freedom in the choice of a reading solution, keeps intact the accessibility of digital publications and does not leak personal data, says Laurent Le Meur, chief technology officer, with EDRLab, the open source software development laboratory which develops LCP and receives funding from publishers, eBooks distributors, libraries and public bodies.
With LCP’s structure, there is no need to go through a third-party source to be authorized to download a protected book. Therefore, there is no threat of personal information being compromised. LCP is interoperable by design and socially engineered to be a sustainable, nonprofit DRM solution.
“Open source technologies like LCP protect authors and their works,” said Maria Bustillos, editor at The Brick House Cooperative, a publishing platform designed, owned and operated by journalists. “As a publisher committed to preserving traditional library rights, The Brick House looks forward to exploring the integration of LCP into our forthcoming projects.”
As a new technology, LCP is being used around the world with Europe and Canada leading the way. For organizations working on accessibility, LCP is the natural solution they have been waiting for, said Le Meur. In 2025, the EU Accessibility act will require all distributors of digital publications to offer accessible services and LCP is a DRM format that complies with the mandate.
“LCP is appealing because it’s not locked,” Cheng said. “There’s a greater sense that it might last. It has more transparency and accountability because the source code is out there and available for anyone to see.”
Case Library and Geyer Center for Information Technology, Colgate University. Photo credit: Colgate University Office of University Communications.
From 1970 to 2004, Colgate University amassed as many as 1.5 million microfiche cards with documents from the U.S. federal government.
The small, private liberal arts institution housed the collection in a central location accessible to the former reference service point and the circulation desk in Hamilton, New York.
“Every single campus tour that goes through the library walks past this collection. Our well meaning student ambassadors would announce ‘Here’s our microfiche that no one uses,’” said Debbie Krahmer, accessible technology & government documents librarian at Colgate.
Since the popularity of the miniaturized thumbnails of pages waned several years ago, many libraries have struggled with what to do with their microfiche collections, as they contain important information but are difficult to use.
Krahmer was looking for ways to offload the materials and discovered the Internet Archive would accept microfiche donations for digitization. It was a way to preserve the content, make it easier for the public to access, and avoid putting the microfiche in a landfill.
“These government documents are meant to be available and accessible to the general public. For many there’s still a lot of good information in this collection,” said Courtney L. Young, the university librarian. “While the microfiche has been stored in large metal cabinets on the main level, many of our users do not see them. This project will improve that visibility and accessibility.”
About the donation
In July, the Internet Archive arranged for the twelve cabinets of microfiche, each in excess of 600 pounds, to be loaded onto pallets and shipped to the Internet Archive for preservation and digitization. Materials include Census data, documents from the Department of Education, Congressional testimony, CIA documents, and foreign news translated into English.
Microfiche cabinets ready for shipping to the Internet Archive for preservation and digitization.
Colgate also gave indexes of the microfiche that will be “game changers” for other government libraries once they are digitized because the volumes are expensive and hard to acquire, Krahmer added.
Krahmer said the moving process with the Internet Archive was easy and would recommend the option to other librarians.
“This is a lot easier than trying to figure out how to get these materials recycled,” Krahmer said. “In addition to improving discovery and access, this supports the university’s sustainability plan. It’s going to get digitized, be made available online, and preserved. This is win-win no matter how you look at it.”
Public access to government publications
Government documents from microfiche are coming to archive.org based on the combined efforts of the Internet Archive and its Federal Depository Library Program library partners. The Federal Depository Library Program (FDLP), founded in 1813, provides designated libraries with copies of bills, laws, congressional hearings, regulations, and executive and judicial branch documents and reports to share with the public.
Colgate joins Claremont Colleges, Evergreen State College, University of Alberta, University of California San Francisco, and the University of South Carolina that have contributed over 70 million pages on over one million microfiche cards. Other libraries are welcome to join this project.
For librarians who specialize in caring for music collections, it can be challenging to keep up with the latest technology and resources in the profession. The Music Library Association recently helped address this problem by making many of its publications openly available online.
The MLA donated 21 of its monographs to the Internet Archive for digitization and worked with authors to make the material free to the public under Creative Commons licenses.
The new collection of backlist titles includes information on careers in music librarianship and history of the field. It also covers planning and building music library collections, which can be complicated and involve individual creators and small publishers, said Kathleen DeLaurenti, who helped lead the partnership with the Internet Archive in her role as MLA’s first open access editor. There are also valuable materials on music library approaches to technical services—everything from how to preserve music materials to how to bind and catalog them.
“Increasingly in librarianship, we have people who are being tasked to do this work who don’t have a specialized background, especially in smaller organizations, rural places, and public libraries,” DeLaurenti said. “We’re really excited to be able to make this content available to folks who may not have access to professional development in those spaces, and who may be looking for some materials to bolster their training and their own work.”
The MLA has been publishing new research of interest to music librarians since the 1970s and wanted to find a platform to make the information easier to discover, said DeLaurenti, director of the Arthur Friedheim Library at the Peabody Institute of Johns Hopkins University in Baltimore. The Internet Archive provided the open infrastructure to share and leverage the work of the MLA, which is a small organization with about 1,000 members.
While the MLA began with 21 of the monographs, it is working to obtain rights clearance for an additional 20 titles and DeLaurenti hopes the online collection will grow. So far, authors have been excited that the association is making their work available as it increases access for scholars with the potential for more citations of their research.
The audience for the online collection will likely be “accidental music librarians”—people tasked with music library responsibilities who aren’t musicians but are looking for professional development resources in the area, DeLaurenti said, as well as individuals considering music librarianship as a career.
“As libraries are looking at what kinds of open infrastructure is out there and available, I think the work that the Internet Archive has done through COVID has really changed our perception and how they can work as a potential collaborator in that space,” DeLaurenti said. “We hope to continue different kinds of collaborations with [the Archive] in the future.”
Albert Wan ran Bleak House Books, an independent bookstore in Hong Kong, for nearly five years, before closing it in late 2021. The changing political climate and crackdown on dissent within Hong Kong made life too uncertain for Wan, his wife and two children.
As they were preparing to move, Wan packed a box of books at risk of being purged by the government. He brought them on a plane back to the United States in January and donated them to the Internet Archive for preservation.
The collection includes books about the pro-democracy protests of 2019—some photography books; another was a limited edition book of essays by young journalists who covered the event. There was a book about the Tiananmen Square massacre and volumes about Hong Kong politics, culture and history—most written in Chinese.
“In Hong Kong, because the government is restricting and policing speech in a way that is even causing libraries to remove books from shelves, I thought that it would be good to digitize books about Hong Kong that might be in danger of disappearing entirely,” Wan said.
“I thought that it would be good to digitize books about Hong Kong that might be in danger of disappearing entirely.”
Albert Wan, owner of the now-closed Bleak House Books
Hearing that Bleak House Books would be shutting its doors, the Internet Archive reached out and offered to digitize its remaining books. As it happens, Wan said his inventory was dwindling quickly. So, he gathered contributions from others, and along with some from his own collection, donated about thirty books and some periodicals to the Internet Archive for preservation and digitization. Wan said he was amazed at how flexible and open the Archive was in the process, assisting with shipping and scanning the materials at no cost to him. (See Hong Kong Community Collection.)
Now, Wan wants others to do the same.
“There are still titles out there that have never been digitized and might be on the radar for being purged or sort of hidden from public view,” Wan said. “The hope is that more people would contribute and donate those kinds of books to the Archive and have them digitized so that people still have access to them.”
Do you have books you’d like to donate to the Internet Archive? Learn more.
Wan said he likes how the Internet Archive operates using controlled digital lending (CDL) where the items can be borrowed one at time, not infringing on the rights of the authors, while providing broad public access.
Before his family moved to Hong Kong for his wife’s university teaching job, Wan was a civil rights and criminal defense attorney in private practice. Now, they are all getting settled in Rochester, New York, where Wan plans to open another bookshop.
