Author Archives: Brewster Kahle

Thank you, Robert Miller, for 2.5 million Books for Free Public Access

Posted on by

Robert MillerI am both sad and happy that Robert Miller has accepted another position so will be leaving the Internet Archive after 10 years of fantastic achievements. He joined to help create a mass movement of libraries bringing themselves digital by scanning books, microfilm, and other media. He has succeeded in doing this by creating positive relationships and distributed teams, working in 30 libraries in 8 countries, to help libraries go digital.

And thank you to Robert, for building organizational and partnership structures that will continue bring more collections online, long into the future. His endless energy and ability to forge long term relationships to create processes that are both efficient and library-careful have been miraculous to behold. The future looks bright and brighter because of his work.

Working with 1000 contributing libraries, the Internet Archive has digitized and offered free public access to over 2.5 million literary works, we are now on our way to the goal of 10 million books, being served by our sites and the sites of thousands of libraries.

With thousands of libraries serving digital materials in new and different ways to their different communities, we can achieve the diverse but coordinated access and preservation opportunity of our digital age. We look forward to the next steps in the programs that have been started with gusto and relish.

Thank you, Robert. We expect more great things in coming years.

-brewster
Founder, Digital Librarian

Open Source Housing for Good

Posted on by

This is from a talk given by Brewster Kahle,  Founder and Digital Librarian of the Internet Archive, at Commonwealth Club panel titled Open Source Housing for Good on March 9th.  [covered by KQED public radio]

Foundation Housing

Foundation Housing

Our employees are being driven from their homes by rising rents; they are commuting great distances because of the lack of affordable housing; they are living in insecurity because of the fluctuation in rent and home prices.

Internet Archive - Non-Profit Library

Internet Archive – Non-Profit Library

I believe it is becoming harder to attract and keep good people working in nonprofits, including the Internet Archive, because of this problem.

Our employees spend an average of 30-60% of their income on housing. 30-60%.

That is a lot more than the “spend less than 25% on housing” that HUD recommends. Turns out that this is not just our employees, and not just the bay area. According to a Harvard study, the average American renter pays 30-60% of their incoming on housing. Similarly, homeowners pay about the same, except for those lucky few that own their houses outright.

The Bay Area is particularly problematic because rents and house prices have been rapidly rising, which is causing dislocations or people feeling locked into apartments and jobs. Nonprofits are particularly hit because their funding does not rise and fall as fast as the market fluctuations. Further, when the market is down, it is exactly the time you want non-profit services to be strong.

So the Internet Archive, and I would say other nonprofits as well, have an existential problem: affordable and stable employee housing.

The Internet Archive and the Kahle/Austin Foundation are trying a new model to help. Foundation Housing as a name for a new housing class : Permanently Affordable housing for non-profit workers.

In this model, a new nonprofit, the Kahle/Austin Foundation House, has been set up to purchase apartment buildings. These rental units are then made available to employees of select nonprofits at a “debt free” rate– basically equivalent the condominium fee and taxes. Typically, the debt makes up about 2/3 of the cost of a building and the other costs (tax+maintenance+insurance) makes up about 1/3.    Since the employee does not pay the debt part, the monthly fee is now about $850-1000/month rather than $2700-3000 current market rent.   This way, the fee to those employees is about 1/3 of the cost of market rent, and we believe more stable than market based rents.

Walking Distance To Work

Foundation Housing Residents

Currently, this is being tried with an 11 unit apartment building in San Francisco 6 blocks from the Internet Archive. As apartments have become available through normal attrition — we do not force the existing tenants out– the Foundation house has made units available to 2 nonprofits, and there are now 3 employees living there. Having a walking commute, lower housing cost, and a nice neighborhood has been well received.

Roxanna used to commute over an hour each way from Bay View on 3 buses, and raising her 8 year old daughter in a building that had drug dealers actively dealing.  Now she walks 6 blocks to work, pays less, and feels safer.

Michelle is a librarian who was being evicted from her apartment and would have left San Francisco and probably would not be now working at the Internet Archive.

And Samantha worried that her rent was continuously on the rise, thinking she might have to leave the city in a few years, likes that the building is feeling more like a community and less like than an anonymous number in an apartment building.

Having housing provided as part of an employee benefit is similar to faculty housing, military, monasteries, and some hospital housing. But having to leave your apartment upon leaving your job is a negative aspect of this model. We have not seen the effect of this because no one has left yet.

So we think we have a model… but how do we make it permanent, and how do we finance it? To help make it permanent, we are borrowing ideas from the free and open source world and creative-commons licenses.  “Some Rights Reserved” rather than “All Rights Reserved“. “Share and Share Alike” rather than “Get Off My Property”.  With free-and-open-source software, the writer is giving up some of the profit potential in return for increased community participation. In the Foundation House, the supporters are giving up the ability to flip the building for a profit in return for making a permanent asset for the public good.

To finance the creation of these, we have thought of 4 ways, and are trying 3 of them already:

We built a credit union with this idea in mind, called the Internet Credit Union. It has plenty of deposits to start creating Foundation Housing, but alas, the credit union regulators (indirectly controlled by the banks) are not allowing us to make mortgages. This is a sad state of affairs for our nations new credit unions, but is not the subject of this talk.

We have tried the “endowment” approach with the current Foundation House, where we appealed to major donors for an endowment in the form of a building. The attraction is that it is much like an endowment, but instead of having money in a Goldman Sachs account, where they do their magic to make some return, the building-as-endowment is both good deal financially, and helps the nonprofit support their employees.

Beyond this, we would like to look into raising money through a low-interest bond, say for $100 million, to government and local investors, to fund the purchase of these houses, then using market based renters to pay off the bond. This way the buildings would slowly transition into debt-free Foundation Housing.   We have not tried this yet.

Lastly, and maybe most promisingly, there are people that are looking for new answers and participating in conversations like this.  A number of people in the Bay Area are starting co-working spaces and group houses .  When these are being started can be a good time to set up a structure to work off debt and keeping it off — then use the benefits to perpetuate a mission. While still in formation, there seems to be interest from people like Jessy Kate and others.    This could be helped by creating a Foundation Housing License that others could adopt or remix.

With about 10% of all employees in the US working in the non-profit sector, maybe we could hope for 5% of US housing to become Foundation Housing to provide stable, affordable housing for those dedicating themselves to service.

Lets create more debt-free Foundation Housing for non-profit workers!

 

[Other pieces on this]

Burning Brewster’s Bitcoin

Posted on by

[Guest post, hope you enjoy. -brewster]

Burning Brewster’s Bitcoin
First Installment – Coinbase offers a service that is contrary to everything the company professes to hold dear
Internet Archive
Morgen E. Peck

This fall, Brewster reached out to me with a proposition. He wanted to know more about what it’s like moving between bitcoin and fiat currencies—where the trades are happening, which ones are scams and which ones are legit, how long they take to go through, how much of my privacy I have to forfeit, and especially what kinds of fees traders are skimming from each individual transaction. In short, what’s it like for people who have no bitcoin and want to get in? And once they do get in, what options do they have?

To get the answers, Brewster sent me on my way with one bitcoin. He told me to sell it and buy it again in as many ways as possible, and not to come back until I had whittled his money down to nothing.

So. This is the mission. Find out how many licks it takes to get to the center of a bitcoin, or lose it all to thievery and grift (crrrruuunch!!!). We’ll be running updates on my progress through this blog with the hope of informing casual bitcoin users and digital currency gurus alike.

______________________________________________________________________ First Stop: Coinbase

Coinbase is a bitcoin “wallet” (I’ll explain in a minute why I put this word in quotes) merged together with an exchange platform. Most of the people I know who are playing with Bitcoin as a whimsical investment arrived at Coinbase as the first point of entry. I suspect this is because Coinbase accounts link up with external bank accounts, thereby offering an intuitive and familiar interface to the financial infrastructure with which we’re all so well acquainted.

After Brewster sent one bitcoin to my address, I opened a Coinbase account and used the blockchain.info browser-based wallet to dump my funds into it.

Before we even get started, I’d like to note that using blockchain.info is the best experience I’ve encountered so far in this little experiment and I want to hold this transaction up as the ideal that we can use to judge all future stunts. The only better option would be to handle my transactions with a full Bitcoin client.

What I like is that the guys at Blockchain.info have done everything they can to keep their software true to the heart of Bitcoin. I can set up a wallet without giving them my name or email address. The private keys are in my sole possession. Basically, it’s all on

me. If I lose the information that I need to access my account or I let it leak into the hands of a thief, well then I’m flat out of luck and I’ll probably learn to be more careful in the future.

This is what Bitcoin looks like without her makeup on, when she’s dragging herself off the couch to open the door for a package. And, the way I see it, she now has two options. She can either gussy herself up for people or she can try to teach people to accept her for who she is. I advocate the latter (and not merely because I’m wearing sweatpants as I type). I think that the best services will be the ones that leave most of the risk with the users while simultaneously taking pains to tutor them on how to manage key pairs, use cold storage, etc. In other words, part of what’s required in getting this whole Bitcoin thing to work is giving people a new way to understand digital ownership and, in general, just making people smarter. That’s not a bad thing.

Which brings me to Coinbase. As an exchange, Coinbase has functionalities, and therefore responsibilities, that surpass my blockchain.info wallet. It has to operate in conjunction with a world of passwords, bank account numbers and identity verification protocols, many of which are determined by federal regulations. But I still think it’s fair and instructive to ask whether or not the service retains any of the features that Bitcoin the network brings to the table.

What are these features? Coinbase lays out three of the most important ones right on the homepage of its own website. It touts Bitcoin as an open, global network, one which is “not controlled by any company or country,” (that’s #1) with transactions that are secure, “fast and cheap,” (that’s #2) which are processed without the need for collecting sensitive details about the user. “There is no need to give companies extra information or a blank check to bill you” (that’s #3).

Unfortunately, transactions made through Coinbase retain none of these properties. Not a single one. Unlike Bitcoin, Coinbase is a company and when you move your bitcoins to a Coinbase account, you give the company complete control over them. This is because, as I hinted at before, a Coinbase wallet is not a real wallet.

I know that Bitcoin has only been around for 5 years and the community is still in a tug- of-war over semantics. So maybe the term “wallet” is a work in progress. But it shouldn’t be. To me, it’s very clear what this word means. When we talk about wallets in the physical world, we’re talking about something we use to carry our cash around (and all the cursed things that accumulate in a billfold). The important thing about a wallet is that we have access to its contents. At any time we can reach in and pull out the money.

In Bitcoin, the proper analog for cash is the private keys that are used to sign transactions on the Bitcoin blockchain. Private keys are the only thing you really own in Bitcoin, and therefore, any real wallet should give you complete access to them.

Query Coinbase as to how to get your private keys and you will be directed to this message:

As Coinbase is a hosted wallet, we do not provide users with their private keys; doing so would prevent us from taking advantage of our secure cold-storage technology to protect your bitcoin funds.

Instead, you can submit transactions and sign messages using our web-based interface, bypassing the need for control of the private keys.

That pretty much does away with feature number one. Trust Coinbase with your bitcoins and you must trust them completely, because they give you no direct control. This is not a gateway to Bitcoin. It is a surrogate.

On to number two. Transactions processed through the Bitcoin network are fast and cheap. The transaction fees are mere pennies and the transactions themselves usually clear within an hour.

The same is true of a Coinbase transaction if all you are doing is moving money from one Bitcoin address to another. But buying and selling them is another matter completely. Hooking my Coinbase wallet up with my credit union account took days. Once that was settled, I sold my bitcoin across the Coinbase online exchange and waited for the money to land in my checking account. This took another four days, which is longer than I’ve waited when using other services like PayPal or Chase’s QuikPay bank transfer.

The fee was actually not too bad. I sold my bitcoin at $372.62. Of that, Coinbase took $3.88, which is just about one percent.

So, on to number three. Bitcoin is a payment network that eliminates the need for users to divulge sensitive information about themselves. Ownership is verified through strong cryptography that references pseudonyms rather than real-world identities.

This one you can definitely say goodbye to if you start trading on Coinbase and even if you just use their wallet. As I mentioned, the company now knows my name and my bank account number (which they also have the ability to dip in and out of), and my email address. In addition to that, I’ve given Coinbase my phone number in order to set up 2-factor identification. And because they possess the private keys to all of the bitcoins I store in my Coinbase wallet, the company can associate my identity with any transactions they process.

Everything that was attractive about the Bitcoin protocol has been sacrificed to make the Coinbase service user friendly in a way that simulates modern banking and that indulges the dangerous, but well-engrained notion that we are better off trusting professionals to secure our digital information than we would be if we took control of it ourselves.

I’m only picking on Coinbase because it’s the first online exchange I’ve used. I hope to take a look at more of them in the coming weeks and I suspect to find these strategies to be endemic.

But if I were to offer an opinion, I would recommend anyone who has any admiration for Bitcoin—and for what this technology is doing to disrupt traditional payment processors —to go ahead and use Coinbase to exchange between Bitcoin and fiat currencies, but to get in and out as quickly as possible. The fees are pretty low compared to what else is available. But once you start using Coinbase to process transactions on the blockchain, you’re throwing everything beautiful about Bitcoin out the window.

Next up, I hit the Bitcoin ATMs in New York City and the open air trading nights at the Bitcoin Center near Wall Street.

Crusading librarian for openness passes: Cathy Norton

Posted on by

cathy-nortonA live wire in the library field, and a firebrand for openness, Cathy Norton helped keep libraries free and open during this current digitization wave.

Fun and opinionated, we learned that she had the background and evidence to make the bold statements she did–  keep the library materials free and open.

Cathy played a very important role in the development of our Book Digitization project in it’s early years. These were years when the future of book digitization’s growth and it’s public access was not certain. She stood up to the biggest tech companies; she took on publishers, she badgered research libraries to be broader than their local agendas and, at the end of the day, made a difference. Cathy remained contemporary, relevant and vocal up to the very end.

I (brewster) was grateful when I would sail Woods Hole and show up with bags of laundry and a salty demeanor, she would be welcoming and helpful.   Always up for an adventure, she had a firm idea of the world she was trying to build.

On behalf of the Digital Readers everywhere, the Internet Archive would like to want to raise a digital book to Cathy Norton, a champion of open knowledge, a positive force for collaboration and just a truly fun person who was up to take on any challenge related to moving libraries and public access forward. Thank you Cathy for what you helped create!

With celebration and sadness,

The Internet Archive, Brewster Kahle, Robert Miller, and the Open World

Here is the obituary that appeared for Cathy.

With sadness, the MBL notes the passing of former Library Director, Catherine N. Norton, who died peacefully at home after a battle with cancer. Cathy graduated from Sacred Hearts Academy, Fairhaven, MA, Regis College, Weston MA, and taught psychology at Chamberlane Jr. College while at Boston College graduate school more than fifty years ago.   She and her husband Thomas J. Norton moved to Falmouth for the “summer” but never left. She is survived by her 4 children whom she idolized and were with her when she passed, Dr. Margaret Molly Norton, Michael Norton, Kerrie Norton Marzot, and Thomas “Packy” Norton; and her grandchildren: Buddy Norton Estes, Toby Marzot, Drew Norton, Kate Norton, Hailey Norton, Roberto Marzot, and Julietta Marzot.

Cathy was active in community affairs. She served on the Falmouth school committee in the eighties and early nineties as chair and vice chair, was a town meeting member, and most recently represented Falmouth on the Steamship Authority board. She was instrumental in naming the new vessel “Woods Hole” that will be serving the islands from the Mainland.

Cathy lived for her family, friends, fun, faith and flowers. She remained long time friends with classmates from grammar school all the way through graduate school and showed how much she valued their friendship. In her professional life at the Marine Biological Laboratory she helped build international networks that spread digital information freely to countries that needed it from South America, to Africa, to Europe, and all the countries in between. A proponent of open access, she loved to travel to these countries and spread the word about the Biodiversity Heritage Library Project. As President of the Boston Library Consortium she helped form a group of libraries that worked with the Internet Archive to digitize open access books and journals, making them available to anyone with an internet connection.

Cathy had a flair for life, and her tremendous energy and can-do attitude guided her more than 30-year career at the MBL. Cathy came to the MBL in 1980 as a member of the MBLWHOI Library staff and earned a Masters in Information Science from Simmons College in 1984. In 1991, as the electronic frontier began to enhance information access, Cathy embraced change to become the MBL’s first Director of Information Systems. In 1994, she was appointed Library Director and became a leader in promoting the digital library and open access.

During her tenure she spearheaded the development of uBio, a digital biodiversity database that served as a foundation for the Encyclopedia of Life project. She helped develop an innovative Biomedical Informatics course sponsored by the National Library of Medicine designed to enable biomedical researchers and practitioners to embrace the power of technology. Cathy was also a founding member and served as Chairman of the Biodiversity Heritage Library, a worldwide collaboration of libraries and museums making biodiversity literature freely available. In 2011 Cathy retired as MBLWHOI Library Director and was named Library Scholar.

Beyond the MBL, Cathy was a Justice of the Peace for 39 years, marrying many happy couples on the beaches and back porches of Cape Cod.

Everyone who knew her has a “Cathy story” – how she inspired them with a project, connected them with another collaborator, worked her “magic” to make the seemingly impossible a reality, or made them laugh, especially with stories of weddings she presided over as a Justice of the Peace.

The MBL has established an endowed fund in Cathy’s honor, and its flag will be lowered in her memory. The family has requested that in lieu of flowers, please make donations to the Catherine N. Norton Endowed Fellowship at the MBL, www.mbl.edu/research/norton-fellowship.

A memorial service will be held on Saturday, December 27 at 11 AM at St. Patricks church on Main Street in Falmouth.

Declaration to be ‘Defensive’ for the Defensive Patent License

Posted on by

The Internet Archive hereby declares itself ‘Defensive’ by committing to offer a Defensive Patent License, version 1.1 or any later version, for any of its patents, to any DPL User.   The Internet Archive does not have any patents at this time.

Our contact address is:  info@archive.org

-brewster

Founder, Digital Librarian
Internet Archive

 Birthday and Announcement about DPL.

Defensive Patent License: Troll Proofed. Innovation Protected.

Posted on by

Today the Defensive Patent License is officially released.   It is designed to bring free software ideas to the patent arena by encouraging patent owners to declare themselves “defensive,” and share their patents with others that have declared themselves defensive.

defensive-patent-license-logo

This way a large number of patents can be used to help create new products and services without fear of being sued.  As more organizations join in becoming defensive, then the set of patents gets larger and the incentive to become defensive grows.

The Internet Archive hosted the “birthday party” as the license was refined, and declared itself defensive.  Brewster Kahle helped spur this generation of the idea by collaborating with lawyers who worked for years to get this to happen.

In celebration of this release, today John Gilmore is dedicating an important portfolio of patents from Pixel Qi to be defensive.   Pixel Qi was a company run by Mary Lou Jepsen of OLPC fame, and partially funded by Brewster Kahle and John Gilmore.

Please consider joining in by declaring your organization defensive, whether you have patents or not.  The Internet Archive has declared itself defensive to support this effort.

 

 

 

430 Billion Web Pages Saved….Help Us Do More!

Posted on by

141117-BrewsterDear Friends,

Today we launch our End-of-Year Campaign.  Once a year, I ask all of you to keep the Internet Archive going and growing stronger.   Please help us reach our goal of raising $1.5 million by the end of the year.  Your support will help pay for servers, bandwidth and our dedicated staff.

I founded the Internet Archive as a non-profit with a huge goal:  to give everyone access to all knowledge—the books, web pages, audio, television and software of our shared human culture. Forever.

Book Scanning with Table Top Scribe

Lan Zhu, a scanner at Internet Archive, at the Table Top Scribe. Zhu can scan a 300-page book in thirty minutes. Since 2005, the Internet Archive has digitized over 2.4 million books.

Together we are building the digital library of the future. A place where we can all go to learn and explore.

At the Internet Archive, we’ve preserved 430 billion web pages. People download 20 million books on our site each month. We get more visitors in a year than most libraries do in a lifetime. The key is to keep improving—and to keep it free. That’s where you can help us.

For the cost of buying a book, you can make a book permanently available for the next generation. Please consider donating $10, $25, $50 or whatever you can afford  to support the Internet Archive before the end of this year. It’s is a small amount to inform millions. Help us do more. I promise you, it’s money well spent.

Thank you,

Brewster Kahle
Founder, Digital Librarian
Internet Archive

Photos by David Rinehart/Internet Archive

Music Analysis Beginnings

Posted on by

As mentioned in our recent Building Music Libraries post, we are working with researchers at Columbia University and UPF in Barcelona to run their code on the music collection to help their research and to provide new analyses that could help with exploration and understanding.

We are doing some pilot runs to generate files which some close observers may see in the music item directories on archive.org.  Audio fingerprints from audfprint are .afpt and music attributes from Essentia are in _esslow.json.gz (download sample) and _esshigh.json.gz.

Spectrogram of a Grateful Dead track

Spectrogram of a Grateful Dead track

We are also creating image files showing the audio spectrum used.  We hope this is useful for those that want to see if files have been compressed in the past (even if they are posted as flac files now).  There is also a .png for each audio file of a basic waveform that is being used in the archive’s beta site as eye candy.

More as it happens, but we wanted you know there is some progress and you will see some new files.  If you have proposed other analyses that would benefit from being run over a large corpus, please let us know by contacting info at archive dot org.

Thank you to the researchers and the Archive programmers who are working together to make this happen.

 

Using Docker to Encapsulate Complicated Program is Successful

Posted on by

The Internet Archive has been using docker in a useful way that is a bit out of the mainstream: to package a command-line binary and its dependencies so we can deploy it on a cluster and use it in the same way we would a static binary.

Columbia University’s Daniel Ellis created an audio fingerprinting program that was used in a competition.   It was not packaged as a debian package or other distribution approach.   It took a while for our staff to find how to install it and its many dependencies consistently on Ubuntu, but it seemed pretty heavy handed to install that on our worker cluster.    So we explored using docker and it has been successful.   While old hand for some, I thought it might be interesting to explain what we did.

1) Created a docker file to make a docker container that held all of the code needed to run the system.

2) Worked with our systems group to figure out how to install docker on our cluster with a security profile we felt comfortable with.   This included running the binary in the container as user nobody.

3) Ramped up slowly to test the downloading and running of this container.   In general it would take 10-25 minutes to download the container the first time. Once cached on a worker node, it was very fast to start up.    This cache is persistent between many jobs, so this is efficient.

4) Use the container as we would a shell command, but passed files into the container by mounting a sub filesystem for it to read and write to.   Also helped with signaling errors.

5) Starting production use now.

We hope that docker can help us with other programs that require complicated or legacy environments to run.

Congratulations to Raj Kumar, Aaron Ximm, and Andy Bezella for the creative solution to problem that could have made it difficult for us to use some complicated academic code in our production environment.

Go docker!