A post in the series about how the Internet Archive is using AI to help build the library.
Freely available Artificial Intelligence tools are now able to extract words sung on 78rpm records. The results may not be full lyrics, but we hope it can help browsing, searching, and researching.
Whisper is an open source tool from OpenAI “that approaches human level robustness and accuracy on English speech recognition.” We were surprised how far it could get with recognizing spoken words on noisy disks and even words being sung.
For instance in As We Parted At The Gate (1915) by Donald Chalmers, Harvey Hindermyer, and E. Austin Keith, the tool found the words:
[…] we parted at the gate,
I thought my heart would shrink.
Often now I seem to hear her last goodbye.
And the stars that tune at night will
never die as bright as they did before we
parted at the gate.
Many years have passed and gone since I
went away once more, leaving far behind
the girl I love so well.
But I wander back once more, and today
I pass the door of the cottade well, my
sweetheart, here to dwell.
All the roads they flew at fair,
but the faith is missing there.
I hear a voice repeating, you’re to live.
And I think of days gone by
with a tear so from her eyes.
On the evening as we parted at the gate,
as we parted at the gate, I thought my
heart would shrink.
Often now I seem to hear her last goodbye.
And the stars that tune at night will
never die as bright as they did before we
parted at the gate.
All of the extracted texts are now available– we hope it is useful for understanding these early recordings. Bear in mind these are historical materials so may be offensive and also possibly incorrectly transcribed.
We are grateful that University of California Santa Barbara Library donated an almost complete set of transfers of 100 year-old Edison recordings to the Internet Archive’s Great 78 Project this year. The recordings and the transfers were so good that the automatic tools were able to make out many of the words.
The next step is to integrate these texts into the browsing and searching interfaces at the Internet Archive.
Hmm… I don’t know that the example transcription provided here really “approaches human level robustness and accuracy”, at least on sung lyrics. Here’s a human transcription (by me) with differences from the Whisper version highlighted.
> AS we parted at the gate, I thought my heart would BREAK
> Often now I seem to hear her last goodbye
> And the stars that SHONE that night will never SHINE as bright
> As they did before we parted at the gate
> Many years have passed and gone since I went away ONE MORN
> Leaving far behind the girl I LOVED so well
> But I WANDERED back once more and today I passed the door
> Of the COTTAGE WHERE my sweetheart USED to dwell
> OH, THE ROSES BLOOM AS fair, but HER FACE is missing there
> I hear a voice repeating “You’re TOO LATE”
> And I think of days gone by WHERE THE TEARS FELL from her eyes
> On the evening as we parted at the gate
> As we parted at the gate, I thought my heart would BREAK
> Often now I seem to hear her last goodbye
> And the stars that SHONE that night will never SHINE as bright
> As they did before we parted at the gate
Also, here’s the initial 90 seconds which Whisper ignored:
> On a clear September’s night, while the moon was shining bright
> I stood chatting with a maiden most divine
> As we watched the stars above, I revealed to her my love
> And she quickly drew her little hand from mine
> All the world grew dark to me when she said it ne’er could be
> “Alas, Jack, you have told me this too late”
> So we spoke our last goodbye, and the tears fell from her eye
> For she loved me as we parted at the gate
Still, though, this is pretty impressive and I hope OpenAI comes up with an improved version which will be able to get closer.
Incidentally, I found scans of some sheet music for this song, based upon which I can make some further corrections to my own human transcription:
– and the tears fell from her eye → and a tear stole from her eye
– where the tears fell from her eyes → when a tear stole from her eye
– I wandered → I’ve wandered
– her face → a face
This is astounding! thank you.
And, whisper is not as good as you are, clearly.
Coincidentally, as I was just about to send this post my town just had a blackout. I am writing in the dark. I have tried reaching out to your media department via email, fb, and Twitter. Apparently, none are working… I hope your AI/bot is American built and better than that by approving this post. Thanks!
You’d think an AI would have the sense to search for sheet music, but it just goes to show what some say: software identified as “artificial intelligence” is often something simpler and limited, just pattern recognition and probabilities.
Well done…good to have the shortcomings of AI demonstrated once again.
I hope this gets video support too! My 2 biggest wishes from the internet archive is “full text” search for the wayback machine & video transcription via ai for the deaf! Amazing Work!
Yes, those would be great. we are chisling away slowly on those. but only small bits.
This post just got this comment submitted– it appears to be a chatGPT entry, so I did not approve it. But a first for me, so I am posting it here. What a world.
Smart Tech Tune
smarttechtune.infox
smarttechtune.info@gmail.com
220.152.114.188
AI@IA is a cutting-edge project that aims to extract the lyrics from 100-year-old 78rpm records using artificial intelligence. These records often contain precious vocal performances that are difficult to decipher due to deterioration over time. Leveraging advanced AI algorithms, the project utilizes machine learning techniques to analyze the audio and extract the sung words, preserving the historical and cultural significance of these recordings. By unlocking the lyrics from these century-old records, AI@IA offers a unique opportunity to rediscover and appreciate the vocal performances of the past, providing valuable insights into the evolution of music and language over the years.
This is extraordinary. Thank you sincerely for all the preservation work (and ancient media in particular)!