Behind all the cool stuff users see on archive.org is some serious hardware. I was curious about the ongoing development of data storage here at Internet Archive. I spent a little time with Mario, Master of the Machines, while he gave me a tour of the newest generation of our staff designed and built Petabox storage units.
Here are some of the specs he gave me for the newest version.
• each has 480 terabytes of raw storage
• each Petabox contains: 240 2-terabyte disks in 4U high rack mounts
• each computer has: 2 – 4 core xeon processors, 12 gigs of RAM each, speed-2 GHz
• each machine has pair of 1Gbit interfaces that are bonded so it’s effectively 2Gbit
• the rack has a switch with uplink of 10Gbit
• Ubuntu OS is stored on a pair of mirrored internal hard drives separate from the data disks
• each has IPMI management interface (allows remote control power cycling and remote console)
• in all there will be a total of 8 units (that’s about 4 million gigabytes).