Yottabytes

As I am in the IT business, with a specialty in Storage Area Networks, I spent time looking at what the NSA facility being built in Utah is being filled with. If you visit AtlantaSANs you’ll understand more about SANs than you probably want to know. Simply put, SANs are specialized servers that store data for use by file servers, not users. You can think of them as external hard drives for the servers. The servers share the pool of raw data stored in the SAN.

If you doubt that the NSA wants to store everything imaginable, forever in this and additional data centers you are fooling yourself. The NSA ultimately wants to store at least one yottabyte of data. You’ve probably never heard of a yottabyte, only terabytes as that’s what is in your PC. As drive prices plunge, capacity will increase.

At this time, the entire planet generates about 2.5 petabytes of data a day or 1 exabyte a year. It’s thought that the NSA already has 5,000 times that storage capacity and is continuing to grow it quickly. I think much less, under 100 exabytes.

A yottabyte is a trillion terabytes. That’s one million, million one terabyte drives! Even with the largest drives available today, it’s still 250,000,000,000 drives. That’s good for the hard drive manufacturers. Who needs the public to buy them, when the NSA will buy everything you produce using taxpayers money?

So why do you think that they want that capacity? Yes, you are correct.

Editors note: July 31, 2013. USA Today reported that our government has confessed to secretly collecting even more data on the American public than reported when I wrote this. Don’t worry, there’s more to come.

They have enough space to store the contents of every single PC in the United States. Or many times of all of the data from every PC on the planet. The next step in the total surveillance program, that puts the former Soviet Union’s to shame, is capturing high resolution video and license plate numbers from cameras that will be placed in as many cities as possible. The rapidly developing facial recognition software will keep track of you, where you are and what you are doing forever. Copies will all be transferred to the NSA facility. All in the name of keeping you safe. From what or whom?

Editors note: July 24, 2013. These photos illustrate how the police are now automatically checking license plates. The information is stored, and almost certainly shared meaning that the images will be copied to the NSA for future use.

It’s common knowledge that if a government agency doesn’t spend its budget allotment, that budget is lowered the next year. The NSA, like almost all agencies will continue to spend to and beyond its budget, by increasing capacity as much as possible.

Our nation’s Fourth Amendment will be gutted by the NSA and other agencies. It will be bypassed, worked around and eviscerated.

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

This data will be used by our government to prove people guilty of crimes by accessing personal information going back years; what they did and when down to the second, where they traveled to, who they met with, what was emailed, who was called including the conversations… everything. Much of it will come from smart phones people carry everywhere.

If it’s you, you will be found guilty as you’ll plea-bargain your way to jail to avoid bankrupting your family. The PBS program, Frontline found that nineteen of twenty defendants plead guilty agreeing to plea-bargains to avoid financial ruin. Only the wealthy or indigent can afford to plead not-guilty.

I suspect that local governments will one day ask the NSA for citizens locations and speed data on roads in their counties. Why? So they can send speeding tickets in the mail. Speed cameras are just the beginning. They need the money. You’ll pay, or else.

FYI, a petabyte is one thousand terabytes, ($95,000 for commercial purchases) an exabyte is one million terabytes ($95 million) and a zettabyte is a billion terabytes, about $95 billion. The government buys far below commercial rates. Commercially, we only quote SANs up to ~100 terabytes, nowhere near a petabyte. It’s believed that the NSA has about 5 zettabytes installed so far, the equivalent of 5 billion one terabyte drives. I think that the estimate is too high, but even 5 exabytes is good for many years, just 1/1000th of 5 zettabytes.

To give you a scale reference, if you lay 5 billion standard 3.5″ drives on the ground they will cover 18,300 acres or over 28 square miles, about 8% of Cobb County’s land area! They are, in fact, stacked on edge from floor to ceiling.

I think that a good proportion of this storage is held using IBM robotic tape libraries. They are very fast and can load requested archival data onto hard drives very fast. They are relatively cheap, compact and reliable.

If you remember the classic Star Wars movies, the character with all the answers was Yoda, close enough phonetically to yotta to make the point.