The black box
I work for the CBC Digital Archives website (I think it’s OK to say that, right?)
Anyhow, we process a boatload of audio and video. English and French combined, there are more than 12,000 clips online already, with at least 400 more hours in the works in English alone. And since we’re redesigning the site to accomodate bigger, better quality video, we thought we’d better upgrade our storage capacity.
The solution, for the moment, is a black box like the one shown here. To be precise, it’s a LaCie 2.5 terabyte Biggest S2S RAID storage tower. To be impresice, it’s an external hard drive about the size of a shoe box. It weighs a ton, and holds an inconceivable amount of data.
A terabyte is 1,000 gigabytes. So this shoe box holds 2,500 gigs - 10 times the amount of data my reasonably new home computer can store. We figure that’ll hold all the high-res videos we are encoding this year.
Which made me wonder how much consumer-level media a drive like this could handle.
So I saved a typical, decent quality YouTube file - in this case, Johnny Cash’s Hurt video (and yeah, I had a little cry first.) It runs about four minutes, and takes up a little over nine megabytes.
Math time:
- 4 minutes = 9 MB
- 1 hour = 135 MB
- 1 day = 3,240 MB (3.24 GB)
- 1 year = 1,183 GB
So… if I were to film myself 24 hours a day in YouTube quality Flash video, I could store a two full years of my life - every second of it - on this one drive.
(I’d upload it, but it’d take you two full years to watch… you might want to fast forward to the good bits. Heck, I’d like to too.)
Yes, I’m playing fast and loose with numbers here - significant rounding of digits, and the drive can’t hold the full 2.5 TB, and Flash video sucks, but you get the point.
Now, how much audio would it hold?
Well, my 15 GB iPod is less than half full (5.6 GB) with 1,567 songs. According to iTunes, that’s enough music to play for 4.2 days without repeating.
If I were to somehow hook up the black box to my iPod, I could hold 446 times as many songs - almost 700,000 songs, enough to last 1,873 days. That’s five straight years of 24/7 music without hearing the same song twice.
Might want to bring the charger, though. Can I borrow some CDs?
One final thought:
Five years ago, the biggest available hard drive for a typical PC was 120 GB, and cost $750. (Our laCie costs around twice that, which would have got you 240 GB in 2002.) Today, that money gets you 10 times as much storage. So it’s reasonable to assume that five years from now, the same drive will hold more than 25 TB.
Meaning that for a couple of grand, you could record every second of your life - from birth to death - on video, and store it in one small black box.
Posted by: Paul Gorbould | 08-08-2007 | 01:08 AM
Posted in: Teh Internets
Loading ...
Paul,
Just to give you some perspective… The storage device that holds all of cbc.ca’s online content (we have another one for off line content) is about 4Tb. Of which, only about 500GB is media. (Real media compresses nicely)
That’s a lot of HTML and images.
Related: http://news.bbc.co.uk/2/hi/technology/6287126.stm
I’d kindof take issue with what he considers to be human experience (a blog and a video feed? really?), but it’s still a good read.
Not so related: http://www.youtube.com/watch?v=57ta7mkgrOU - SFW but you probably shouldn’t show it to kids.
Hey Paul, All those lines SciFi author Charlie Stross wrote in May “shaping the Future”
http://www.antipope.org/charlie/blog-static/2007/05/shaping_the_future.html
enjoy
how many of those things do you have? do you have a plan to digitize and make available everything? or only little clips (far more interesting to make everything available).
Probably the stupidest question ever asked, but I’m genuinely curious:
Did this technology originate from the black boxes stored on aircraft?
Thanks to everyone for their comments!
Blake: Wow, 3.5 TB of text/code/images is indeed impressive. And I’m proud to be responsible for at least one floppy’s worth
Kev: Thanks for that BBC Stoss article - very interesting stuff. But you are right, a handycam recording isn’t the sum total of a life. And I’ve seen that Kermit/Hurt video… very disturbing. But the Henson photo moment is touching.
Ian: Another Stoss link, and very interesting (and Kev has promised to lend me his latest book.) I certainly make no claims on being the first to think about this stuff - I’m probably closer to the end of the line
Still, it amazed me that such technology exists for me, right now - I could start today taping my entire life, and it would cost less than a grand a year at today’s prices.
Hugh: We only have one, for now, but we may need to buy another one next year. Right now the archives site plans call for a focus on whole shows - by the fall, we’ll have at least a dozen programs with a sampling of full episodes up. We’ll never get to everything - there are close to 1 million tapes in the archives. Long term, there is a CBC plan to digitize the entire collection, but for that the issue is manpower, not storage. And rights… I suspect that will determine what gets made public.
Jen: Not stupid at all - I used the term “black box” deliberately, because I picture it used the same way as a flight recorder - except in this case it would be a life recorder. According to wiki, the boxes are orange, not black. And they use magnetic tape or solid state memory chips instead of hard drives. Blake can probably tell us more about this, too.
cool.
Hard drives wouldn’t be the best choice for black box recorders - knock one off a table and it stands a good chance of being b0rked.
The Stross pile-on was funny - hopefully I can remember to bring in the book next week. If you were more of a maths geek than an a/v one I’d lend you The Atrocity Archives instead - picture Lovecraft crossed with Len Deighton with a liberal dose of the BOfH archive.
Ooooh, that sounds like my sort of thing too… Lovecraft and Deighton, check, and now I’ve got some BOfH to read in the meantime. I’m more of a math geek than I let on… I even got called a “math-hole” once.