ND – Cold, Blue
Apple has just updated their MacBook Pro range…
…with welcome, though expected, bumps to the processors, RAM and storage capacities. However, the update that I’m welcoming most is the adoption of Intel’s bourgeoning LightPeak technology, rechristened yesterday as ThunderBolt.
I have worked with computers for 29 years now, from a bedroom hobbyist in the early 1980’s to a video game industry veteran in 2011 and (hopefully) beyond. When I say I worked with computers, I don’t just mean I could create a simple spreadsheet, but I was a programmer, and subsequently an Engineer (sounds posher, would have pleased my father). We used to call the language we wrote in “Machine Code”. It too got a posher name in “Assembly Language”, but the knowledge of writing code at very low level, even in Hex in the very early days, is a rapidly dying art today with everyone jumping on the C++ or C# bandwagons.
…There are two fundamental things I learned the best part of three decades ago that still hold true today:
1) Nothing is unhackable!
2) If you don’t backup your data, you WILL lose it.
I still find it amazing that so many people don’t actually backup, but then again I’ve learned the lesson in a harder way than the myriad of casual “computer users” today will when they lose a cellphone JPG of their cat with a funny stare.
I remember the double sided 3” disks that Amstrad used to use in the 1980’s on their CPC and PCW machines, and that on one occasion I turned a disk over to find the directory of the files from Side A had been written over the directory of even more important files on Side B. That was a long day I can tell you. It was a project I had mostly completed after months of work, and yes, you guessed it, this stupid low quality weird format plastic disc from China was my only copy. Dumb or what?
Luckily, projects were much smaller then, the 180Kbyte capacity was divided up into 512Byte sectors, so I only had a jigsaw puzzle of slightly under 360 sectors to stitch back together by examining the text contents and then “poking” (a magical art in the 1980’s) the sector numbers by hand back into the directory structure until I had rebuilt it.
Voila, my data had returned, but you’d be hard pressed to get away with that on a Terabyte Hard Drive today.
I digressed again…
…We constantly hear the stories of computers, accounts, lives being hacked today, and that’s not going away in the near future, however careful we all try to be. However, backing up your data, or rather NOT backing up your data is a stupid fault that most people are oblivious to until that data is lost.
Now I admit that losing your LoLCat collection may not be the end of the world for some, but as a (primarily) digital photographer, who is building up a lifetime of memories and work, the more time and effort I put into my collection, the more of my life I will lose if it vaporises overnight.
So I Backup!
I think I’m quite good at backing up too, though I’m not perfect and there’s certainly room for improvement. Mrs ND thinks I’m mad, and won’t buy me another “Terabyte Box” for Valentine’s Day anymore. However, being the long-experienced computing, ahem, “expert” that I believe I am, I don’t think I’ve quite reached total backup security Nirvana yet.
10 Terabytes just isn’t enough…
…in 2011 in my opinion. Here’s how that works…
Firstly, modern digital SLR cameras are pumping out RAW files at 10-15MB in size. With Image sensors going to higher resolutions as I type, those files will get larger. If Nikon pumps up the pixels for their D700 and D3S/X replacements over the next year or so by the same proportions that they did jumping from the D90 to the D7000, then we’ll be seeing RAW files in the 18-30MB bracket before long.
I have a folder that contains many folders on my iMac. Each time I import RAW files from my Nikon DSLR’s I divide them into batches depending on subject matter and create appropriate folders, 1 per subject. I import these into iPhoto which I use as my one-stop-shop for “holding everything”, however iPhoto has its own database, and is set not to reference my original photos, but to import and effectively duplicate my original data.
I like this duality as my folders hold the data in a place that my 1980’s brain can understand it and get at it when all else fails, yet my iPhoto database not only allows me to keyword up all my images, and find them in virtual groups depending on common subjects, but also offers a backup (of sorts) from the prime risk of being a numpty and accidentally deleting or corrupting my original files.
I say “of sorts” because originally both the folders and my iPhoto Library lived on the same physical hard disc drive in the same partition, so not a safe backup at all really.
However, then something happened, well, two things really. Firstly I realised that having everything on just that one disc was stupid as the iMac could get stolen, burned, flooded, or nuked by someone who doesn’t like the country I live in. Secondly, I ran out of space on that disc. I used to write entire releasable video games in 32Kbytes on a ZX Spectrum, yet now a 1 Terabyte monster was too constrictive for my doubled up photo collection, amongst other things.
So, I got another 1 terabyte drive, a Western Digital MyBook Essential with a USB2.0 port and a bargain basement (at the time) price because my local electrical retailer had not only priced this top of the range model lower than the otherwise identical 500GB device next to it on the shelf, but had confirmed that this was “correct” twice after I kept questioning it and pointing out that they must have got the price wrong.
“No Sir, the older model uses shinier plastic than the new one Sir, that is why the price is more”. Hmmmm, I briefly thought for a microsecond before whipping out my Visa card.
My 1TB MyBook was quickly attached via USB2.0 to my iMac and I triumphantly moved my iPhoto database across to it, leaving my original folders intact on the iMac.
Two drives (1 internal), two terabytes, my data was safe, Yeah!
Ah, actually, not quite. Because I realised that:
1) I held my original folder solution in higher regard than I had previously thought (because if I lost it then I surely wouldn’t be able to retrieve all of my data from this magical iPhoto Library file),
2) I had now put so much time into importing and keywording my photos into this iPhoto Library file that I didn’t want to lose that either.
Yes, you guessed it, whilst my data was now split across two storage solutions in two different pieces of hardware, I felt a compelling urge to back them both up.
Whilst this was all going on, I was still generating new data, lots of new data. A five-week jaunt around Hong Kong and New Zealand (not the least photogenic place on this wonderful planet), culminated in Mrs ND and I producing over 11,500 RAW files from a Nikon D80 and D200. That’s all got to go somewhere, and was more valuable than most of our data as it was our honeymoon.
I’d picked up a second 1 Terabyte MyBook at some point, just to split some of the folders and other data across multiple drives because it was all getting too big, but my storage Nirvana was still out of sight. The truth was 1 Terabyte in a single device was just not cutting it anymore. I needed more storage.
Thankfully Western Digital helped to alleviate this problem by releasing 2 terabyte MyBooks. So I got one. Then, a few months later, I got another!
These one’s were cool because they had little permanent e-ink displays on them to tell me how bad my storage capacity was even when they were off. It felt like Chinese Water Torture.
Then it hit me, or at least my house – English Water Torture. Whilst away one winter, a very cold winter, a pipe burst in my loft and poured water through the house for 2 days until a neighbour managed to get it shut off.
Through complicated life circumstances my data was safe, as it was elsewhere, however it was now clear to me that a hard drive failing or simple human error with file management was not my only fear. Fire, Flood and Pestilence (probably) were now my new worst enemies. I needed an off-site backup.
If you have followed me this far then indulge me a little more. I promise, I’ll get to the point.
If you recall, I had a folder system with all my photos, this system is split across 2 external USB drives because of its size and is then backed-up onto a 3rd USB drive with twice the capacity of either of the first two. Then, I have all of this data sucked into a mystical iPhoto Library that currently stands at almost 700GBs. Yes, I know, it’s a lot. This file is a main working file for me and sits on an external USB drive, with, yes you guessed it, a copy of it on another external USB drive.
BTW, I’d like to stress at this point, that the numbers are mind-boggling sometimes, but on three occasions now due to a dodgy USB hub, and some other random instance, the file system on an external drive has been corrupted forcing me to completely reformat them and (because I backup) copy everything back on that was there before.
I have already saved my data 3 times!
Having all of my drives sitting next to my iMac though was now the weak link, and a burglary, fire or flood could wipe the lot out. So I got another 2 Terabyte MyBook, made a 3rd backup of (almost) everything onto it, and then took it to the office 27 miles away and further inland (so I’m hopeful it’s Tsunami-proof), where it now lives. Once every six weeks or so I bring it home, update it with all of my latest data and then immediately take it back to the office again.
I’ve been told that if I was truly serious about backing things up then I would have 1 more store in a 3rd location some distance away from ground zero and my secondary nuclear bunker…
Apple ThunderBolt Logo, Image (c) Apple Inc.
So what’s this all got to do with Apple/Intel’s ThunderBolt?
1) Copying 700GBs, let alone 2 Terabytes, of data at odd intervals is a pain. It takes a day over USB2.0 and I mean a Day; about 24 full hours to truly fill a 2 Terabyte drive before taking it offsite again.
2) The primary databases that iPhoto and Aperture use on my iMac are NOT held internally but externally and are accessed over USB 2.0 which makes photo editing a little more sluggish than I’d like.
ThunderBolt will be my saviour. To be fair, the MyBook that my primary working databases reside on is actually hooked up to my iMac over Firewire 800 which is a little quicker than USB2.0, however I need a fundamental shift in transfer speeds to make dealing with such data a more pleasant experience.
USB3.0 was starting to look like my future needs would be solved. It runs up to 10 times faster than USB2.0 and about 8 times faster than Firewire 800. USB3.0 is also backwards compatible so that all of my old MyBooks would still be accessible from a new USB3.0 laden computer, albeit at the lower speed, but usable all the same.
Certainly, when my iMac gets renewed in a couple of years, USB3.0 may have been my option if Apple have adopted it by then, and I could have bought a 4-6 Terabyte USB3.0 MyBook for my primary working storage relegating the other 10 Terabytes of cumulative USB2.0 and Firewire800 disc space to backing up only.
At this point, the increases in connection speed and drive capacities would be heavily outstripping my actual digital needs, but now it seems like it’s not stopping there.
ThunderBolt is apparently twice as fast as USB3.0 in its first incarnation, with promises that it could go 10 times higher still in coming years. Thunderbolt is also out there now, with Apple having just adopted it for the MacBook Pro (all flavours) and iMac and Mac Pro upgrades incorporating it can’t be far behind.
The external storage manufacturers will be chasing this market, and I believe Lacie have already bitten with some RAIDed solid state magic now available with Thunderbolt connectivity.
It’s going to be a while before I upgrade my iMac, but if (well, WHEN actually) I buy another MyBook before then, It will come with USB for sure, but also with Thunderbolt alongside it. I’ll enjoy the extra capacity on my current computer for now, but when I replace it then the external drive(s) I’ve subsequently purchased will accommodate this new speed.
Though in 3 years’ time the latest Nikons may be pumping out 60-80MB Raw files, so maybe I’ll need to think again.
Now, where can I use as a third storage site after my home and office?