I outlined my previous setup in an earlier post on the Prairie Rim Images blog. In short, all of the machines in my house get backed up to my desktop workstation / server. It's the big dog on the block, with more storage space than all the others put together. The main drives in that machine get backed up every night to another set of drives in that same computer which are usually off-line during normal operation. If a primary drive fails, I could simply mount the backup drive & continue working.
To protect against theft or destruction of my entire house, I also had two encrypted USB hard drives that I kept at my office across town. I'd alternate which one I brought home each night to sync up with my workstation. Worst case, if my house disappeared in a puff of blue smoke, I'd still have one copy of my data at the office that was no more than a couple days old.
Fast forward to 2012, when my disk usage is growing by about 750GB per year, thanks largely to the RAW+JPEG images from the 18MP camera I use in my photography side job. The state of the art in single-spindle hard drives was barely keeping pace with my off-site storage needs, and my current 3TB drives were 98% full. I could have replaced them with 4TB drives, but that would have lasted me just over a year... and what would I do with the 3TB drives? The near-line backup drives inside my workstation were also full, and the machine had no more empty SATA ports or drive bays to which I could add more drives (like the 3TB ones I had just outgrown in the USB enclosures). In fact, I was still using a small, slow, 7.5-year-old IDE hard drive as my boot disk because I didn't have any free SATA ports to use for a new one. It was time for the incremental upgrades to stop.
I toyed with some more expensive solutions that would have provided plenty of convenience with no additional storage purchases for the next 4 or so years. However, given some other expenses in our lives (like the new house) I couldn't bring myself to shell out that kind of cash just for a little convenience.
The solution upon which I settled is actually a little simpler than my old solution, and only cost me about $400 (down from $1300+ for the more convenient option).
|New Vantec NexStar MX enclosure|
Into each Vantec, I placed one of the 3TB drives from my old USB enclosures, plus one of the 1.5TB drives that used to be combined to form my in in-workstation backup. Striping the pair would have given great performance, but would have capped me at 3TB (double the smallest drive), which is no better than I had with the old setup. I instead concatenated them to create a single 4.5TB filesystem. When that fills up in a couple years, I'll replace the little 1.5TB drives (which will be getting rather old anyway) with something larger. Dual 3TB drives would give me 6TB per enclosure, which should last me another four years at my current rate of increase.
[Update: In October 2014, Vantec informed me that this enclosure will handle dual 6TB drives. They merely update their published specs to match the largest drives available at the time. I'm now running one 4TB and one 6TB drive, giving me 10TB in a single enclosure.]
Removing the old backup drives from my workstation freed up two SATA ports. Previously, a 2-disk stripe (1.5TB + 2TB disks) housed my /home filesystem (filled mostly with photos). I augmented that with a third 2TB disc, giving me a 5.5TB /home partition. Again, that should last me at least three years before I need to add more spindles. Because I'm short on SATA ports and drive bays, I chose to configure the three drives as a stripe (RAID 0) instead of RAID 5. I've already got a couple backups of this data, and nothing I do at home is so critical that I can't tolerate a day of down time if I lose a disk & have to rebuild the filesystem. Besides, a 3-spindle stripe is really bloody fast--on par with a typical SSD drive, but considerably larger and cheaper.
During the last year, I've had to replace one WD and one Samsung (post-buyout) drive. My experience with WD support was wonderful. Seagate's support was a nightmare. You can read all about that in an earlier post. I'm not inclined to give Seagate any of my money any time soon.
In addition, if you look closely at the hard drive specs, Seagate appears to be catering toward the cheap, careless masses, while WD is designing for a slightly more discerning clientèle. Seagate's warranties are typically only 1 or 3 years, and the drives are priced dirt cheap. For the new 2TB drive I put into my workstation, I chose a WD Caviar Black drive, which has a 5-year warranty and costs almost twice as much as the cheapest Seagate 2TB drive. As painful as it was to deal with Seagate customer support, the last thing I want is to make it an annual occurrence.
All that said, one of my USB enclosures now contains a WD and a Hitachi. The other houses a WD and a Seagate. The /home stripe in my workstation is comprised of two WD's (different models) and one Seagate. I think that's sufficiently diverse.
|New Kingston SSD and WD Caviar drives|
The one place in this setup where I do want near-real-time redundancy is for the boot disk on my workstation. I do still run a few public services off this workstation, and even if my /home partition pukes, I still want to be able to log in to correct the problem or mount my USB backup drive instead. By moving my boot disk to a SATA port, I can bring in another newer (but still old) 400GB IDE drive to act as an in-machine warm spare for the boot disk. It'll get synced up nightly whether I tell it to or not, and I can boot from it simply by unplugging the other boot disk.
"Holy cow," you're saying. "Why did you spend $400 and go to all that trouble ferrying drives around instead of just putting your backup on the cloud?"
Fair question. I actually used that system long ago, before I did much photography. Let's do a little math. I currently have about 3TB of data that needs to be backed up, and I add about 750GB of new data every year. On a heavy weekend of photography, I can shoot over 50GB of images. At my current home network upload speed (upload is usually far slower than download), it would take me 7.2 days to upload 50GB of data, and that's if nobody in my house used our Internet connection for anything else during that time. Uploading the initial 3TB would take 434 days. Yes, you can often ship a hard drive to the cloud managers in order to prime the pump, but then you incur the expense of a hard drive. I'd also have to upgrade to a considerably faster business-class Internet connection to make this setup work. My annual cost would then be far more than what I've currently invested in USB drives.
Oh, and don't forget that you'll then have to download all 3TB of this data again if you ever need to restore a drive. That'll take 11 days over a 25 Mbps downlink. How badly to you need to get access to that data? Do you think your file ownership, time stamps, and permissions will be retained during that process? I'm betting not.
I know not everybody has as much critical data as I do. Unless you're into photography or video editing, most heavy users are just archiving movies or things that they can easily re-rip from the original optical media if they lose a disk. The vast majority of home computer users probably measure their critical data in GB rather than TB. For those people, cloud storage works quite well. However, for those heavy users like me, I hope my setup provides some insight into how to backup your own data.
If you've got any more specific questions, or if you'd like to share your own backup scenario, please speak up in the comments below.