Storage strategies

A few people have asked me to elaborate on my storage strategy so here it is. Some background information is needed first. I started taking photography seriously in 1998. Buying my first SLR back then was a big thing and working through the lenses and other options. In late ’99, was basically scanning the prints on a flatbed scanner. Quality was amazing, or at least what I thought was amazing until I stumbled upon a very expensive (at that time, I wasn’t making much a month) option; the slide scanner. By skipping the printing stage and scanning directly from film, it delivered unmatched results from anything else unless you’re comparing to a drum scan. Since then, I’ve been scanning almost all my slides and negatives but most of what I shot were from my travels. Film costs a fair bit, especially if you shot transparency film such as Fuji Velvia, Kodachrome or Elitechromes, so the everyday life doesn’t often get captured on film.

So with digital scans dating all the way back to 2000 (the 2700 dpi scans tend to produce on average 14MB LZW compressed TIFF files), storage is key, along with a proper backup strategy. I went with CD-Rs and then DVD-Rs but they never were convenient (not fun, not quick, and laborious process) meant that you will almost never will do it timely enough so I decided to stick with hard drives. They get cheaper and are inherently quicker to access if they’re stuck into a spare machine that can be booted up anytime you needed to.

I upgraded scanners so the file sizes got bigger and films like Velvia and EliteChrome 100 pushed the resolution (grain?) further. Then came my first DSLR, a 300D (my shortest-lived DSLR ownership) and my 20D several months later. So came RAW files, 16-bit PSDs, and the story continues until I hit my 5D Mark II with 21-megapixels, Full HD movies. Backup is getting more important but also important is accessibility.

I needed to backup and offload my main PC but also need to access the “older” stuff occasionally. Leaving your main storage server switched on/off daily or 24×7 wasn’t something I want to do on a daily basis so I got a NAS which I quickly outgrew within a year so that too got upgraded to a 4-bay unit.

I have a storage server with 6x1TB disks. It’s all RAID 1. Mirrored. So I get a net space of 3TB. I have a “_photos” folder shared out that further breaks down by year, i.e. “2001”, “2003”, along with some odd directories. I also have a “_videos” folder where I back up my created DVDs (ISO format) and Adobe Premiere Pro project files. The problem is occasionally accessing stuff from say, 2006. My main PC only stores past 16-odd months so I ended up going to my storage server regularly. So what’s running 24×7 is my NAS box with 4x500Gb in RAID5. That gives me about 1.4TB of disk space. That way, I get that past 2-years on it; i.e. 2008 & 2009 along with my 60Gb music collection on the built-in iTunes server. My IP cameras also record to NAS and they keep the past-7 days recording. I also have imaged backups of all the machines in my home so that if the primary disk crashes, recovery is just a click away. Keeping everything in sync is the unix utility rsync (the windows equivalent is DeltaCopy).

With the growing file sizes, I’ll be moving my storage server to 1.5TB disks by Q4 this year and my NAS will grow to 4x1TB*… one can never have enough disk space!

* That’s one thing I like about my NAS. I can live swap the disks and expand it without powering down and losing my data. I can easily move from 500Gbx4 to 1TBx4 and later to 2TBx4.