2007-05-01

Amazon S3 for media backup

I've been concerned about backing up my videos and pictures for as long as I've been taking them, but until a week ago I never bothered to think much about a commercial service. I would periodically burn DVDs, multiple DVDs of the same data- worried that the DVDs would degrade after a few years. I'd also make hard drive backups and then store the hard drives and some of the dvds offsite.

DVDs are poor for backup as soon as a platter is full of them- it takes a lot of time to find specific files later on. Hard drives are better, but expensive. Recently I had two hard drives fail in a row- they still sort of work, I can recover data off them, but they don't work as boot drives and were at times making clunking noises.

Another feature I've been in need of is a way of transferring videos to people, or hosting high quality videos. Youtube/vimeo/google-video/etc. is low quality and I don't necessarily want to give the videos to the entire world.

I'd heard about Amazon's S3 service a while back, but it doesn't have a friendly user interface by default- and I wasn't sure about paying for third party software like jungledrive, and I didn't want to pay another service like that something-else-drive. I've bought things from Amazon before, they have my credit card information already, and I like their pricing scheme. Other services want you to pay an excess- $10 a month for 100 GB, great deal if you fill up that 100 GB immediately, but poor if you only use 10 GB at first.

Then I found S3 firefox plugin. I can't vouch for the security of this plugin, but it seems to work and the media I'm backing up isn't that security critical. The plugin allows simple uploading and downloading and access control setting of files.

Over a cable modem, it took me about 3-days of uploading to get 10 GB of pictures onto S3. This is slow, but once they're uploaded the rate at which I take new pictures is much lower than backing up several years of pictures all at once.

At they're current pricing this 10 GB will cost $1 .50 a month, and I anticipate they'll lower the price per GB as storage prices fall. In the next week I'll probably have 20 GB up, so $3 a month or $36 a year. If I was just storing pictures this would start to compare poorly with flickr or other photo sites, which charge a flat yearly rate for 'unlimited' storage- but I like the raw interface for uploading and the truly unlimited feeling of S3, and I also don't plan on having too many GB of just pictures.

Video is the real hard-drive killer, and unfortunately S3 isn't going to be a good solution for raw off the camera DV or HDV or other kinds of video. The files are too big: S3 file limit size is 5 GB also, though I can chop up some of the bigger video files, it takes too long to transfer, and it will cost too much. I could easily have a terabyte of video pretty soon. I suppose it's not all worth backing up, maybe if I keep my S3 total under 50 -100 GB I won't mind the cost too much.

For now, I'm just backing up editing compressed video- wmv files under 100 MB. I might progress to backing up less heavily compressed files later. It's easy to share the files with others, set them to global read-only and email people the link- or if the third party has an Amazon account and gets Firefox + the S3 plugin going you can specifically allow them and only them to have read or even write access.

1 comments:

Jakob said...

You can post videos privately to Vimeo, and we keep the original version. So maybe we're perfect for storing your compressed WMVs privately, even if we haven't made that clear on our site.