data redundancy

[mk162]

So I am working on setting up a really good backup system.  I just installed a RAID 5 with 3 1TB drives, I will be adding a 4th later.

I am testing out backblaze since they include external drives on their service.

I am also going to download Acronis True image for each computer and have it create complete system restore points, Windows 7 backup is ok, but I have some issues with how it works.  Like it's hard to control the size of the backup, it filled a 1TB drive and then was throwing error messages about not enough space.

Does this seem like a good setup?  I thought about setting up another RAID at the house, but for the cost for everything I could use backblaze for 10 years.  Not to mention this way it's totally offsite, not just 15 minutes away.

I still might put in a 3tb drive at the house and put in a VPN tunnel to back everything up there, it just wouldn't be a raid.

[GraphicDisorder]

A lot of the correct way to address this would have to do with the amount of data we are talking about.  But in general here are my thoughts.

The main concern for me on "online" back ups is 2 fold:

Connection or the speed which you can upload/download "ALL" of your data in the event of a drastic issue on site (fire/complete drive failures/flood/etc).  This is a issue 2 ways, what is your upload speed and how long will it take you to upload all of your data and how long will it take daily for it to keep up.  That said will it affect your other needs for your internet connection.  The other side is keep in mind some of these online service throttle the speed, so you could be waiting awhile to put 1tb in the cloud or retrieve it.  If you create gigs a day like we do often it could be tough to keep up with it being able to be uploaded in a reasonable amount of time.  Just something to consider. 

The other issue is how much do I trust someone else with my data.  I wouldn't discount this for a second.  Seriously assume someone is looking at all your data.  True or not, I wouldn't hand someone my data without being ok with them looking at it.  Cause it very well could happen.

Now to address your original concern, yes that is a decent way to say redundant without heavy cost of server/etc. 

[blue moon]

I've run few RAID 5 setups and for the most part they are more trouble than benefit. With four drives, you have four times the chance of something going wrong (actually a lot more than that). While I would love to give up on RAID all together, I've finally settled on mirroring and having to deal with the problems.

What problems you ask? For example:
 in case of a crash some of the cache is not saved and the image gets corrupted.
Trying to recover the array, I've recovered the wrong drive before or even managed to lose the remaining partition by doing something stupid (we all do it, and as Mr Murphy guaranties, at the worst possible time. The added pressure of getting back on line does not help).
I've even had two drives fail within few days before we managed to get the replacement in.
battery on the RAID controller died and it lost the data during shutdown corrupting the array
controller failure, now stuck with two drives that don't work without identical controller and if the unit itself contains the array info, trying to recreate it can wipe everything out.

there's more, but I think you get the point.
I would mirror the two drives and take what's left home for an off site backup.

pierre

[blue moon]

A lot of the correct way to address this would have to do with the amount of data we are talking about.  But in general here are my thoughts.

The main concern for me on "online" back ups is 2 fold:

Connection or the speed which you can upload/download "ALL" of your data in the event of a drastic issue on site (fire/complete drive failures/flood/etc).  This is a issue 2 ways, what is your upload speed and how long will it take you to upload all of your data and how long will it take daily for it to keep up.  That said will it affect your other needs for your internet connection.  The other side is keep in mind some of these online service throttle the speed, so you could be waiting awhile to put 1tb in the cloud or retrieve it.  If you create gigs a day like we do often it could be tough to keep up with it being able to be uploaded in a reasonable amount of time.  Just something to consider. 

The other issue is how much do I trust someone else with my data.  I wouldn't discount this for a second.  Seriously assume someone is looking at all your data.  True or not, I wouldn't hand someone my data without being ok with them looking at it.  Cause it very well could happen.

Now to address your original concern, yes that is a decent way to say redundant without heavy cost of server/etc.

very good points! I've been contemplating offsite backup and trying to decide which route to go. Based on this it makes more sense to backup to the house computer as everything can be back on site in an hour. Downloading GB's of info would take significantly longer than what it would take to drive home and get the drive.

thanx!

pierre

[mk162]

with the internet speeds we have here, upload/download isn't a problem.  Most of our art files are vector, so there isn't that much size to the files.  I uploaded everything in about 12 hours.  Download would obviously be much faster. 

Also, if i had a complete and total loss of all data here, chances are I wouldn't care about download speeds since I would probably have to replace most computers and equipment here anyway.

With backblaze, if you use a passphrase they tell you to not lose it, because even they won't be able to retrieve your data.

With the RAID drives, I went with enterprise grade.  With a fail rate at around 1.4 million hours.  I was thinking of going raid 10 when I get the next drive, that is slick, FYI.

[blue moon]

With the RAID drives, I went with enterprise grade.  With a fail rate at around 1.4 million hours.  I was thinking of going raid 10 when I get the next drive, that is slick, FYI.

FWIW, many (but not all) of the RAID controllers I had problems with were enterprise grade ($3k per controller). . . Yes their rate of failure when up and running was lower, but part of the problem is in the system you are using, nut just the controller. For example, if the power supply fails your array is going to crash (as will a regular drive), but recovering a crashed array is more difficult than recovering a single drive.

In the end, chances are you will not have any problems, but I felt compelled to throw my $0.02 in!

pierre

[GraphicDisorder]

Raid's are amazing when they work.  When they break though, almost nothing worse.  Because it's not always fixable and you will likely lose data. 

[ScreenFoo]

I'd still +1 Brandt's point on security--Maybe they can't get your passphrase back, maybe they say that to cut down on idiot customer service calls. 

Not sure if you have any local and friendly Linux wizards, but you can set up heterogeneous RAIDS in a dizzying array of configurations, and it's quite transparent--i.e. you can still 'turn off' the RAID and access each drive separately with a normal ext3fs like a single drive user would, and possibilities with integration on just about any type of network is unparalleled.

Probably won't help, sounds like you already dropped the coin on something pretty fancy, although it may help someone else considering it.  Then again, those Linux gurus don't come cheap either.   

[mk162]

i went with the advice of my IT guy that sets these up at other places.  We had a server and he talked me out of that thing.  It was overkill for us.  Heck, it cost around $50 a month to run it.

This setup is a lot simpler. 

[ScreenFoo]

Why did it cost you $50 a month to run?

[mk162]

i checked the power usage of the server and the cooling fans for it.  Servers are energy hogs.

[Zelko-4-EVA]

what did you measure the power usage from?

was it the rating of the power supply?

my home server only uses about 125-150 watts even though it has a 450 watt power supply. 

it has 5 drives connected to the motherboard. 

i measured with a device called "kill a watt"

http://www.p3international.com/products/special/p4400/p4400-ce.html

[mk162]

I measured it with a killawatt, then remeasured after installing the new setup.  I am shaving off about 350 watts, not including the savings in cooling cost.  I could have installed server on the system that is in there, no biggie, but I would have had to buy a new/used copy.  I am happy with the peer to peer, wndows 7 p2p is actually pretty darn good.

[GraphicDisorder]

i checked the power usage of the server and the cooling fans for it.  Servers are energy hogs.

Sounds to me like you had something way over kill.  My server is about as big as a tall coffee cup and draws near nothing for power. 

[mk162]

it was overkill, that is why we dropped it.  the nice thing was it came with server installed...so no extra cost there.