Jump to content
RESET Forums (homeservershow.com)
jdell42

DE or RAID for 24-48 TB?

Recommended Posts

ikon

Reading more about software RAID I think its a better alternative in terms of efficiency over DE... and it seems more fault tolerant than hardware RAID in term of using existing hardware and replacement drives / controllers. I really like the ability to access the drives on any PC adn the elimination of RAID build and rebuild times.

 

At this point I really dont know why I would go for hardware RAID considering my existing hardware and future flexibility

 

I think you might be confusing things a bit. It sounds like you're mixing up different RAID levels.

 

For example. RAID5 is RAID5. Whether hardware or software, it works on the same principle. All drives in a RAID5 array have to be the same size. You can use different sized drives, but the RAID will limit each drive to using the amount of space available on the smallest drive in the array. If you use a 1TB, 2TB, and 3TB drives in a RAID5 array, each drive will have only 1TB of usable space, with the equivilent of 1 drive being used for parity. The array will have 2TB of space you can use.

 

The same goes for RAID0 and RAID1 in that software or hardware RAID operate basically the same.

 

There are some differences between hardware and software RAID however:

  1. hardware RAID tends to be faster;
  2. hardware RAID tends to be more reliable;
  3. hardware RAID tends to be more portable;
  4. hardware RAID is usually more expensive.

The main reason for hardware being faster & more reliable is due to it using firmware (higher end RAID cards even have their own CPUs). This makes it possible to isolate the RAID environment from the host system, which makes for greater reliability. Likewise, because the RAID card handles the physical access to the drives, it can be optimised for that purpose. Less reliance on the host system equals faster & more reliable.

 

If you want to use a bunch of different sized drives from different manufacturers and with different characteristics, you can use JBOD. Now, AFAIK, JBOD is technically not a RAID level at all (IOW, it was never approved by RAID steering committee). However, I think people think of it as RAID because RAID card makers started including it with their cards. It comes on a RAID card so it must be RAID, right? Not really, but OK. The advantage of JBOD is that you can use just about any drives; the disadvantage is that it has zero redundancy.

 

Where software RAID has taken the lead in the last few years is in advancing the concept. For example, there are software solutions out now that basically combine JBOD with parity, making it possible to use different drives and still have redundancy. But, there is a price for this - performance. Because the software has to deal with so many variables, it's simply impossible for it to be as fast as the older RAID levels. Overall reliability could be a concern too. I think DrivePool and DriveBender fall into this category.

 

Being somewhat slower isn't necessarily all that bad. The real question is, is it fast enough? In the past, optimising RAID was important because all of the hardware was slower: drives, cards, motherboards, etc. These days, hardware has gotten so fast that perhaps ideas that would have been unusable in the past are now viable.

 

I have to admit I've wondered why RAID card makers haven't started incorporating some of these newer ideas into their products. It could be that it's very difficult to do and still maintain the reliability they've become known for. Perhaps it's very expensive to do in firmware.

 

Some of the newer ideas seem to basically be ways to 'index' an existing file system. They don't change the underlying file system, or even write the data in their own format, they seem to provide a 'filter' that allows you to get at the data in different ways. Because of this, you can remove a drive from the system, attach it to another system, and still read the data, something that's difficult to do with a true RAID configuration. And, they can provide true redundancy by ensuring redundant data is written to more than 1 physical hard disk. However, it's not hard to imagine that such solutions are going to have a lot more overhead than traditional ones, and are going to be slower. DrivePool and DriveBender fit into this category. As I stated earlier, the real question is whether they're fast enough. If they're fast enough for you, then go for it.

Share this post


Link to post
Share on other sites
daveyboy37

Well im really late to the party on this topic. Im a long time user of Stablebit Drivepool but the duplication overhead was getting me down. About 9TB of HD movies all duplicated. So along with other data, my 13 drive server was rapidly running out of space. I decided to look at the FlexRaid and Drivepool combination that Hopester mentions. Awesome!!! Two systems that both allow you to rip a drive out and read on any other pc. Im still really struggling to see why people think 3rd party apps like these are risky.The alternative is buy a very expensive raid card if youre looking at above 8 drives. Oh and make sure all the drives are the same size and ideally make.Then pray it never fails (unless u can afford to have a spare sat in a drawer), if it does fail ( and u cant afford to carry a spare) then say bye bye to ALL your data. Now thats risky isnt it ?

Edited by daveyboy37

Share this post


Link to post
Share on other sites
ikon

Well, typically, you only need 1 spare drive in a drawer; at least with RAID-5. You don't actually need to have it on hand either; you can just go to the store and get one if needed. While same size is ideal, it's not mandatory; as long as the replacement drive is at least as large as the smallest drive in the array.

 

RAID-5 would make much more efficient use of your drives, so you would probably not need 13 drives. For example, 6x2TB drives would give you 10TB of usable space; 6x3TB would give you 15TB.

 

If you really need more than 8 drives, you can use multiple RAID cards. Certainly not cheap, but better than the very high end >8 drive cards.

 

I'm not against the software products; I just wanted to clarify how the hardware could be used to maximum effect. The reason people are a llttle leary of the software products is that many of them are pretty new, while RAID hardware has been around for decades.

 

I know you said you're a long time user of DrivePool but, really, it's only been out about a year now. That doesn't qualify as anywhere near a long time for me.

Share this post


Link to post
Share on other sites
daveyboy37

Well, typically, you only need 1 spare drive in a drawer; at least with RAID-5. You don't actually need to have it on hand either; you can just go to the store and get one if needed. While same size is ideal, it's not mandatory; as long as the replacement drive is at least as large as the smallest drive in the array.

 

RAID-5 would make much more efficient use of your drives, so you would probably not need 13 drives. For example, 6x2TB drives would give you 10TB of usable space; 6x3TB would give you 15TB.

 

If you really need more than 8 drives, you can use multiple RAID cards. Certainly not cheap, but better than the very high end >8 drive cards.

 

I'm not against the software products; I just wanted to clarify how the hardware could be used to maximum effect. The reason people are a llttle leary of the software products is that many of them are pretty new, while RAID hardware has been around for decades.

 

I know you said you're a long time user of DrivePool but, really, it's only been out about a year now. That doesn't qualify as anywhere near a long time for me.

 

 

 

i do see where youre coming from with product maturity. That said, although the drives dont necessarily have to be identical a failed raid card would (i think) need to be. For someone with half a dozen drives then raid isnt too expensive. For anyone with more data than that it will be.... and yes maybe people dont NEED to keep their 10TB collection of HD movies, but thats a different matter lol.

But despite however mature raid may be, i still would never trust anything that doesnt allow me to rip out an individual drive and read it on any system.

As a side thought.. I do wonder how many people are gonna trust microsofts "storage spaces", purely on the basis that its a microsoft feature. More importantly will they do a "DE" and decided they cant be bothered in a year or so :-)

Share this post


Link to post
Share on other sites
ikon

Actually, one of the advantages of a RAId card is that, in fact, you do not need to replace a failed card with an identical one. pcdoc has, for instance, documented in these forums how he has moved arrays from one model or RocketRaid card to another, even moving arrays from a 4-drive card to an 8-drive. I agree there might be less success trying to move an array from one make of RAID card to another, but that would be expecting a lot anyway.

 

As far as space is concerned (e.g. your 10TB HD collection), imagine an 8-port card with 8x3TB drives. You could start with only 4 drives (10TB) and keep adding to get a total of 22TB. 4TB drives expand this to 30TB.

 

RAID cards give something else that many motherboards can't - a lot of SATA ports. Just think; you can have 8 drives on the RAID card and still have all the mother board ports free for other purposes, such as an OS drive or drives (e.g. mirror) and external drives (perhaps in an enclosure).

Share this post


Link to post
Share on other sites
daveyboy37

I'm sorry I have been so long in replying to this...But yes I can imagine the setup you mention. Currently I have a single 3TB drive the rest are all 2TB drives so that already really diminishes my data capacity. I am still really struggling to see your point. I'm in the UK.. a 3TB drive is currently still £100+ Buy a minimum of 5 and an 8 port card and a spare card in case the first card ever fails. Really?? I paid $15 for Drivepool and $25 for flexraid.. Both running in harmony. Both totally non destructive systems. By that I mean IF a drive dies and Flexraid fails me all at the very same instance, then I will need to download a few movies again. The other drives are still fully readable. But Then again I guess its all subjective. I did see a post on here somewhere stating that a movie collection isn't critical data (in their eyes.) I tend to agree. Personally what is critical, are pictures and documents. Nothing that duplicated backup to SkyDrive and google docs cant handle! Maybe its down to different pricing on this side of the pond or maybe we just have massively different budgets. You suggest what will cost me near on £1000+ pound system(here in the UK) that hasn't failed you. Im always gonna go for a $40 system ( £25) that hasn't failed me. Each to their own budget I guess!! lol. That's not with standing the comment someone made that I "don't need to have that many movies, because I "PROBABLY" wont watch them again." I can just envisage bin fulls of Retail DVD's and Blu-Rays out on the street for collection with that logic lol.

Edited by daveyboy37

Share this post


Link to post
Share on other sites
ikon

I'm struggling a little with your logic. You mention a £1000+ system, but doesn't that include the drives? You require drives no matter which strategy you take, so I don't really see how their cost enters into the picture. No doubt, DrivePool is considerably less costly than a RocketRaid card, but that is really where the cost difference lies. It's nothing like a £1000, more like £100.

 

That said, if DrivePool is meeting your needs and, most importantly, you're happy with it, then it's the best solution for you.

Share this post


Link to post
Share on other sites
pcdoc

I agree with Ikon. The real comparison is in the solution that combines your drives such as the card or software. In my opinion, the small difference in price is offset by the flexibility and transportability. Not to mention some redundancy if you need it Replace you motherboard one time using the different solution that value proposition changes quite a bit. Just me two cents.

Share this post


Link to post
Share on other sites
ikon

I agree with Ikon. The real comparison is in the solution that combines your drives such as the card or software. In my opinion, the small difference in price is offset by the flexibility and transportability. Not to mention some redundancy if you need it Replace you motherboard one time using the different solution that value proposition changes quite a bit. Just me two cents.

 

I am wondering how portable DrivePool is. From Joe_Miner's testing we know that individual drives in a DrivePool array are readable on any system as native drives, so that's a plus. I can't recall for sure, but I think he also said something about moving a set of drives to another system and having them automatically detected as a DrivePool array once DrivePool was installed. If true, that would certainly be a huge plus for portability, especially since the drives are individually readable.

Share this post


Link to post
Share on other sites
Joe_Miner

I am wondering how portable DrivePool is. From Joe_Miner's testing we know that individual drives in a DrivePool array are readable on any system as native drives, so that's a plus. I can't recall for sure, but I think he also said something about moving a set of drives to another system and having them automatically detected as a DrivePool array once DrivePool was installed. If true, that would certainly be a huge plus for portability, especially since the drives are individually readable.

 

Yes, I've done it with DrivePool and it was pretty easy http://homeservershow.com/forums/index.php?/topic/2521-possible-to-clone-system-drive-in-whs-2011/#entry50819

 

I just wish I could DrivePool with S2012E -- with the ability to read data off individual drives and the ability to move DrivePool from machine to machine it is pretty resilent -- you would just about have to destroy the drives to make the data unrecoverable (of course -- we have 3-2-1 backups right?)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...