Jump to content
RESET Forums (homeservershow.com)
jdell42

DE or RAID for 24-48 TB?

Recommended Posts

jdell42

New to the forumn so thanks in advance for some advice.

 

I have been running WHS v1 for ~3-4 years on an Acer H340 and dealt with the annoyances. Over the years I have added a Sans Digital 8 bay expansion with a RocketRaid 622 card. I was at 23TB and nearly full (with folder duplication turned on) before I ran into problelms.

 

Recently the server would lockup after about 2 min of run time. Several reboots and the same problem. Removed all drives and ran check disk and found no problems. I attempted a system restore on the system drive and it would continuously fail. After about 20 attempts I finally managed a full restore of the system drive and have a working sever with 11 drives of data that do not belong to the drive pool and the annoyance of fixing that.

 

I realized a while ago I was probably pushing the limits of the hardware and OS with 23 TB of data and the possibility for multile HD video streams accessing it simultaneously. I deciced to bite the bullet and go for a 24 bay Norco Chassis and upgrade my HS setup.

 

Now given the large existing capacity I have and the opportunity to double it over time I was wondering what hte prevaliing opinions are regarding managing the data. WHS V1 drive entender worked OK for me.. but I always had a very uneasy sense of a melt down and it finally came. I am cautious about the 3rd party apps for replicating DE in 2011 but equally cautious about trying to run a Raid 5 or 6 array under WHS 2011 as I dont know how easy expanding and contracting the pool would be. I would like to stay with Windows as thats what I know. I read / viewed PCDOC's posts on using RAID and it looks easy enough... but looking for the prevailing opinions and pros/cons of the options.

 

Thanks,

John

Share this post


Link to post
Share on other sites
Technogod

If you still had your original setup migrating to WHS 2011 and Stablebit Drivepool would have been perfect. Now with the Norco chassis I'm not really sure. If you go with Stablebit Drivepool I'd also recommend Stablebit Scanner.

Advantages of Stablebit Drivepool over RAID are:

1)If 2 or more drives die the data on all the reset of the drives is safe.

2)Drives can be removed and installed on another machine and you can still read the data.

3)No rebuild times which can often be long slowing your system down.

4)Stablebit Drivepool R/W speeds are in the 100 MB/sec. range on my system so it is plenty fast.

5)No RAID controller card needed. Can't run RAID 5 off the motherboard. RAID 0 or 1 should be fine.

6)BSOD's can often cause verification's that slow the system down or corrupt data.

7)Drive diagnostic software doesn't support RAID.

Edited by Technogod

Share this post


Link to post
Share on other sites
jmwills

If you still had your original setup migrating to WHS 2011 and Stablebit Drivepool would have been perfect. Now with the Norco chassis I'm not really sure. If you go with Stablebit Drivepool I'd also recommend Stablebit Scanner.

Advantages of Stablebit Drivepool over RAID are:

1)If 2 or more drives die the data on all the reset of the drives is safe.

2)Drives can be removed and installed on another machine and you can still read the data.

3)No rebuild times which can often be long slowing your system down.

4)Stablebit Drivepool R/W speeds are in the 100 MB/sec. range on my system so it is plenty fast.

5)No RAID controller card needed. Can't run RAID 5 off the motherboard. RAID 0 or 1 should be fine.

6)BSOD's can often cause verification's that slow the system down or corrupt data.

7)Drive diagnostic software doesn't support RAID.

 

Just curious as how to big your data volumes are? I just don't think I would trust 48TB of data to a DE type product. But the most important thing left out of this equation was the mention of a backup from either of you.

Share this post


Link to post
Share on other sites
jdell42

Just curious as how to big your data volumes are? I just don't think I would trust 48TB of data to a DE type product. But the most important thing left out of this equation was the mention of a backup from either of you.

 

Not entirely sure what you mean by data volumes but I have 12, 2TB drives and 1, 1TB Drive of various manufacturers. The file sizes I typically access are HD movies from 5-20 GB and various other smaller media such as pictures and music. I have no array setup currently but only the drives from the previous WHS v1 setup that contain all my data but not part of the DE pool.

 

In terms of backup or rather redundancy I have been using folder duplication. I have no other mechanism nor plan to put in place anohter layer of backup greater than raid 5 or 6 or some type of folder duplication. Critical data is stored on multiple PCs / clound.. ie family photos are on both the WHS and the PC and carbonite.

Edited by jdell42

Share this post


Link to post
Share on other sites
ikon

Like jmwills, backup was one of my first thoughts. Glad to hear the critical data is protected. My other thought was, "Why does this person need 24TB of storage? And what on earth are they going to store that would require doubling that?" I mean, think about how many hours of video that is. Who could possible watch all of that? What is the ultimate purpose of keeping all this video?

 

I suspect a lot of people are storing multi-terabytes of data that they will never access. I used to collect DVD's. One day I was looking at the collection and realized I hadn't accessed 80+% of them in the past 3+ years, and likely never would again. I digitized some of them and got rid of the rest. It just didn't make sense to keep them. It's up to you of course, but you might want to peruse your collection and see how many TB you could delete.

Share this post


Link to post
Share on other sites
pcdoc

Wow, I thought I had too much storage. :D Of course house cleaning is always a good idea, but personally, given the size of your array, I would use a RAID card "providing" that your drives are the same type and MFG, and that they are not green drives. Using something in the RAID 5/6 Catagory is going to be far more efficient. Using a DE replacement will certainly work but it will reduce your storage in half just like DE did when you wanted redundancy. You had 23T of storage Space, but where only effectively using about 12-13T. Using a RAID 5/6, you would get a 20/18T of actual usable space with that many drives. The trick is making sure you have the same drives. If you have a variety of drives, then you will be stuck with a drive extender and the loss of some usable space or possible consider some new drives. I am using 5 3T drives on a Highpoint 2720 as well as 8 2T drives on a HP 2680 and have the same usable space as you have with the 12 drives. With this much storage, you really have to plan on what you want and how you are going to expand it. The DE replacement are initially cheaper and easier, but the storage loss (assuming you want redundancy) is 2 to 1 and you will have to have a pill of drives to get what you want. The benefit quickly fades as you storage needs increase. Technogod, had some good points and I agree with all but a couple, but that said, I still would not trust data with a pool this size to anything but a RAID card. Ultimately it boils down to what you are comfortable with.

 

Here is an 8 drive card:

http://www.newegg.com/Product/Product.aspx?Item=N82E16816115111

 

Here is a 16 drive card

http://www.newegg.com/Product/Product.aspx?Item=N82E16816115085&Tpk=highpoint

Share this post


Link to post
Share on other sites
hopester

I setup a smaller scale version of what you are doing for a friend. In the end I used stablebit drivepool and flexraid together. For all of his music, photos etc (small files) I used drivepool duplication on the shares with crashplan backing up anything critical. For the static data large files, ie movies shares, I turned off duplication and used snapshot raid to protect the drives. The latest beta version of drivepool allows you to select which drives get duplicated data and which drives dont so its easy just to use flexraid on the non duplicated data.

 

All of his movies are rips from bluray so flexraid is perfect in protecting his time spent in ripping the disks.

Share this post


Link to post
Share on other sites
jdell42

PC Doc and JMWills.. thanks for taking the time to weigh in on a solution.

 

No chance of having the same drives. I do have a mix of enterprise and commercial gear but at different speeds

 

I have 4 of these: http://www.newegg.com/Product/Product.aspx?Item=N82E16822148610

3 of these: http://www.newegg.com/Product/Product.aspx?Item=N82E16822148825

 

3 or so WD caviar green and a few other random drives. I really dont want to go out and replace drives.. and I really dont wnat to be tied to a specific drive or mfg. I guess my wish list is not dissimilar to others hence the difficulty in making a choice.

 

So based on that I would assume DE is somewhat more viable... albiet less efficient it terms of storage, but a lower entry cost. RAID is probably better suited for handling current and future needs but I would need an investment in HDDs and be tied to a similar drive moving forward along with an investment in a large RAID controller and/or an expander.

 

Still debating but leaning torwards DE.

 

I setup a smaller scale version of what you are doing for a friend. In the end I used stablebit drivepool and flexraid together. For all of his music, photos etc (small files) I used drivepool duplication on the shares with crashplan backing up anything critical. For the static data large files, ie movies shares, I turned off duplication and used snapshot raid to protect the drives. The latest beta version of drivepool allows you to select which drives get duplicated data and which drives dont so its easy just to use flexraid on the non duplicated data.

 

All of his movies are rips from bluray so flexraid is perfect in protecting his time spent in ripping the disks.

 

Saw your post after my reply.... that is an interesting solution. Sor for a very large movie share... ie 8TB how is that setup.. I assume you use flexraid to make a raid volume and then use WHS 2011 to assign that volume as a share?

Share this post


Link to post
Share on other sites
pcdoc

Obviously it is your call, as you have to feel comfortable with your choice. Another option is to setup two arrays, the 4 one and the 3 on the other. That would give you 10T of usable space which is still more than than a mirror that you get from a DE. If you really want to add variables, use a DE (or storage spaces) to pool the two arrays together without the duplication (JBOD). You end up with more usable space, have redundancy from the RAID 5, and the one large volume to work with.

Share this post


Link to post
Share on other sites
hopester

Saw your post after my reply.... that is an interesting solution. Sor for a very large movie share... ie 8TB how is that setup.. I assume you use flexraid to make a raid volume and then use WHS 2011 to assign that volume as a share?

 

You set everything up with Stablebit Drivepool first then use flexraid (snapshot) to protect the unduplicated movie share. Flexraid is file level raid so all the disks will be ntfs format. Say if you had 8x 1TB disks with movies on them, you then have 1 parity disk to protect the set. Once a day overnight (the snapshot part) parity is created from the set so you are protected from a single disk failure. The more drives you have, the number of parity disks can be increased so you can have 2+ failures at the same time and still be covered. Its great for protecting static data like movies but not so good for small files that change quite a bit, thats why you can get the best from both worlds combining drivepool with flexraid.

 

So you use Drivepool to make everything appear as 1 large volume, you then use drivepool to duplicate some shares (small files, changing data) and the other shares in the volume, you turn duplication off but protect those drives with snapshot raid.

 

OS Drive

 

Drivepool Volume comprising of:

2x 1TB Drives (Duplicated Data) - Music, Docs, Photo Shares

9x 1TB Drives (8 Data Drives + Parity) - Movie, TV Episode Shares.

 

Goes without saying you need a good understanding of WHS2011, Drivepool and Flexraid before you attempt the above ;)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...