Jump to content
RESET Forums (homeservershow.com)
Sign in to follow this  
Nicholas James

Worst Case Scenario

Recommended Posts

Nicholas James

Hi Folks

 

I recently purchased two HP MicroServers for the purpose of replacing my existing storage server, I have +- 8TB of data and doing RAID's other than striping were getting complicated and expensive so I figured let's just get two of these, stripe the drives and then keep the data on both in sync.

 

Everything was going well, I had moved all the data from my current server so that I can reuse those drives in the slave, the master was working 100%, (2008 Server R2) and I was about 20% done with syncing all the data on the master to the slave. So the master had been up and running doing media streaming etc. for about 3 weeks, I had defragmented the drives to test heavy read writes etc. No errors, problems or anything else.

 

This morning I decided to reboot both the servers as they had been syncing continuously for more than a week or so, to my shock the master booted up with no D:\ drive, I checked storage management and it showed 3 x 3TB unassigned drives, no partitions and no data. I rebooted to check the BIOS, IDE was still RAID, went into the RAID utility and it simply showed 3 x 3TB single drives, there was however no sign anywhere of the array, it was just GONE!

 

Unfortunately the data is more than 10 years worth of collected info and all that has happened since everything worked was a normal soft reboot.

 

I am at a loss here and hope you guys have some ingenius options for me to get this RAID working again or at least find a way to recover the data.

 

Server specs:

2GB

OS is on shipped 250GB HDD

3 x 3TB Western Digital

Windows 2008 Server R2

 

All other hardware is standard config.

 

HP01.jpg

HP02.jpg

HP03.jpg

HP04.jpg

HP05.jpg

Edited by Nicholas James

Share this post


Link to post
Share on other sites
ikon

Welcome to the forums Nicholas.

 

You're not going to like this, but I would recover the data from 1 of my numerous backups.

 

I'm really not trying to be sarcastic, even though it might seem that way. The point is, if you have data, you need backup. There has been quite a bit of discussion of the 3-2-1 technique in these forums. I really recommend you check it out. Having 2 systems mirror each other is a start, but it's truly not enough, as you're finding out.

 

I realize this doesn't help you in your current predicament. I bring it up so that you can plan for the future.

 

BTW, which MicroServers did you buy -- it could make a difference in how to try to recover your data? What were you using to sync the systems up?

 

Also, I hate to say it, but booting the servers when you did might be what caused the issue. If they hadn't finished syncing yet, it might have led to the problem. Of course, you could also have a bad drive in the source array.

Share this post


Link to post
Share on other sites
jmwills

Three weeks to copy only 8TB of data?  I presume you already know that RAID 0 has no redundancy whatsoever.

 

As ikon mentioned, what sort of backups do you have?

Share this post


Link to post
Share on other sites
ikon

He'd been syncing the data between the boxes for about a week; he had the master server up and running, streaming videos, etc. for about 3 weeks. Granted, even a week is a long time for 8TB, but I was just trying to say that you shouldn't reboot until you know the job has completed.

Share this post


Link to post
Share on other sites
jmwills

So wait, we have the original server plus two MicroServers. shouldn't the source data still be in tact?

Share this post


Link to post
Share on other sites
ikon

I don't think so. I think it's 2 MicroServers. He's designated the 1st one he set up as the 'master' and, presumably, the 2nd will be the 'slave'. He got the master all set up and running, and had it running great for about 3 weeks. Then he went to sync it to the slave. He waited about a week for the sync to complete then, as I understand it, just arbitrarily decided to reboot both servers.

 

There was an 'original' server as you suggest. The issue is that he used the HDDs from the original to populate the slave so, in effect, there are now only 2 MicroServers :(

Share this post


Link to post
Share on other sites
jmwills

I don't see how a reboot could destroy an array.  Corrupt some data, yes, but completely destroy it............

Share this post


Link to post
Share on other sites
ikon

It is kinda odd, I admit, but we don't have all the info, so it's a bit of guesswork. That's why I mentioned about a drive going bad in the master server's array. It could just be really bad timing -- a drive in the array fails just as he's about to complete the syncing.

 

Which, come to think of it, brings up another question: what about the data on the slave array? If, as you suggest, a week should have been enough time for the sync to complete (and I agree), shouldn't he have a complete copy of the data on the slave? At the very least, he should have the bulk of it -- not ideal, but way better than losing it all.

Share this post


Link to post
Share on other sites
Nicholas James

Wrote a lengthy reply with the whole history of like 20 min or so and second time around the forum just hung and I lost everything... Did the same thing on a different computer...

 

Anyways, I have since used a program called "ReclaiMe" and it can read all three drives, the data, even the RAID configuration so I doubt the drives are the cause, it somehow seems as if the onboard controller simply lost it's configuration or alternatively it's a problem using Western Digital green drives, I read on the MicroServer wiki and they do not recommend green drives with these servers, something to do with how the green drives cache or power down or something like that. I asked the question on this forum previously but there wasn't much of a response.

 

Can't paste anything into this window, seems the forum might not be IE11 compatible, but there is another post I made previously where this same machine lost the RAID config after a reboot but I didn't really get any sort of a response and everything worked fine since so I hoped it was a once off random happening.

 

The only option now seems to be to buy this software for $79 and restore the data to my 4x2TB slave after I remove it's hardware RAID and rather create a software RAID as these are also WD Green Drives.

 

Unfortunately technology in South Africa is way too expensive for me to be able to afford proper large NAS drives and I can't afford the data loss with RAID 5 plus the cost of the controller for both machines, mirroring two pieces of hardware really would have been an ideal solution for both space and redundancy in my case.

 

Thanks for the replies, apologies for being grumpy.

 

The sync application I was using is called BTSync, great little app that uses BitTorrent technology to sync multiple machines on both LAN and internet.

The servers are N54L

The slave backup was destroyed so that the drives can be used for the new master

The sync was taking so long because I only have a 10/100 switch between these machines

No other backups, it's 8TB of data, keeping two copies is hard enough.

 

Share this post


Link to post
Share on other sites
Andne

I understand trying to keep track of that much data, but you really should have something besides just the two copies.  Another option may be to determine if parts of the data are more critical than others, and only back up the stuff you can't afford to lose instead of everything.  I do this with my setup, the folders used by MyMovies are too large to keep as part of the server backup, so I don't, but I do have my documents and other things that I can't recreate or would be difficult to recreate in the backup.

 

One of the issues with green drives in RAID configs is lack of TLER - Time Limited Error Recovery (or it used to be at least).  When a bad sector is encountered, the drive will spend some time trying to recover it, during that time the drive is pretty much unresponsive to the controller.  Most consumer drives, such as green drives, will not have much of a limit to how long they wait, while higher-end drives do.  If the drive doesn't respond to the RAID controller often enough, then the controller will drop the drive from the array as failed.  I don't know if that's what happened in your case or not.

 

Sometimes (and I won't guarantee that this won't lose data, but I think it has been known to work) you can get a RAID controller to recreate an array without initializing it, and then it starts working again.  I have pretty limited experience with hardware RAID controllers, mostly prefer to do software-based solutions myself since I find it easier to monitor the status.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  



×
×
  • Create New...