Jump to content
RESET Forums (homeservershow.com)
ShutterBug365

Home Server Help

Recommended Posts

ShutterBug365

Hi

New member here, going to take a while to look around and read many great threads. 

I am at a crossroads with my home server at the moment and wanted to ask for some advice if possible.

Currently got a Windows 10 Pro server (G4500, 8Gb Ram) working away nicely in the corner of my study, its working that well I cannot remember the last time I had to touch it apart from the renewal of anti-virus and firewall and a cleaning of the fans. It currently runs 4 x 6TB Wd Red with a 128GB SSD as the Operating System drive. The motherboard only has 4 slots so 1 of the 6TB is on a PCI Sata Card.

Usage wise it is for a family of three, 2 basic point and click users and myself. The other users struggle with the difference between the server and the internet router.

We have two Amazon Fire Boxes which I have installed Kodi on and we are using that to access MKV files from the server and so far it is handling everything thrown at it. Streaming to two devices and handling file transfers without issue.

The issue is one of storage capacity, when is it not I suppose, currently I have a very basic setup that is probably not the most efficient or cost effective but it is simple.

1 x Wd Red holds all our personal files on it which as we are all three photographers includes many thousands of RAW files and these are growing constantly. This is backed up using a programme called GoodSync to a 2nd Wd Red on a daily basis. I am studying a Computer Science degree, write my own websites and we have our Outlook PST files on the drive so plenty of changes each day.

1 x Wd Red holds the media for the MKV files this is then backed up to a 2nd Wd Red on an as and when required basis using GoodSync. These are nearly full and more storage is required, hence the crossroads.

I do have a license for StableBit Drivepool that I used to use and for some stupid reason that I cannot think of stopped using it when I added an extra 2 x 6Tb drives into the system. I think the plan was to eventually have two pools one for media and one for personal as the data grew which could then be backed up to other pools.

As mentioned I have probably gone about this totally the wrong way and if so please tell me. 

As I see it currently I have a few options:

  • Just add a 12TB drive and use it for the media and use the 2 x 6TB as the backups
  • Add 2 x 6TB, drivepool them together etc.
  • Move to something else: Synology, QNAP perhaps

 

As mentioned my motherboard has no more free slots, and at the moment there is 1 x PCI slot free that I could add another Sata Card to and 1 Sata Port on the current PCI Sata Card.

Any advice, thoughts would be appreciated. I am trying to read about RAID but trying to learn two programming languages this year is already making the brain hurt.

Many many thanks & Happy New Year.

Paul

Edited by ShutterBug365

Share this post


Link to post
Share on other sites
nrf

you didn't mention where your backups are. copying to a disk in the same box doesn't count.

Share this post


Link to post
Share on other sites
ShutterBug365
10 minutes ago, nrf said:

you didn't mention where your backups are. copying to a disk in the same box doesn't count.

 

Thanks NRF, therein lies just one of my problems.  There are no "outside of the box" backups, just "inside".  I have been looking at some external enclosures but am struggling in the UK to find one with good reviews.

Edited by ShutterBug365

Share this post


Link to post
Share on other sites
BSR

Interesting dilemma - as NRF pointed out and you confirmed, none of this is really backed up.  A flood, fire, theft or even something as random as an electrical surge or lightning strike could wipe out everything you have with little chance of recovery.

 

Something most members of this site are very familiar is the "one is none" and "two is one" saying about backups.  You may already realize most of this, so I apologize if I'm beating a dead horse.  But you really need actual backups that are not part of your server (and preferably not right next to it on the shelf).  You also need something in a different location, which could be a cloud backup or the ever-popular "Grandma" backup where you fill up a cheap external drive and take or send it to someone's house far away, where it would be safe in the event of a large natural disaster at your home.  There's lots of good reading and many ways to implement this here if you have time to dig through the threads.

 

But getting a backup solution in place doesn't help your current issue.  If you are happy with how your current setup works (except for space) I wouldn't throw out your server and start from scratch -- if it does everything well but just needs more room that's not hard to fix.  Replacing one or more drives with larger-capacity drives would be simple. 

 

I would suggest keeping most of your current 6TB drives but changing what they do.  Stop using them for backups inside the server as that only protects against a failure of one of the drives - not much else.  If you want all your movies on one drive, replace that with a larger one.  Move the .PST and RAW files to separate drives...this will give some breathing room, and you can always replace with larger later.

 

But then add an inexpensive backup solution to all this.  Get a 2-bay Synology or QNAP and fill with two drives large enough to back up everything, plus room for growth.  Put it somewhere else in your home and set your server to back up to that device regularly. 

 

THEN, also implement some sort of cloud or remote-location solution.  If cloud, there are plenty of ways for the NAS or your server to back up automatically to a cloud service.  If Grandma, make sure it's far enough away and happens with some regularity.

 

Also (and this is personal preference, but something I recently struggled with) maybe make some hard decisions about whether you want to keep ALL of the .MKV and RAW files you have.  I polled the family and found there were about a third of the movies we had on our NAS that nobody was interested in watching again.  Gone!  Yes, it feels like (since I paid good money for them years ago) I should keep them forever, but if nobody is ever going to watch them again, what's the point?

 

Similarly, I also had thousands of RAW files - was I really going to go back and work on those pictures from Italy from eight years ago?  Or was this just another project I would probably never get to?  I did find I could prune a good chunk of those files and not need to upgrade or reconfigure my storage so often.

 

I think that since disk space is so cheap, we often feel we should keep everything forever.  That's possible, but as the collection grows, it becomes more and more unwieldy to manage and especially to back up.  I ended up classifying my stuff into things I back up daily (family records and paperwork that changes almost daily, current pictures, things we are actively working on), and things that change less frequently or never.  I have family pictures dating back to the 1930s but those are never going to change - once they are backed up in a couple of places I don't need to worry about them.  I only back up new movies we've ripped about once a year.  I back up our music collections just a couple times a year....this is becoming less important as my kids are almost purely streaming listeners now.

 

The best analogy I have is the way our collection of storage boxes grew every time we moved (12 at last count since marriage).  Since most of these were corporate-paid moves we didn't need to think hard about what to keep - the movers took anything not nailed down (including a full garbage pail once).  But after years of this we had no idea what we had and where it was - we just kept storing them.  At the point where we moved into a smaller house and had to put a bunch of stuff in a storage unit, I began systematically going through and tossing or giving away what we no longer needed.  We now don't have the storage unit - my basement storage room is half as full as it was this time last year, and I'm getting close to thinking I'm done. 

 

But just like the household stuff, I find data increases if not managed, pruned and judiciously disposed of.  I am hoping by the time I retire (6-10 years) I can be down to half what I have now, and when I finally kick the bucket, my kids aren't stuck with a huge mess of stuff to sort through.

 

Just suggestions - ymmv, but good luck and let us know what you end up doing.

 

 

  • Like 1

Share this post


Link to post
Share on other sites
JackoUK

Hi Paul

1st thoughts …

10.            10. Alter your backup regime to operate in 3 parts now, so that you can plug each of 3 backup drives in turn into the caddy.

You now have 18TB on the server and 18TB of drives that you can take ‘offsite’.

Your next capacity upgrade might be 2 more 6TB drives – one for the server and one for backup. You don’t have to stick with same size drives: 8TB is probably the value sweet spot at present but this will change over time. However if you later plan to try a RAID configuration then keeping all drives the same will be best.

Plan B with Drivepool

  • -          Remove all 4 HDDs from the server.

  • -          Install the 2 new HDDs and format.

  • -          Install Drivepool, creating a 12TB virtual disk.

  • -          Play around with Drivepool options and backup thereof on some test data while you have the new HDDs.

  • -          Insert the HDD containing your movies into the caddy and copy the data into the pool. Could take a while with 6TB!!

  • -          Check the copy has been successful.

  • -          Format the HDD in the caddy.

  • -          Remove the HDD from the caddy and install it in the server.

  • -          Add the reformatted drive’s capacity to the pool, creating an 18TB pool.

  • -          Put the remaining data’s HDD in the caddy and copy the data to the pool.

  • -          Verify successful transfer.

  • -          Reformat the drive in the caddy.

  • -          Use the option to balance the data spread across the 3 drives. Takes a while.

  • -          Run backup to each of the 3 external drives in turn.

 

You get the idea: play around on the new disks (you’ll have 3 copies of your data at times) and yes do play around to avoid catastrophes.

Plan C with another server, NAS or RAID levels

Countless options … but I think once you start bonding disks into pools and RAID arrays the convenience/performance gained is lost to simplicity and flexibility: you just can’t beat Disk15’s backup being on Disk15B. In an enterprise environment one needs availability to prevent downtime but the worst that can happen for a disk failure in the home setting is an ear-bashing from the partner or kids when the backup lacks the video required!

Having said that a fast storage system for large-scale video editing, or VM migration across servers … and data integrity protection via RAID scrubbing, ZFS or REFS do have value on prosumer networks.

My best guess at the groupthink on the HSS is:

  • -          Keep it simple and organise external backups

  • -          Stablebit Drivepool (+ scanner) does almost all that required for a big home storage system

  • -          If you have to ask about the basics of RAID, ZFS, Storage Spaces and REFS … then be prepared to put some serious effort and money in (which you might do of course with your studies)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×