Jump to content
RESET Forums (homeservershow.com)
Anatoli

Ultimate Gen8 Upgrade

Recommended Posts

Anatoli
Posted (edited)

I've pinged Schoon about some questions I had regarding the Gen8 upgrade but I guess a community discussion will be more fun.

 

The ultimate config

The final config I'm planning to have is the following:

 

Basically, what I would like from the Gen8 is to be a container host as well as a data storage unit. The OS and the VMs will be located on the SSD mainly, if larger storage is needed it could be nice to link it to the 4 disk array. I would like to RAID the thing using mdadm and virtualize using KVM so I would more likely use FreeNAS, Debian or Ubuntu as OS. Any thoughts about this? Suggestions?

 

Getting down to the ports

After looking into it for quite a long time now, I see the Gen8 motherboard has a max 5 disk capacity using all SAS/SATA ports. This won't fit my need since I need the 6th drive, and I need to get a RAID controller but since I'm going to RAID via software I won't need that and can definitely go for just a SATA PCI-e card with ya boy Marvell 88SE9215 instead.

 

What do you think about this? Am I doing good at computers so far?

Edited by Anatoli

Share this post


Link to post
Share on other sites
netware5

My current config is very similar - MS Gen8, E3-1265L, 16 GB original HP RAM, 2x4TB WD Red ZFS mirrored, running Nas4Free Embedded as a host OS, and one Ubuntu Server 16.04 VM as a guest. The Nas4Free is providing storage services, while Ubuntu VM is serving as OpenVPN server and CUPS print server.

 

My advice is not to use RAID. It is is much better to use native ZFS "raid" features - it is faster, reliable and portable. In case of problems with MS hardware your ZFS data disks will be readable on any PC with SATA controler just by plugging a USB stick with Nas4Free embedded OS.

 

FreeNAS and Nas4Free are close relatives. Nas4Free has been forked from old FreeNas when part of the developer's team not agreed with the change from PHP to Pyton. Personally I prefer Nas4Free, as it is closer to the "pure" NAS OS, but has sufficient additional services.  Nas4Free GUI is faster and more intuitive. The Nas4Free community and forums are also much more friendly place to novices compared to FreeNAS community and forums.  The Virtualbox as part of Nas4Free works like a charm an virtualization is very easy.

 

So in any case for NAS purposes stay with ZFS and FreeBSD based OS like FreeNAS and Nas4Free. My preference is for Nas4Free.

 

Other suggestion is to use "embedded" installation, i.e. your OS will be on USB stick or MicroSD Card. In such case you will not need two SSDs mentioned in your post. The MS Gen8 has an internal USB port and internal MicroSD slot, so nothing will  be visible on external USB connectors. Nas4Free has also a very stable "Full Root on ZFS" version, which can be installed on SSD. Generally it is needed only for special purposes, so my advice is to stay with embedded, except you have some special needs. 

 

  • Like 1

Share this post


Link to post
Share on other sites
schoondoggy

Your OS choice will help guide the direction you may need to go for more SATA ports. The Marvell based cards are good, but the 88SE9215 cards use a x1 PCIe interface, which could get saturated if you ran four SSD's on it. If you need to add ports and want full performance I would look for a LSI based card. The LSI SAS 9212-4i are affordable.

  • Like 1

Share this post


Link to post
Share on other sites
Anatoli
 

My current config is very similar - MS Gen8, E3-1265L, 16 GB original HP RAM, 2x4TB WD Red ZFS mirrored, running Nas4Free Embedded as a host OS, and one Ubuntu Server 16.04 VM as a guest. The Nas4Free is providing storage services, while Ubuntu VM is serving as OpenVPN server and CUPS print server.

 

My advice is not to use RAID. It is is much better to use native ZFS "raid" features - it is faster, reliable and portable. In case of problems with MS hardware your ZFS data disks will be readable on any PC with SATA controler just by plugging a USB stick with Nas4Free embedded OS.

 

FreeNAS and Nas4Free are close relatives. Nas4Free has been forked from old FreeNas when part of the developer's team not agreed with the change from PHP to Pyton. Personally I prefer Nas4Free, as it is closer to the "pure" NAS OS, but has sufficient additional services.  Nas4Free GUI is faster and more intuitive. The Nas4Free community and forums are also much more friendly place to novices compared to FreeNAS community and forums.  The Virtualbox as part of Nas4Free works like a charm an virtualization is very easy.

 

So in any case for NAS purposes stay with ZFS and FreeBSD based OS like FreeNAS and Nas4Free. My preference is for Nas4Free.

 

Other suggestion is to use "embedded" installation, i.e. your OS will be on USB stick or MicroSD Card. In such case you will not need two SSDs mentioned in your post. The MS Gen8 has an internal USB port and internal MicroSD slot, so nothing will  be visible on external USB connectors. Nas4Free has also a very stable "Full Root on ZFS" version, which can be installed on SSD. Generally it is needed only for special purposes, so my advice is to stay with embedded, except you have some special needs. 

 

Great suggestions here, but I think I'll stick to the OS on the two drives 'cause I don't trust USBs or SDs enough and I don't want to lose much time in case of USB failure reinstalling OS and configuring if needed.

 

 

Your OS choice will help guide the direction you may need to go for more SATA ports. The Marvell based cards are good, but the 88SE9215 cards use a x1 PCIe interface, which could get saturated if you ran four SSD's on it. If you need to add ports and want full performance I would look for a LSI based card. The LSI SAS 9212-4i are affordable.

I'm not sure about which OS should be the main OS. The VMs will definitely run maybe one Windows VM and some Ubuntus since it has great compatibility to Docker but I can't tell if it could work fine as RAID manager. I read FreeNAS and friends are more suited for that so the choice will be between FreeNAS and mdadm I guess...

Share this post


Link to post
Share on other sites
netware5
 

Great suggestions here, but I think I'll stick to the OS on the two drives 'cause I don't trust USBs or SDs enough and I don't want to lose much time in case of USB failure reinstalling OS and configuring if needed.

 

 

 

There is no "installation" of embedded versions of Nas4Free and FreeNAS. That is the beauty of embedded installations. All your data is on data disks protected by super reliable ZFS. Your OS is on USB stick bootable on any PC.  The all "installation" related information is kept in small XML config file saved on the USB. Usually I keep backups of that file on other places also. So if your OS USB stick fails, just use another one and then "import" configuration from config file. This is very, very convenient. I just stopped to thing about the usual nightmare - damaged OS boot disk.

 

In my case I even use two identical USB sticks and replicated them. The first one is in the internal USB port of my MS. The second one is kept ready in a box on my desk. If something happens with primary one, I've just plug-in the backup one in the external USB port and reboot the server (external ports have priority over internal port) - just 30 seconds to do it and time needed to boot Microserver. If you had recently changed something in configuration and it is not reflected on the backup USB stick, just upload via Web GUI the backup configuration file - another 30 seconds :)

 

Share this post


Link to post
Share on other sites
Anatoli
 

 

There is no "installation" of embedded versions of Nas4Free and FreeNAS. That is the beauty of embedded installations. All your data is on data disks protected by super reliable ZFS. Your OS is on USB stick bootable on any PC.  The all "installation" related information is kept in small XML config file saved on the USB. Usually I keep backups of that file on other places also. So if your OS USB stick fails, just use another one and then "import" configuration from config file. This is very, very convenient. I just stopped to thing about the usual nightmare - damaged OS boot disk.

 

In my case I even use two identical USB sticks and replicated them. The first one is in the internal USB port of my MS. The second one is kept ready in a box on my desk. If something happens with primary one, I've just plug-in the backup one in the external USB port and reboot the server (external ports have priority over internal port) - just 30 seconds to do it and time needed to boot Microserver. If you had recently changed something in configuration and it is not reflected on the backup USB stick, just upload via Web GUI the backup configuration file - another 30 seconds :)

 

Sounds definitely interesting. I will try this as well and see how good it performs :)

Share this post


Link to post
Share on other sites
netware5
Posted (edited)
 

Sounds definitely interesting. I will try this as well and see how good it performs :)

 

You may choose three types of using Nas4Free - Live (CD or USB), Embedded and Full.

 

Live is using a boot CD or USB with fresh default settings. To use it just download the OS (CD or USB) image and burn/write it on CD/USB. In such case you have to manually import the config file after each boot. It is very similar to "Live" Linux or other OS-es CD/DVDs. But normally this mode is used only for initial boot and then using console menu preparing and writing the Embedded installation to an USB stick.

 

Embedded - in such case the configuration file is written on the USB stick and OS boots like any "installed" OS - know everything about your configuration, data disks, IP addresses, user names, passwords, etc.

 

Full - real installation as any other OS. You will need a boot HDD or SSD.

 

You can easily switch between different types. Just keep backup copies of your configuration file on other media and/or devices. So you may start with Embedded and in case you see need of full one you may further buy and install boot SSD. The main rule is not to mess data disks with OS disks/USBs. It is much more safer if your data is separated from your OS boot media.

 

****EDIT****

Please note that for some reasons Nas4Free is working better on HP Microserver Gen8 if your boot USB is MBR partitioned. GPT partitioning often creates issues. I don't know the reason but it is proven.

Edited by netware5

Share this post


Link to post
Share on other sites
qianjizhao

Similar to mine Gen8.

1365L+16GB+SSD*1+ 3.5"*4+2.5"*1

I had a PCI-E SATA card (no raid)installed, SSD linked to the PCI-E card as OS drive. So no need for USB or SD card to start from 2.5" SSD.

4 3.5" SATA for storage, additional 2.5" drive in the optical drive bay as additional backup.

 

 

Share this post


Link to post
Share on other sites
Anatoli
 

Similar to mine Gen8.

1365L+16GB+SSD*1+ 3.5"*4+2.5"*1

I had a PCI-E SATA card (no raid)installed, SSD linked to the PCI-E card as OS drive. So no need for USB or SD card to start from 2.5" SSD.

4 3.5" SATA for storage, additional 2.5" drive in the optical drive bay as additional backup.

 

 

If I got you correctly, that's about the same setup I want.

So far, I've ordered the PCI-e SATA card and will stick to 2xSSD + 4xHDD with software RAID using mdadm on CentOS or Debian, 'cause I want to use KVM for the virtual machines I'm planning to run.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • stoney73
      By stoney73
      Wonder if anyone can point me in the right direction here. Have just replaced the CPU on my gen 8 microserver. Went according to the video above. Upon booting the server up, there is a clicking sound coming from the main board. Regular as clockwork, and the server isn't posting. Any ideas? I've swapped back to the G1610T CPU, but still the same issue. And just make it interesting, I don't have access to a monitor or keyboard.
       
      https://photos.app.goo.gl/zyLJWBZQ4mTQ818o2
    • arefiev.dm
      By arefiev.dm
      Hello! I put P222 on my Gen8 server, but I can not use RAID6. Can someone share the key?
    • milev
      By milev
      Hi,
       
      I have upgraded from G1610T to E3-1265Lv2 with success but the memory speed which is still stuck at 1333MHz instead of 1600Mhz.
       
      I have 2 x 4GB DIMMs with one be the original HPE DIMM and the other a Crucial 4GB 240 Pin DDR3 1600 MTps PC3-12800 CL11 Unbuffered ECC UDIMM Memory Module )(https://www.amazon.co.uk/dp/B00IW4M9PK/ref=pe_385721_37986871_TE_item)
       
      The iLO memory summary is always showing:

       
      and while I am on the latest firmware ....

       
      The BIOS options available are : 1066, 1333, Auto
      I have also tried the power setting to "max performance" but still the memory speed remains to 1333Mhz
       
      Any ideas on how to set the memory operating frequency to 1600Mhz ?
       
       
    • acidzero
      By acidzero
      Hello,
       
      So, after several days of testing various different configurations, creating custom ESXi install ISOs and numerous reinstalls, I've managed to get ESXi 6.5U1 installed on my Microserver Gen8 and have working HP Smart Array P410 Health Status showing. For those that are struggling to do the same, firstly here's how. I used original VMWare ESXi 6.5U1 ISO, build 5969303 then made the following modifications:
       
      Remove driver "ntg3" - If I left this in, I had a weird network issue where Port 1 or 2 would repeatedly connect/drop every few seconds. This forces ESXi to use the working net-tg3 driver Remove driver "nhpsa" - this Smart Storage Array driver is what causes array health monitoring to not work. Remove to force ESXi to use working "hpsa" driver Add the Nov 2017 HPE vib bundles Remove hpe-smx-provider v650.01.11.00.17 - This version seems to cause the B120i or P410 to crash when querying health status Add hpe-smx-provider v600.03.11.00.9 (downloaded from HPE vibsdepot)
      Add scsi-hpvsa v5.5.0-88 bundle (downloaded from HPE drivers page)
      Add scsi-hpdsa v5.5.0.54 bundle (downloaded from HPE drivers page)
       
      I did the above by getting a basic/working ESXi/VCSA installation and then creating a custom ISO in VCSA AutoDeploy and exporting it. But the same can be achieved by installing VMWare's original ISO and modifying via the esxcli command.
       
      I have a SSD connected to the SATA port, onto which I am installing ESXi. The 4 front drive bays are connected to the P410.
       
      Configure the Microserver Gen8 B120i to use AHCI mode - the B120i is a fake raid card so only reports physical disks to ESXi. Leaving in RAID mode works but I got a false Health Alert on Disk Bay 5 Install my modified ESXi ISO to the SSD  
      With these modifications I have a working ESXi 6.5U1 on my Gen8 with fully functioning HPE tools and array health monitoring:
       

      I also tested disabling the vmw_ahci driver, which is why the AHCI controller shows it is using ahci in the above image.

       
      If I pull out a disk to test a raid failure, when the health status next updates I can see a raid health alert in the ESXi WebGUI:
       

       
      However I'm now stuck at the next stage - getting this storage health to pass through to VCSA. VCSA can successfully see all other health monitors (under System Sensors in ESXi) just not the storage, which is the most important.
       

       
      Does anyone know how I can get the storage health working in VCSA?
       
      Thanks.


×