Jump to content
RESET Forums (homeservershow.com)
RyanG8

HP Proliant ML350p G8 Watercooling!

Recommended Posts

DenKarp

Continuation to previous thread: 

 

Here is what it looked like with the new fans instead of the deltas and before I put in a voltage regulator. I tried to use a toggle switch that can be adjusted from 12v to 7v but it was finicky in where is would randomly not supply power. Saw my temps increase at idle and was able to troubleshoot that the switch was causing the issue. Since the motherboard has limited fan power ports (only 1) on the mobo I put some molex connectors in parallel (since they came with the fans and already had 4 pin power prongs on them). 

 

20190725_214437.jpg.72b0dab546ee202bce7ec6c783e966d5.jpg

 

Of course that was still not enough to keep the cpu cool. Got myself some Arctic Freezer 12 heat-sinks that have fans on them (one of them was a tad bit larger than the other since it was a mistake on amazons part but didn't want to go through the hassle of returning and waiting for the correct part so decided to install anyway.) Another issue when I received the heat-sinks was that the brackets to hold them down was not the 2011 NARROW but instead just the regular squared 2011 socket type. Very unfortunate but decided to cut off the ends since each side was only 1-2 mm off in order for it to fit properly. No bigee. 

 

91282042_FanandHeatsink.thumb.JPG.92d986b945b790bc2a9d50a13a87f197.JPG

 

Installed a voltage regulator (instead of the 7/12 v switch) and did a bit of cable management. 

 

1325388411_VoltageRegularCompilation.thumb.JPG.ff43fe792060e0491629f0c16a7562a6.JPG

 

Here is what my temps currently stand at. As you can see the idle temps is at an average of 38 C and once I put some load on it (rocket league at maximum settings) it does not go above 63 C (9V on the regulator at all times). The orange graph line represents the GPU temp. 

 

Note: 6 hours total on the graph.

Capture.thumb.JPG.06955f7a0362b76e77766c10dd468ab2.JPG

 

 

For storage I decided to utilize the on-board RAID controller with 3 6TB HDD's in the drive bay. Two in RAID 0 and one standalone to back the important data up. I had a PCIe card RAID controller card but after a little research found out the mobo had one already, plus the onboard had 2GB chache card instead of the 1GB on the PCIe. OMG...what a world of difference. Here are some read/write benchmarks I was able to pull. 

 

1180654441_AllRaidResults.thumb.PNG.f967e7dd6ec994e9494175d06637bd4c.PNG

 

So far so good. Has been handling everything I have been throwing at it like a champ. There are a couple downsides such as no 3.0 usb's and no UEFI boot support (for now). It does take about 1 minute for the unit to boot but I don't mind. I have looked into power modes but doesn't seem to support any of them. Did figure out a way to hibernate but that caused issues where it would randomly hibernate my machine so decided to disable that in the registry. What I really enjoy is I am able to ssh into the on-board ilo and turn it off/on remotely and then essentially RDP if I want to grab files/work on things. 

 

One thing to note is that if you do have dual cpu sockets filled it will require a windows pro license in order to utilize both. Figured that out that hard way. There are ways in order to 'tweak' registry in order to get it to work without pro but i'm assuming it will cause a whole string of issues down the line.

 

Please let me know if anyone has any questions in regards. Cheers!

Edited by DenKarp
Incorrect reference to a component.

Share this post


Link to post
Share on other sites
ook4mi

Congratz for your config, it looks great ! I like the idea to control volt so you can control speed of fan. I did a few thing to make mine quiet but it still a bit loud :( ...

 

What is the config of your HP ML350P g8 ?

 

3 sata drive ? You don't have hdd cage in your machine ? You boot directly from Sata port on the mother board ?

 

Did you install Win10 using HP Intelligent Provising ?

 

The cache module part of your message really interested me because I'm looking to buy one to have more than 2 logical volume :) . With some distro like UnRaid you can use a SDD as a cache to improve speed :) .

 

Thanks ;)

Share this post


Link to post
Share on other sites
DenKarp

Thank you for the response! 

 

Config of my machine? Not sure in what manner you mean. Don't have it for anything other than just the normal desktop use.

 

Yes I do have an hdd cage which houses them. Its the original that came with the unit and also came with the short length connectors (fiber i believe) that pop into two slots on the motherboard. You can't see it from here but they are right behind the drive bay.

20200220_193647.jpg.0a9a069ede84a4cc3662786d5ab6019a.jpg

I believe I already had Win10 installed when it was just a standalone drive connected to the sata port. I then enabled the RAID controller and adjusted it for RAID 0 and it took a few minutes, I believe. It stripped whatever was on disk 0 to the other drive and left the third one alone, as specified.

 

I could use a SSD as a cache but prefer the robustness of an actual physical controller instead....besides, the max GB/Ts is I believe 6 on this motherboard and am already very close to it. 

 

Give UnRAID a try and post results!! Would like to see how it pans out! 

Share this post


Link to post
Share on other sites
HJB

Hi there fellow hardware hackers :-)


I'm in the process of setting up a Home Server based on this ML350P Gen8. 


While awaiting some deliverys like Noctua Coolers and 2.nd CPU I decided to do some basics first.

  1. Collect the relevant Documentation
    1. QuickSpecs
    2. SpareParts List
    3. HP ProLiant Gen8 Troubleshooting Guide, Volume 1: Troubleshooting
    4. HP ProLiant Troubleshooting Guide, Volume 2: Error Messages
    5. HPE ProLiant ML350p Gen8 Server Maintenance and Service Guide
  2. Collect the latest Firmware
    1. ILO Channel Interface Driver Version: 3.30.0.0(24 Oct 2016)
    2. ILO Version 2.73(13 Feb 2020)
    3. Power Management, Version: 3.3(B)(20 Mar 2018)
    4. Storage Controller, Version: 8.32(C), Nov 27, 2018
  3. Create a Bootable SSD with all nessesary Stuff installed/available as temporary Bootdisk.
    1. Unregistred Windows Server 2016. (English with Swiss-DE, Swiss-FR and US Keyboards)
    2. Installed Smart Storage Administrator
    3. Installed ILO Channel Interface Driver
    4. HP Documentation and Firmware from Chapter 1 & 2 in C:\HPE-ML350P-Gen8
  4. Update the Firmware
    1. Upgraded ILO to V2.73         (Yes the HTML5 Remote Console works like a Charm).
    2. Upgraded Power Management to V3.3
    3. Upgraded Storage Controller to V8.32
  5. Change Smart Array P420i Controller to HBA (Host Bus Adapter) Modus.  Beware!!! System cannot Boot from Disks on HBA Mode P420i Controller. You can alternatively boot from SATA SSD, USB, SD Card, or CD-Rom if need to :-)
    1. Start Smart Storage Administrator
    2. in P420i Configuration check Set Port Mode HBA
    3. Terminate Smart Storage Administrator
    4. Reboot System
  6. Installed extra RAM    It was running with only 8GB RAM as I got it. As I got hand on some old Gen7 VMware hosts I checked for compatibility and sure it runs with 1333Mhz DDR3 from Gen7 just fine. Of course it's a little slower than the 1866 MHz the Gen8 is capable of. But since I got them for free I'm just going to ignore the speed difference. As of now the 1'st CPU has got 6 x 16GB + 6 x 8GB. In total a decent 144 GB of RAM. I got an identical Set waiting for the 2'nd CPU.  I guess 288 GB Ram should cover some ground
  7. HDD Install
    1. Installed 4 x 4 TB Seagate SATA LFF HDD
    2. Awaiting Delivery of some Drive Cages for 2 more Disks

Currently I'm at this Point of my Endeavour awaiting some deliveries.
Future Points to be Coverd are 

  • CPU Upgrade
    • Remove current 1 Xeon E5-2603 @ 1.80GHz
    • Install 2 x 2 Intel Xeon E5-2680 8 Core Processor 2.7GHz
  • Cooling modification (Noise reduction)
  • Remove Temporary Windows SSD
  • Install dual SSD's
  • Install Proxmox
    • Boot from dual SSD ZFS Mirror.
    • Main Storage 6 x 4TB Sata HDD as 3 x 2 ZFS Mirrors

 

I'll keep you posted when I get futher down the List :-)

Edited by HJB

Share this post


Link to post
Share on other sites
XeonUnicorn

FYI as the pic and post didn't really explain it very well, the green wire is the pwm signal. 

Share this post


Link to post
Share on other sites
XeonUnicorn

A helpful way to think of it is like this:

 

Black - ground

Red - 12v at a variable amp range. When not connected it is redundant. 

Yellow - (earthed) it tells the BIOS I'm a fan and I'm spinning. It also detects rpm and signals red to supply amps accordingly. 

Green - redundant when not connected. When connected it will pwm according to relevant tmp sensor. 

 

The idea is that when you give fans an alternate standard 12v supply it will be at the amp range the fan wants and not at a possible higher amperage thus destroying your fan or not working well with the hijacked pwm signal. 

Share this post


Link to post
Share on other sites
HJB

Thanks, I'll put the good old osciloscope on those leads with the original Fans to confirm their behaviour.

Share this post


Link to post
Share on other sites
XeonUnicorn

Cool the results of that would be good reading and helpful to clarify the relationship of the pwm and "tach" signal. Some testing of fans 1 and 4 pwm signal against the ram temp sensing which can be seen in speedfan would also help to confirm if it is a separate pwm from fans 2 and 3. 

Share this post


Link to post
Share on other sites
HJB

Got to do some first measurements today and was in for a suprise. 

 

According to Intel the PWM Signal is specified as follows:

 

PWM-Specs.png.5883cdf4c0a0502c6d8cd65fb37fd7f5.png

 

Well it seems easy near continuous 5V ath the PWM Lead gives Max RPM and near continuous 0V is Fan Minimum. So let's see what we've got a little while afther booting with all Fans running at steady 27% according to ILO219122656_2020-02-2918_45_28-iLO_WIN-MB6SF2RS143-ILOCZ3235V73H_home.png.eec02b5fcc4eb39b3559f156514c1c6e.png

  1. In Pink we see a nice steady 12 V supply nothing going on there.
  2. In yellow we see our PWM Signal. But wait that's a lot of 5V and little 0V 
    72,65% of 5V to be ecxcactly. So according to Intel our Fan should be at 3/4 of fullspead. 

SDS00010.png.eb78886fb003173239365c6751f51ef7.png

 

Lets confirm this with the heat gun and heat up the lower part of the server near the PCIE Slots for CPU 1

 

So her it's running with Fan4 at 94% according to ILO. And sure enough its only 7,3% of 5V. 

(Dont mind all the spikes on the traces, the stock fans are noisy even electrically noisy when the ramp up)

SDS00011.png.c798cef7df779978120726af9a4a7874.png

 

So to the ones of you who have just used the PWM Signal from the Green wire. Your FAN is slowing down when it needs to spin up. 

Now wouldn't that make for nice results when stresstesting the CPU 😈

 

So we need to invert the signal to get it usable. I'll be back when I get to it.

 

Oh and the sources for the PWM Signal come from a whole lot of Sensors with each fan reacting to the sensors of it's own Lane and it's direct neighbours. It's save to use Fan 1 & 4 PWM Signals for Case Fans and Fan 3 & 4 PWM Signals for the CPU Coolers.

569348058_2020-02-2918_12_01-iLO_WIN-MB6SF2RS143-ILOCZ3235V73H_home.png.6fa4041e16009d0aab6f5d7eb20e2d62.png

Share this post


Link to post
Share on other sites
ook4mi

I think I'm game over... My HP ML350p G8 won't run anymore, when I checked in ILO. I have Critical Health and log say : " System Power Fault Detected " . I'm running under last bios 😕 and I did a clear on switch 6 but it still not booting up...

 

Did I kill my machine .? I think and feel pretty down... 😕

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • Sokoban
      By Sokoban
      Hi 
      My HP Proliant Microserver Gen 8 won't install HP Tools when I have running HP Smart Update Manager. 
      even when I have trying to install via Intelligent Provising, Windows Server 2019.
       
      Se image below 🙂 That's I missing.. ;-(
       

    • Karazzi
      By Karazzi
      Hi everyone.
       
      New to this forum.
      I have a gen8 microserver running at home, where I have installed an SSD in the ODD sata slot. On the SSD I have installed Ubuntu Server 18.04.
      I have then created a logical volume witht the raid controller and can then boot to the SSD. So far so good.
      My problem emerges when I reboot the system. The logical volume disapears and it gives the error: "boot logical drive is configured but is missing or offline".
      I then have to go into the utility and reconfigure the logical volume. Kind of annoying, when trying to run a headless setup.
       
    • Mark Belanger
      By Mark Belanger
      Greetings,
       
      I'm new to the HPMSG8 world and so far I'm loving it. I have a stock Xeon with 16G of aftermarket memory with iLO patched to 2.70. Next up is adding a RAID card, so I can put my boot SSD on the onboard RAID. I picked up a LSI SAS9211-8i on Amazon, but am struggling to flash it to IT mode on the MicroServer. Is this even possible? Unfortunately, the only other system I have with PCI slots is 32-bit.
       
      Any pointers would be greatly appreciated.
       
       
      -MB
    • iwilltry
      By iwilltry
      Hi there,
      I hope this topic fits here just right.
      So a few days ago, I moved my Microserver Gen8 to a friend's place and resetted it.
      When we set it up with new disks, the raid-creation in OpenMediaVault (Debian 9.6) failed.
      `smartctl` tests passed just fine though and vendor-tools also didn't report any drive errors.
      What happened?
      - I start the Raid creation via `mdadm` and after a while (random blocks and times), it fails and the second disk seems to fail (e.g. sda and sdb are creating a raid, then sdb gets "kicked out" and reread as e.g. sdd, sometimes even with only a few GB of space instead of 3TB).
      I tried attaching the second disk on ata2 and ata3, but it keeps failing. The one on ata1 works just fine though.
      I tried different disks and vendors, but get the same results.
      There are some errors in `dmesg`, but I don't know what they mean. I pasted them here: https://pastebin.com/raw/XASQJ6cy
      What I did here: I stopped the raid to re-create it, but then didn't do anything for a while and the error still came up.

      Can someone help me with this? (As a wild guess, I'd say that somethings wrong with the SATA connections at least on ata2 and ata3).

      Thanks in advance
    • stoney73
      By stoney73
      Wonder if anyone can point me in the right direction here. Have just replaced the CPU on my gen 8 microserver. Went according to the video above. Upon booting the server up, there is a clicking sound coming from the main board. Regular as clockwork, and the server isn't posting. Any ideas? I've swapped back to the G1610T CPU, but still the same issue. And just make it interesting, I don't have access to a monitor or keyboard.
       
      https://photos.app.goo.gl/zyLJWBZQ4mTQ818o2


×
×
  • Create New...