Jump to content
RESET Forums (homeservershow.com)
seiburii

ESXi - SSH and Web UI slowness

Recommended Posts

seiburii

I have two Gen 8s, one of which I have been having problems with connection via SSH and Web UI, also the ping to it drops often.

The trouble is both servers are connected much in the same manner, so I believe this to be a ESXi software related issue however I am open to any ideas on this (please see my current Network Diagram link for more info).

 

 

Server 1 - not working - click here for ESXi configuration details

CPU - E3-1265L V2

RAM - 16GB

PCI-e: HPE Smart Array P420 + 1GB FBWC

ESXi boot device: MicroSD (see configuration details section "[2.3]")

Connection: Single Shared Port 1 (click to view iLO configuration)

Problems:

  • SSH - login connection is slow, readily times out without connection.
  • SSH - When SSH login does work, freezes typically then times out.
  • WebUI - Slow to login, at times requires refresh to actually show the resulting screen.
  • WebUI - Slow/Freezes to access different navigation options - really hard to navigate at all
  • WebUI - Slow to check Datastores and create new ones.

Notes:

  • iLO4 connection works correctly without issues. I can control Host using the ESXi Shell (via iLO and typing "Alt + F1")

 

Server 2 - workingclick here for ESXi configuration details

CPU: E3-1220L

RAM: 12GB (differing manufacturers)

PCI-e: H220 HBA

ESXi boot device: MicroSD (see configuration details section "[2.3]")

Connection: Single Shared Port 1 (click to view iLO configuration)

Problems:

  • none found
  • SSH works promptly and even continues to work if left for a duration of time
  • iLO4 connection works without any issues.

 

Example Ping Test - click here to view - https://ibin.co/4eTaCXbYZDIY

Network setup 1 (desired/current) - https://ibin.co/4eoqHQe83V8o

Network setup 2 (temporary to test Server1) - https://ibin.co/4eov6X9JbR61
ILO4 settings (for both Servers) - https://ibin.co/4eptFfLzZKP5

 

 

I am not sure exactly what the problem is with Server 1, but was wondering if anyone can suggest/help me with this.

 

Other Related Questions :
As my Powerline Adapter takes only 3 ports, I have to use one port for each of my Gen8s in Network Setup 1; Does using Shared ESXi/iLO port effect ESXi ping at all (although this does not occur on my Server2 - so I assume not)?

Share this post


Link to post
Share on other sites
pumek

Check nic firmware version.

Bios version. Ilo version.

 

Check LAN cables

 

 

Try connect both directly to switch

 

 

Swap file is on sd card ? Diffrent SD cards?

Share this post


Link to post
Share on other sites
seiburii
Posted (edited)
On 4/25/2019 at 12:54 PM, pumek said:

Check nic firmware version.

Bios version. Ilo version.

 

@pumek

 

Server 1

NIC firmware version: bc 1.39 ncis 1.3.16

BIOS version: J06 05/21/2018 (from smbiosDump "BIOS Info > Date" field)
iLO4 version: 2.62 (previously 2.61, upgraded yesterday)

 

Server 2

NIC firmware version: bc 1.39 ncis 1.4.12.0

BIOS version: J06 05/21/2018 (from smbiosDump "BIOS Info > Date" field)

iLO4 version: 2.62 (previously 2.61, upgraded yesterday)

 

 

So my Server1 has a lower NIC firmware than Server2.

I have now updated Server1 with HPE Service Pack SPP2018060.2018_0618.64.iso  to NIC firmware "ncis 1.4.22.0". I tried to use  HPE Broadcom NX1 Online Firmware Upgrade Utility for VMware - CP036110.zip , managed to copy the CP036110.vmexe, however although /var/cpq/CPxxxxxx.log claimed it was installed, the NIC firmware did not change. So I was not able to install following the guide inside the CP036110.zip.

If anyone knows how to update the Broadcom BCM5720 to the latest firmware, that would be most helpful.

 

 

Even with the upgraded Server1 NIC firmware to "ncis 1.4.22.0" and iLO4 to 2.62 I still experiencing similar issues.

 

 

 

Edited by seiburii
Additional info added

Share this post


Link to post
Share on other sites
seiburii
On 4/25/2019 at 12:54 PM, pumek said:

 

Check LAN cables

 

 

I have swapped the ethernet cable used for my Server2 and Server1 connected to the Powerline Adapter, and was able to use Server2 without issues with Server1's cable. So I do not believe this to be a contributing factor, although please let me know if that test is not sufficient.

 

On 4/25/2019 at 12:54 PM, pumek said:

 

Try connect both directly to switch

I have not got the space/plug sockets to connect both servers to where the router is setup however I have attempted to connect only Server1 directly to the Router via Ethernet (see Network Setup 2). The same ping and symptoms occur.

 

On 4/25/2019 at 12:54 PM, pumek said:

Swap file is on sd card ? Diffrent SD cards?

I will test using Server1's memory card in Server2 and check results.

Share this post


Link to post
Share on other sites
pumek

ive run some tests on my g8 and encountered some freeze only when playing with xpenology vm but after while it all startetd working again (15vms total: 2x windows, 1 xpenology, 12x linux), swap on ssd, non raid sata mode, hdd rdms on xpenology, ilo shared port bc 1.39 ncsi 1.3.16.0, ilo 2.62, sps 2.2.0.31.2

 

 

where did you find ncis 1.4.22.0? spp 2019.03 didnt upgrade anything on my g8

Share this post


Link to post
Share on other sites
seiburii
Posted (edited)

@pumek

4 hours ago, pumek said:

where did you find ncis 1.4.22.0? spp 2019.03 didnt upgrade anything on my g8

 

On SPP2018060.2018_0618.64.iso (SUM 8.3.1 - click name for info, or type into google for this ).

Package Name: firmwware-nic-broadcom-2.21.3-1.1.x86_64

4f9P3F1gH9JT

 


Network Testing

In order to rule out whether this is a hardware related issue, I have tried pinging Server1 with an ubuntu 18.04 Live CD mounted with the Network1 setup, and I was first able to ping successfully without issues or loss of pings.

4f99NJ4POhS3

 

Then the live desktop timed out (screensaver) and once restore then I received a intermittent ping (usually in 5 successful pings intervals)

4f9TKJYksXG2

 

Then after a short while the pings stabilised again😖😖😖....

4f9BTBnX830D

 

...so this could very well be hardware related, but there must be a software component to it, as the ping is typically consistent with the ubuntu live CD.

 

Edited by seiburii
change of image

Share this post


Link to post
Share on other sites
sorted

Here's an extract from my Gen8 firmware. NIC is on 1.3.16.0 and everything is working normal (ESXi SSH, WebConsole etc.)

 

Dynamic Smart Array B120i Controller       5.50    
HP Ethernet 1Gb 2-port 332i Adapter         bc 1.39 ncsi 1.3.16.0    
iLO                                                                     2.62 Jan 09 2019    
Intelligent Provisioning                                  1.63.192    
Redundant System ROM                               J06 07/16/2015    
Server Platform Services (SPS) Firmware  2.2.0.31.2    
System Programmable Logic Device          Version 0x06    
System ROM Bootblock                                 02/04/2012

 

However the ILO console displays a totally different online firmware version that can be installed (2.17.6) from current version 17.4.41. I think that has to do with HP SUM being not backwards compatible with newer firmware. As you can see SUM is also stuck on ILO ver. 2.50.

 

image.thumb.png.73eb05a9c123f259c1d2f8ccb4fc0b08.png

 

 

I will try to upgrade to 1.4.12.0 and see what happens.

Share this post


Link to post
Share on other sites
sorted
Posted (edited)

Done updating the NIC firmware. Same experience as before. No lagging, fast console, fast SSH. However... HP SUM shows it as 20.12.41.

 

@Seiburii: I do see you use ILO via the shared network port. Have you tried using the dedicated ILO port?

 

image.thumb.png.89c485faf14a18842d3f4c5d425c6a67.png

Edited by sorted

Share this post


Link to post
Share on other sites
sorted

BTW: I upgraded the NIC firmware by booting into the HP SUM using ISO 2018.6. Worked like a charm.

Share this post


Link to post
Share on other sites
seiburii

Thanks @sorted and @pumek for your help.

I think I have found out my issue...It was one of these two aspects: 

  • IP Conflict with Router reserved IPs and swapping of MicroSD media between Servers confusing the expected IP per Ethernet MAC Address.
  • MicroSD of low quality and USBs not being bootable

IP Conflict

Due to my booting issues:

  •  finding and installing ESXi to multiple MicroSDs and testing them between my two Gen8's
  • using Shared Network Ports for both
  • alongside using Static IPs when my router was configured to also assign IPs

I believe the combination caused the 

I have since set IPs to be dedicated in the router and allow my Gen8s to DHCP receive their respective IPs in a Shared Network Port setup. This appears to be working correctly. 

 

Trouble ESXi booting using USBs (as well as USB3.0)

I have been suffering from another problem (which some others also faced on other forums with new Gen8's) regarding installation of ESXi onto USBs and booting up. Up till now I have only been able to consistently boot with ESXi when installed onto my MicroSDs (not my USBs, or which I had USB 3.0 and  then USB 2.0 available). Alongside that (which I believe may have been a contributing factor), the quality of the MicroSD may also affect the speed of SSH and Web UI lagging (although I will do another test on a terrible budget MicroSD I own to check if that is the case).Howver I understand that ESXi is written to RAM and so the quality of card may not affect SSH and WEB UI.

 

In any case the way I realised the reason I was not able to boot my USBs after installing ESXi to them was due to the ESXi install preferring to format using the GPT partition table over MBR. From what I have read, the Gen8 does not work that well with GPT with USB, however I have found that GPT seems to be fine for MicroSD boot (which all my current MicroSDs are set to). I was even able to use my USB3 to boot (in the internal slot) which people on other forums have posted having problems with.

Therefore forcing the installation to use MBR formatting is required. This is done by pressing "Shift+0" when prompted and entering "formatwithmbr" as a booting option (a guide demonstrating this is shown here http://sparrowangelstechnology.blogspot.com/2012/11/no-bootable-device-after-successful.html).

 

1203.png

 

 

In any case, both systems are responsive to ping, SSH and WebUI and working atm. So now to undo the damage of an automatic SPP update (when I updated my NIC) and crossflash my H220! 😓

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...