Jump to content
RESET Forums (homeservershow.com)
Altecice

Low I/O performance with ESXi 6.0

Recommended Posts

GotNoTime

As you can see the installed version of the Network is 1.34 but the latest is 2.13.5, I have tried 3 times to let it upgrade and it succeeds with a code 1, Then it tried to re scan again for more updates but cant connect and so I reset my server and its still at 1.34.

You're confusing the firmware version and the update package version.

 

2.13.5 is the version of the Linux Broadcom update package you're running. It contains 1.34 firmware for the Broadcom 332i NIC which is what it is showing you.

 

However, you appear to be running the old SPP. There is a newer update package 2.14.17 which contains 1.3.5 firmware. It doesn't help with your problem though as the change is related to multicast MAC filtering which isn't going to affect you. You can download the update package from the HP site if you really do want it.

 

The latest SPP was removed from the HP download site without any notification as to why. Our best guess is that there is a bug or problem in the latest SPP which is why HP took it down.

If you are having trouble updating with IP you may want to try the SPP route:

http://homeservershow.com/forums/index.php?/topic/9116-new-service-pack-spp-v2015030-is-out/

HP took down the SPP download for unexplained reasons. The website never let you download it but you could grab it via FTP for about a day. It disappeared from the FTP site the day after. I'm guessing there is a bug or problem with the new SPP so it probably isn't a good idea to run it if you do get a copy.

Share this post


Link to post
Share on other sites
Altecice

You're confusing the firmware version and the update package version.

 

2.13.5 is the version of the Linux Broadcom update package you're running. It contains 1.34 firmware for the Broadcom 332i NIC which is what it is showing you.

 

However, you appear to be running the old SPP. There is a newer update package 2.14.17 which contains 1.3.5 firmware. It doesn't help with your problem though as the change is related to multicast MAC filtering which isn't going to affect you. You can download the update package from the HP site if you really do want it.

 

 

Ohh I see, Well that's another option ticked off the list, I am really starting to run out of ideas as to what the issue could be now.

Share this post


Link to post
Share on other sites
GotNoTime

Ohh I cant edit the above post anymore so here is a picture of the firmware upgade page I was going to post above: 

 

http://i.imgur.com/9CczmUQ.png

 

As you can see the installed version of the Network is 1.34 but the latest is 2.13.5, I have tried 3 times to let it upgrade and it succeeds with a code 1, Then it tried to re scan again for more updates but cant connect and so I reset my server and its still at 1.34.

Your system ROM and iLO are both old. Your system ROM appears to have been intentionally downgraded since the backup is the latest version?

Share this post


Link to post
Share on other sites
Altecice

Your system ROM and iLO are both old. Your system ROM appears to have been intentionally downgraded since the backup is the latest version?

 

I did notice this before, my system ROM is dated older than the backup ROM. I guess I can try to update them both again and see if that upgrades them correctly via Intelligent Provisioning. I would use the offline SPP but as above it was pulled.

Share this post


Link to post
Share on other sites
GotNoTime

I've reinstalled my gen8 microserver twice now. First time was running CentOS 7 directly on it and second time was running CentOS 7 inside ESXi 6.0. I'm using 2x3TB WD Reds in RAID1 on the HP b120i fakeraid controller.

 

ESXi VM write = 20MB/s

ESXi VM read = 20MB/s

No ESXi write = 140MB/s

No ESXi read = 40MB/s

 

It looks like the previously mentioned issues with the b120i driver in ESXi and the b120i in general do exist. My P222 card hasn't arrived yet so I can't test it.

Edited by GotNoTime

Share this post


Link to post
Share on other sites
Altecice

I've reinstalled my gen8 microserver twice now. First time was running CentOS 7 directly on it and second time was running CentOS 7 inside ESXi 6.0. I'm using 2x3TB WD Reds in RAID1 on the HP b120i fakeraid controller.

 

ESXi VM write = 20MB/s

ESXi VM read = 20MB/s

No ESXi write = 140MB/s

No ESXi read = 40MB/s

 

It looks like the previously mentioned issues with the b120i driver in ESXi and the b120i in general do exist. My P222 card hasn't arrived yet so I can't test it.

 

I am getting the same thing using the inbuilt controller as well as AHCI mode within ESXI. I get full gbit speed until its filled the RAM cache 'modified' RAM and then my transfer speed will plummet. I have not tried without ESXi however.

 

Did you get continual good speeds on large transfers on just a host OS and no hypervisor? If you did I guess its somthing to do with ESXi its self.

 

---

 

I just tried upgrading again, system ROM is saying its installed and up to date (current/new date is 11/09/2013)

I believe ILo upgraded however

Edited by Altecice

Share this post


Link to post
Share on other sites
saq

Performance under ESXi is not an ESXi issue, its a fakeraid card issue. Fakeraid cards lack the actual processing chips to ruin the raid calculations and io transactions at the same time and rely on the operating systems driver stack to pass the commands to the CPU to be handled. The driver subsystem of the ESXi hypervisor is where this would get passed to the CPU, but the hypervisor is intentionally resource restricted so as much of the resources go towards processing VM operations as possible, even if you have no VMs on, the hypervisor is just there to direct IO to the appropriate HBA. This means the IO transactions sent to the b120i only have the mediocre processing power of the little processor of the b120i, which as many people have seen is crap in RAID mode.

I was able to do ~400MB/s sequential read and ~400MB/s sequential write with a Vertex3 SSD with the b120i in AHCI mode using port0 on the 8087 port, which is a 6g port. It wasn't quite as fast as JBOD or RAID1 mode on the M1015, but it was fairly close.

If you are having poor AHCI performance under ESXi I would suggest to update all of your firmware and make sure you are using the HP customized ESXi 6 iso.

Share this post


Link to post
Share on other sites
Ricimer

Is this a 2012 R2 DC? What is the client? Try pulling the file from the server rather than pushing from the client.

 

You may need to turn smb signing off on the 2012 R2 VM (and reboot), for me on a HV machine this changed 40MB/s to 110MB/s, it may not be a Microserver issue, but a VM guest one.

Share this post


Link to post
Share on other sites
Altecice

Performance under ESXi is not an ESXi issue, its a fakeraid card issue. Fakeraid cards lack the actual processing chips to ruin the raid calculations and io transactions at the same time and rely on the operating systems driver stack to pass the commands to the CPU to be handled. The driver subsystem of the ESXi hypervisor is where this would get passed to the CPU, but the hypervisor is intentionally resource restricted so as much of the resources go towards processing VM operations as possible, even if you have no VMs on, the hypervisor is just there to direct IO to the appropriate HBA. This means the IO transactions sent to the b120i only have the mediocre processing power of the little processor of the b120i, which as many people have seen is crap in RAID mode.

I was able to do ~400MB/s sequential read and ~400MB/s sequential write with a Vertex3 SSD with the b120i in AHCI mode using port0 on the 8087 port, which is a 6g port. It wasn't quite as fast as JBOD or RAID1 mode on the M1015, but it was fairly close.

If you are having poor AHCI performance under ESXi I would suggest to update all of your firmware and make sure you are using the HP customized ESXi 6 iso.

 

I have always been using HP images of 6.0 and 5.5.0.

 

Is this a 2012 R2 DC? What is the client? Try pulling the file from the server rather than pushing from the client.

 

You may need to turn smb signing off on the 2012 R2 VM (and reboot), for me on a HV machine this changed 40MB/s to 110MB/s, it may not be a Microserver issue, but a VM guest one.

 

 

I've reinstalled my gen8 microserver twice now. First time was running CentOS 7 directly on it and second time was running CentOS 7 inside ESXi 6.0. I'm using 2x3TB WD Reds in RAID1 on the HP b120i fakeraid controller.

 

ESXi VM write = 20MB/s

ESXi VM read = 20MB/s

No ESXi write = 140MB/s

No ESXi read = 40MB/s

 

It looks like the previously mentioned issues with the b120i driver in ESXi and the b120i in general do exist. My P222 card hasn't arrived yet so I can't test it.

 

 

If you are having trouble updating with IP you may want to try the SPP route:

http://homeservershow.com/forums/index.php?/topic/9116-new-service-pack-spp-v2015030-is-out/

 

 

----

 

Ok so, Just out of curiosity I wiped everything and put all the logical raid 0 arrays back onto the raid controller (so from AHCI back to the on board RAID). I installed Windows Server 2012 R2 straight onto a 500gb disk and I retested with the same setup as before... 

 

Laptop drive: 112-50mb/s (sustained write is around 75mb/s so this is expected)

Desktop drive flat 112mb/s (full 1gbit speeds)

 

Going back to the original issue about the modified RAM going out of control and throttling the download. When not using ESXi the system can keep it under control. So in conclusion I can only say that it was ESXi being the issue. Its a bit of a pain but I guess I better get learning Hyper-V :P.

 

Any thoughts on this?

 

Thanks for everyone's help!

Edited by Altecice

Share this post


Link to post
Share on other sites
schoondoggy

Not to be nit picky, but it is a good idea to use 'b' for bit vs 'B' for byte. 

100 years from now it will help our robot overlords understand how they came to exist,,,,,

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×