Jump to content
RESET Forums (homeservershow.com)

Suspected slow disk performance on B120i with ESXi 5.5 on Gen8


r5e
 Share

Recommended Posts

Problem Summary: ESXi 5.5 on Gen 8 runs VMs impossibly slowly with much freezing/lagging, suspected due to disk access. While ESXi boots & runs OK, VMs take forever to copy to the datastore and when run, they work OK, but they are just so slow they are almost unusable. They seem to work OK for about 30 seconds, but then they spend another minute just frozen, then they spring back and work well for another minute, freeze again and just keep doing this. Also, when using the standalone converter to migrate a VM from old ESXi host to Gen8 ESXi, disk access also seems to pause constantly, make the transfer painfully slow (12ish hours for an 80G VM).

 

Background

I’ve had an N36L Microserver running ESXi 5.0 for many years now happily hosting a few VMs I need. I’m only looking to use the free version. I’m looking for simple, cheap, as little power as possible and reliable (which the old one was, albeit no RAID support).

 

One of the VMs I now need requires an Intel CPU and simply won’t run on this AMD box. The Gen8 looked like the perfect choice to upgrade to. The intention is to migrate the VMs to the Gen8 and then retire the N36.

Recently I bought a Gen8 2020, Kingston 8G KTH-PL316E (so 10G total with 2G it shipped with) and added 2 WD Green 2Tb drives I had lying around (WD20EARX and WD20EARS – both SMART OK). I installed the 2 x 2Tb in slots 1&2 and created a RAID 1 array.

 

I went into Intelligent Provisioning and ran the updates on everything I could. BIOS is J06, B120i 3.54.0, Intelligent Prov v1.60.1 (the 1.61(B) update offered won’t install), iLO v4 2.00, Broadcom 2.13.5.

 

Then I downloaded the custom images of ESXi from HP, first 5.5 U1 and more recently I’ve also tried U2. I had little success using Intelligent Provisioning with USB for install, so I just created bootable USBs and booted directly to them, which worked fine. I’ve also put in a 32G MicroSD which works fine, but still with the same problems of slow VMs. Recently I’ve signed up for a 60 day evaluation of the iLO “essentials” license, so I’ve now also tried installing via Intelligent Provisioning, mapping the ISO as a virtual drive, which worked OK. Still has the same slow VM problem though.

 

The VM datastore is on the only array, (the 2Tb raid 1 mentioned above). The same poor results are present when ESXi is on the SD, or on the array.

 

Questions:

  1. Can anyone recommend the easiest & quickest way to do some basic diagnosis so I can determine that I have not got a dud box? I had thought I might load W7 (which I now read is not really possible), or Ubuntu on there, but it looks like I may have to mess around with drivers for the B120i a little.
  2. The ability to run disk speed checks from my VMs seems limited. Suggestions I read elsewhere for disk speed apps recommended creating a LUN to use, but this gets difficult in the increasing limited area of free VMWare. Can anyone recommend a method for testing drive/array performance from scratch (including which OS to install)?
  3. Any other easy suggestions for running tests?
Link to comment
Share on other sites

I thought I'd be thorough, but surprised no suggestions yet.

 

I have since tried to install HyperV 2008 via Intelligent Provisioning, which failed near the end. I have also tried re-installing ESXi 5.5 to MicroSD which installs OK, but I am still seeing very poor disk speeds. Yesterday I installed a 1TB drive in my old N36L Microserver and then used SSH to command line copy some VMs to it. Speeds were not record breaking, but we got 1Tb worth of data in the day and it chugged along as I expected it to. Today, I installed the same 1TB dive in slot 3 of my Gen 8 and then using the VSphere Client, I was able to browse to the datastore and copy/paste 1 of my VMs (about 80Gbs) to the 2Tb RAID 1 array that is on slots 1 & 2. That was just over 10 hours ago and there is still nearly 2 hours to go!

 

Tomorrow, I will try the same operation from the command line and see if the speeds are still as awful.

 

If anyone has any useful suggestions of things to try or tests to run, I'd be grateful.

Link to comment
Share on other sites

I've started copying the same stuff via command line and looking at my performance graphs (listing read/write speeds of the storage adaptor), it is showing similar extremely poor speeds, less than 10% of the old microserver (with big lags of nothing or no transfer inbetween). I'm going to leave it running and I expect it will also take forever to complete.

 

Please, I'd welcome suggestions here. Do I have a dud machine? Is the B120i meant to be this bad? Is there anything else I should be trying?

Link to comment
Share on other sites

Hmmm. True. I had not thought to look there. With each cable being mostly "built in", I have not thought to run through a process of elimination. It could take a while though to unpick each one. Perhaps I could try simply configuring single disks in the B120i for each slot and then running simple transfer speed tests on each one.

Any other ideas?

Link to comment
Share on other sites

Slow going here, but I ended up breaking the array into 2 single disks. I pulled one, then swapped and installed ESXi on each at a time. I then got a 1G file and uploaded it to the datastore via vSphere Client & timed it and did the same for download, while watching the network traffic on my PC from task manager and the storage array read and write rates in the performance graphs of the ESXi host. For the 1st disk, I got the same problems. The transfer would have long pauses and I could see the network traffic stop for up to 30 secs, then kick in again. The second disk was reliable. I then swapped the first disk to the second slot and tried again - fault duplicated.

 

Now I had said SMART tests were OK and they were, however, only the quick tests, not the full. I used the diagnostics in Intelligent Provisioning and ran a full test on the suspected bad disk. It failed pretty quickly on a sustained read test. It is now removed and I have found another pair of 1Tb disks and built a RAID 1 array with them instead.

 

I then inserted my VM backup disk in the 3rd bay and began a file copy of nearly 1Tb of VMs to the new datastore. It took about 12 hours to transfer disk-disk on the old server and I expect it will be close to this on the Gen8 as well (though the read & write rates shown are slightly higher). I think this problem is now solved, but the proof will be when I try to run VMs on it.

 

Lesson learned: never assume old disks to be good based on initial SMART tests. The time for diagnosis was a little hampered by this fussy box with lacking drivers for many of the easy OS options, but I got there in the end. (In any other system, I would have loaded Ubuntu Live CD, or installed W7 or something, but the B120i RAID made this not possible).

Link to comment
Share on other sites

Glad you've got things working. Situations like yours are why I always run burn-in tests of HDDs, new and old (SpinRite if the drives are 2TB or smaller, at least until the new SpinRite is released; otherwise HDtune).

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...