Dave

WiFi Testing Standards

31 posts in this topic

I recently had a "Fail" in some wifi testing and chatting with you guys here on the board brings me to something I have wanted to do for some time.  Testing and Measuring standards.  Not just wifi but testing and measuring in general.  I propose we gather up a set of standards for testing here at HSS Forums and reporting as well.  Let's start with what is fresh on our minds and that is Wifi testing.  I want to also cover LAN speed, disk speed, Server speed tests, NAS speed tests, etc.  If we set a standard in which we test as well as a standard in which we present the data, we can compare a little better.

 

One thing we have to agree on is that we all can't afford major testing tools or laboratories.  We cant all build anoechoic chambers for wifi testing or purchase ixchariot.  We can however, recommend tools, test distance, setup, test processors, and certain methods of reporting in order to make it happen.  

 

If you want to assemble a team and take volunteers for such endeavors I would be happy to set up some private forums where you can hash out the data and reporting forms.  Otherwise, open forum it is.  There will be disagreements so just chill and recognize that we all won't agree 100% on the process.  There will have to be some give and take.  It's possible we could agree on a "good, better, best" test method and incorporate a little of all your ideas.

 

Take my recent fail in wifi measuring and run with it.  We need to be able to test routers, AP's, wifi cards, wifi USB adapters, etc.  Maybe start with AP's since that is what is hot at the moment.  Eight foot test, one drywall away at 10 feet, 20 feet, etc.  What to use to run the test. What to use to monitor strength of signal. How to report all this back. Do we do it with a desktop and wifi card, or a phone, a laptop, etc?  Geez, after typing this all out I'm thinking it's going to be quite the feat to accomplish!  What do you guys think?

Share this post


Link to post
Share on other sites

I am a glass 1/2 empty sorter so bear with me.  One issue is going to be your neighbors.  I live in a community where our houses are on top of each other.  I can pick up about 15 wireless SSID's from inside my house.  This many wireless radios have to be creating some issues on my bandwidth.  My old house was out in the country.  I could only see my own network.  That alone is going to create some differences.

 

I still think there is merit to doing this but there are many non-obvious variables that may mess around with the results.

Share this post


Link to post
Share on other sites

We also struggle with closeness of neighbours, I cant see 15 SSID's though, thats harsh.. then there's Bluetooth on everything as well these days..

Share this post


Link to post
Share on other sites

So true.  Rural Wisconsin i saw ZERO and here I constantly see new SSID's.  

Share this post


Link to post
Share on other sites

Consistency.

Are we measuring WiFi or Internet speed? I like Speedtest for Internet and file transfers for throughput, but it is best to use different files each pass to rule out caching.

Same tools before and after changes on the same devices you will be using for comparisons. Xirrus WiFi inspector gives good connect and signal strength data.

I had four laptops setting on the same table running the same tests all with very different numbers. Tablets are all over the place. 

Connect speed versus throughput? Even though you are connected at a high speed you will not transfer at that speed..

 

Last test, get every desktop, tablet, laptop and console in the house streaming something. Are you saturating your Internet? WiFi?

1 person likes this

Share this post


Link to post
Share on other sites

I pretty much do what schoondoggy does for testing.  One point to make is the testing will be against yourself and it is really hard to compare against anyone else due to too many variables.  Test your old setup and then the new one exactly the same way preferably around the same time and you should have pretty good results.  One last thing is make sure you test latency.  Wireless, especially with a mesh network can be heavily impacted with latency and it is good information to know.

Share this post


Link to post
Share on other sites

I recently had a "Fail" in some wifi testing and chatting with you guys here on the board brings me to something I have wanted to do for some time. Testing and Measuring standards. Not just wifi but testing and measuring in general. I propose we gather up a set of standards for testing here at HSS Forums and reporting as well. Let's start with what is fresh on our minds and that is Wifi testing. I want to also cover LAN speed, disk speed, Server speed tests, NAS speed tests, etc. If we set a standard in which we test as well as a standard in which we present the data, we can compare a little better.

 

One thing we have to agree on is that we all can't afford major testing tools or laboratories. We cant all build anoechoic chambers for wifi testing or purchase ixchariot. We can however, recommend tools, test distance, setup, test processors, and certain methods of reporting in order to make it happen.

 

If you want to assemble a team and take volunteers for such endeavors I would be happy to set up some private forums where you can hash out the data and reporting forms. Otherwise, open forum it is. There will be disagreements so just chill and recognize that we all won't agree 100% on the process. There will have to be some give and take. It's possible we could agree on a "good, better, best" test method and incorporate a little of all your ideas.

 

Take my recent fail in wifi measuring and run with it. We need to be able to test routers, AP's, wifi cards, wifi USB adapters, etc. Maybe start with AP's since that is what is hot at the moment. Eight foot test, one drywall away at 10 feet, 20 feet, etc. What to use to run the test. What to use to monitor strength of signal. How to report all this back. Do we do it with a desktop and wifi card, or a phone, a laptop, etc? Geez, after typing this all out I'm thinking it's going to be quite the feat to accomplish! What do you guys think?

Dave, I just listened to your last podcast and I think that iperf is a better tool to do local testing.

 

My experience has been that using iperf yields pretty accurate results. You can run iperf in a way that you measure memory to memory speeds thus one less thing to worry about.

 

Some APs (like ubiquiti) have iperf server built into the code. However there is a performance impact by running a client to AP test since processing resources are required to run iperf.

 

The problem with real-life wifi testing is that the results are extremely specific to that network, client and environment (distance from AP, obstructions, SNR, etc) combination due to all the factors at play. So I am not sure that a test would mean much to others.

Edited by diegolrz
1 person likes this

Share this post


Link to post
Share on other sites

One other item of note, when you are testing WiFi with mobile devices, be sure your body is not between the device and the access point. The human body is a fairly good filter of WiFi. The bigger the body, the better the filter! :o

Share this post


Link to post
Share on other sites

One other item of note, when you are testing WiFi with mobile devices, be sure your body is not between the device and the access point. The human body is a fairly good filter of WiFi. The bigger the body, the better the filter! :o

Thanks & noted :)

Share this post


Link to post
Share on other sites

Currently I am using LAN Speed Test V:3.5 

http://totusoft.com/lanspeed/

I have it pointed at a network share on a SSD in a small PC, to keep drive latency from effecting the network speed numbers.

Share this post


Link to post
Share on other sites

Currently I am using LAN Speed Test V:3.5 

http://totusoft.com/lanspeed/

I have it pointed at a network share on a SSD in a small PC, to keep drive latency from effecting the network speed numbers.

Nice little program, Been using it for years.

Share this post


Link to post
Share on other sites

So I have been trying to run some basic tests.  One thing I noticed was that once I got some distance between myself and the AP that I switched from 5Mhz radio to 2.4Mhz.  This significantly impacted throughput.  The phone I am using to do these tests seems to want to lock onto 2.4Mhz once it determines that 5Mhz isn't good enough.  I can go back near the AP and turn off WIFI or even switch to a different WIFI network and then back and it won't connect to 5Mhz.  A power cycle seems to be the only fix.

 

Any thoughts?

Share this post


Link to post
Share on other sites

So I have been trying to run some basic tests.  One thing I noticed was that once I got some distance between myself and the AP that I switched from 5Mhz radio to 2.4Mhz.  This significantly impacted throughput.  The phone I am using to do these tests seems to want to lock onto 2.4Mhz once it determines that 5Mhz isn't good enough.  I can go back near the AP and turn off WIFI or even switch to a different WIFI network and then back and it won't connect to 5Mhz.  A power cycle seems to be the only fix.

 

Any thoughts?

What kind of phone is it? WiFi tools in phones are getting smarter about grabbing the strongest signal.

I have been testing with a Surface 3. I need to check if Windows 10 is doing any auto switching.

Share this post


Link to post
Share on other sites

One thing I noticed was that once I got some distance between myself and the AP that I switched from 5Mhz radio to 2.4Mhz.

I assume you mean 2.4GHz and 5GHz. Propagation is significantly worse the higher you go in frequency as objects in the way will absorb the RF energy more. 5GHz WiFi is less congested because of more channels and just because the signals from your neighbours just don't go that far. At an equal distance from the AP, you'll get a stronger 2.4GHz signal than a 5GHz signal. Your phone is preferring the stronger signal despite the higher congestion.

 

The phone I am using to do these tests seems to want to lock onto 2.4Mhz once it determines that 5Mhz isn't good enough.  I can go back near the AP and turn off WIFI or even switch to a different WIFI network and then back and it won't connect to 5Mhz.  A power cycle seems to be the only fix.

The device itself determines what AP to connect to and what specific frequency band it wants to use. The usual algorithm is just to pick the stronger signal available and the AP can't influence this decision normally. There are some new extensions (802.11k) to the WiFi standard which are designed to make this work better by sending out information about frequency bands + utilisation but it is optional + new and I don't believe any consumer grade WiFi gear supports it yet. The fancy bandsteering features work by attempting to trick the device. If it sees your device on 2.4GHz and 5GHz then it'll stop replying to it on 2.4GHz in an attempt to force the device connect to 5GHz. However, some devices don't like this and you'll see it taking a long time to connect to WiFi as eventually the AP gives up trying to force it to 5GHz and lets it onto 2.4GHz. This is why bandsteering is usually not enabled by default.

 

The low tech fix is just to have two different SSIDs being broadcast. One is your 2.4GHz SSID and one is your 5GHz SSID. Your 5GHz devices will stay on 5GHz unless the signal completely drops out.

Share this post


Link to post
Share on other sites

Yeah I meant GHz but was typing quick on my way out the door.

 

The Eero devices have both 5Ghz and 2.4Ghz radios but they use the same SSID. This phone I am using will start on 5Ghz but will switch to 2.4Ghz and will not switch back.

Share this post


Link to post
Share on other sites

The Eero devices have both 5Ghz and 2.4Ghz radios but they use the same SSID. This phone I am using will start on 5Ghz but will switch to 2.4Ghz and will not switch back.

Eero don't appear to support bandsteering and it isn't listed as a coming soon feature either.

 

There isn't much you can do without bandsteering support or the ability to have a different SSID for each frequency band. You just have to put up with how each device decides to handle its WiFi connections.

Share this post


Link to post
Share on other sites

I went a different route.  I bought a Netgear AC1200 high gain WIFI USB adapter.  With their Genie software I can tell the adapter which network I want to join.  It even shows me the channels and BSSID so I know which Eero I am connected to.  Now to do some experimenting as I wander around the house and see if it switches to the other AP or what will happen.

 

BTW - there have never been wireless deadspots in my home even when I used a single Airport Express for my WIFI.  My biggest problem was the connection would get really slow when I was on the 2nd floor.

Share this post


Link to post
Share on other sites

What kind of phone is it? WiFi tools in phones are getting smarter about grabbing the strongest signal.

I have been testing with a Surface 3. I need to check if Windows 10 is doing any auto switching.

Even though that maybe true I feel there are to many problems with smartphones to be used for WiFi testing. Better to use a laptop or maybe a Surface something with a Windows/Mac OS will give you the best results.

Edited by itGeeks

Share this post


Link to post
Share on other sites

Eero don't appear to support bandsteering and it isn't listed as a coming soon feature either.

 

There isn't much you can do without bandsteering support or the ability to have a different SSID for each frequency band. You just have to put up with how each device decides to handle its WiFi connections.

No bandsteering on Eero and its not on there road map? That's not good.

Share this post


Link to post
Share on other sites

Even though that maybe true I feel there are to many problems with smartphones to be used for WiFi testing. Better to use a laptop or maybe a Surface something with a Windows/Mac OS will give you the best results.

I test with a Surface 3. I am not advocating testing with a phone.

Share this post


Link to post
Share on other sites

No bandsteering on Eero and its not on there road map? That's not good.

Neither eero nor Luma support band steering. It goes with the whole "let us handle that for you" mantra.

Share this post


Link to post
Share on other sites

Neither eero nor Luma support band steering. It goes with the whole "let us handle that for you" mantra.

I respectfully disagree with you, If you go back and listen to the interview Dave had with Dr P Judge he clearly stated that Luma does 'band steering' as part of describing how Luma works. That said 'band steering' does not seem to be working or at least working 100%

 

Here is another review though its from Jan 2016 but all the key points there talking about the Dr also confirmed on the interview http://www.thinkcomputers.org/luma-the-all-in-one-wifi-solution/

 

It all comes down to waiting now and giving Luma time to catch up with there promise, I believe they will get there hopefully sooner rather then later.

Edited by itGeeks

Share this post


Link to post
Share on other sites

I test with a Surface 3. I am not advocating testing with a phone.

I was responding to cskenny using a smart phone for testing when I quoted you, I only quoted you because you don't use a smart phone for testing as you use a Surface 3 so I was just trying to reinforce you in saying using a smart phone for testing WiFi is not the best option.

 

Sorry for the confusion....

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now