Jump to content
RESET Forums (homeservershow.com)
Poppapete

10G Network

Recommended Posts

itGeeks
8 hours ago, Poppapete said:

itGeeks,
10G switch is Netgear XS505M
I bought new short CAT6A cables for the exercise
I have gone to ASUS site and DL Win7 64 driver package. It has same name and size as the Win10 64 package but I unpacked and setup the drivers again.  Device Manage says 10G Nic is working properly on server.
I removed teaming and double 1G nics from server a week ago.
I removed static IP’s from NICs on server and DT  a week ago but still grabs old IPs in some cases.

DNS servers are ones I get from using Steve Gibsons DNS benchmark tool.  I am in Australia don’t forget.

 

SO.  I have just connected to DT and the Server to the 10G switch again after installing the Win7 drivers.  It has been an hour and DT seem to be connected to the internet but server is not.  I ran the internet trouble shooter on the server but it could not find the problem  I give you ipconfig printout of both 10Gnics.  Can’t wait to have time to pull the 10G card out of the server and put it in a W10Box but that will be next week sometime.

Ethernet adapter Ethernet 2 (Integrated 10G on DT):

   Connection-specific DNS Suffix  . :

   Description . . . . . . . . . . . : Aquantia 10G Ethernet connection

   Physical Address. . . . . . . . . : B0-6E-BF-82-9A-13

   DHCP Enabled. . . . . . . . . . . : Yes

   Autoconfiguration Enabled . . . . : Yes

   Link-local IPv6 Address . . . . . : fe80::a938:2622:773b:6464%7(Preferred)

   IPv4 Address. . . . . . . . . . . : 192.168.0.13(Preferred)

   Subnet Mask . . . . . . . . . . . : 255.255.255.0

   Lease Obtained. . . . . . . . . . : Thursday, 1 March 2018 2:54:17 PM

   Lease Expires . . . . . . . . . . : Thursday, 1 March 2018 5:10:08 PM

   Default Gateway . . . . . . . . . : 192.168.0.1

   DHCP Server . . . . . . . . . . . : 192.168.0.1

   DHCPv6 IAID . . . . . . . . . . . : 129003199

   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-21-88-3F-07-B0-6E-BF-82-9A-14

   DNS Servers . . . . . . . . . . . : 203.0.178.191

                                       203.215.29.191

   NetBIOS over Tcpip. . . . . . . . : Enabled

 

Ethernet adapter Ethernet 3 (PCIe on Server)

 

   Connection-specific DNS Suffix  . :

   Description . . . . . . . . . . . : ASUS XG-C100C 10G PCI-E Network Adapter

   Physical Address. . . . . . . . . : 88-D7-F6-A8-BA-8F

   DHCP Enabled. . . . . . . . . . . : Yes

   Autoconfiguration Enabled . . . . : Yes

   Link-local IPv6 Address . . . . . : fe80::992d:2f87:b47d:3a15%17(Preferred)

   IPv4 Address. . . . . . . . . . . : 192.168.0.14(Preferred)

   Subnet Mask . . . . . . . . . . . : 255.255.255.0

   Lease Obtained. . . . . . . . . . : Thursday, 1 March 2018 2:58:57 PM

   Lease Expires . . . . . . . . . . : Thursday, 1 March 2018 4:58:57 PM

   Default Gateway . . . . . . . . . : 192.168.0.1

   DHCP Server . . . . . . . . . . . : 192.168.0.1

   DHCPv6 IAID . . . . . . . . . . . : 512284662

   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1A-D7-E1-D5-0C-C4-7A-00-C6-AE

   DNS Servers . . . . . . . . . . . : 203.0.178.191

                                203.215.29.191

   NetBIOS over Tcpip. . . . . . . . : Enabled

 

Thanks for the info. Is my eyes playing games with me? Looks like the 10G NIC is now showing connected vs before disconnected, If that's correct we are indeed making headway:D Looks kike your where trying to use the Windows 10 driver before and that was causing you trouble because there where many changes to the network stack in Windows 10. That's said I would like to pause for a moment and focus on getting the foundation of your network sorted before we go any further with this.

 

As good as Steve Gibson's utilities is for bench marking DNS many times its inconclusive on the best DNS server to use and it also says so in the final results. Steve is also anti OpenDNS because he does not like the fact that OpenDNS may redirect you in some cases if a website is not found exactly as you typed it. That said OpenDNS is a very good service and provides DNS caching for quick access and most of all it provides some level of protection to reduce the chance of malware. IF your not running any kind of edge protection on your network I would highly recommend a service such as OpenDNS, The benefits far out way the disadvantages. Turns out OpenDNS has a server located in Sydney Australia so performance should be good.

 

I still have concerns about your network configuration so I would like to spend a bit of time getting that sorted 1st before we move on to more testing and trouble shooting, Hopefully at the end of the next several steps it will solve your problems so bare with me here..

 

For some reason your DT and server are still getting the public IP address for the DNS server and as stated that's not a good idea, a public DNS server knows nothing about your internal network and could/will cause you problems so we need to fix this....

 

1) What I would like you to do is see if there are any firmware updates for your router found here and install it, Make sure you select the correct hardware version from the drop-down list, The hardware version can most time be found on the Tag on the router, If not login into the router for that info. Looks like there is a new firmware dated Feb 7th 2018 for v3 hardware if that's what you have? I would even go as far as doing a router reset after the update, That way you no your clean and back to defaults and that's a good starting point to set things up correctly.

 

2) After the firmware update and or the factury reset login into the router and on the WAN interface setup the IP's of OpenDNS as follows, Primary: 208.67.222.222 Secondary: 208.67.220.220.

 

3) Also consider changing the IP range your using for your network from 192.168.0.x to something like 192.168.99.x The reason is because its not a good idea to use the manufacturer's default range, There has been malware developed to target default ranges in the 192.168.0.x and 192.168.1.x and not only that seeing that these are the most used ranges the chances of you plugging in a new device that's using the same IP as somthing on your network is strong and could crash your network so changing it to something not standard is a win/win.

 

4) now that all that is sorted, Check your DHCP setting on the router and make sure its handing out the new IP of your router to all the clients as the DNS server for your network, It should not be handing out the public IP of the DNS server, This will help you confirm to best practices and limit DNS problems on your network...

 

5) After all that's done open an elevated command aka as administrator on both devices and clear both the DNS and IP settings by typing the following

5a) type ipconfig /flushdns and press enter, (make sure there is a space before the slash/)

5b) type ipconfig /renew and press enter, (make sure there is a space before the slash/)

5c) wait a bit while the DHCP server assigns a new IP

5d) Follow the same steps on the other device...

 

6) After all that's done, Check to make sure OpenDNS is alive and well by opening a web browser on both your devices and go to

http://opendns.com/welcom If configured correctly you will get a message saying "Welcome to OpenDNS", Your internet is now safer bla bla bla.

 

7) Great now we know all that is in good shape lets continue. I would like to no if your ISP is giving you an IPv4 or IPv6 address? if you don't know you can find out on of several ways by clicking here, Please do NOT post your IP address here. All I need to know is if its using v4 it will be something like 123.34.543.32, if its using v6 it will be something like 2001:4860:4860::8844, If your using IPv4 I would highly recommend you disable IPv6 on your router if your able, Both on the WAN interface and LAN interface, I have seen this cause problems way to many times so just disable it if not in use.

 

8) Now open a elevated command prompt on your DK and type ipconfig /all (make sure there is a space before the slash/)

8a) Make sure the IP address for DHCP, Gateway, DNS server is pointing to the routers IP Address, If there is any other address such as the public IP of the DNS server then something is still miss configured and we need to investigate further. If everything looks good on the DT then repeat the process on the Server and please post results.

 

9) Now open a elevated command prompt on your DK and try to ping the server, then try to ping the DT from the server and please post results.

 

On closing I know this all seems daunting but as schoon said early on sometime you need to start with the basics and work out from there or maybe a better way to put it as Dave says, "No matter how much we screw something up, there is usually a reset button" Not saying you screwed something up just saying starting from the edge of your networking doting all the eyes and crossing all the T's will help solve these kinds of problems and if nothing else it will make sure your network is setup correct and performing its very best. Hang in there, There is light at the end of the tunnel. Just be grateful you have this kind of help at a cost of free, Back in the early 90's when I started learning all this there was no internet as we know it today so I either had to learn it myself without the help of others or pay someone to fix it, I chose the former.

 

 

Edited by itGeeks

Share this post


Link to post
Share on other sites
Poppapete

First, I should mention I have another box in the system that is not in the diagram.  A UTM in the form of Untangle (NG HomePro) in bridge mode between the router and the 24 port 1G switch.  I did bypass it for a few days last week when I was testing the 10G switch connected to the 24-port switch but it is now back in service.  I note that I am testing the 10G switch directly connected to the router now so the UTM is not between them.

The Server did connect to the internet for a short time and that’s when I obtained that IP config printout.  At that stage I thought the change to the W7 drivers had solved my problem, but I could not access the server from the DT and when I went back to the server it had lost internet access (but the DT had not and I am still using the DT via the 10G nic and switch).  Then my family’s comlaints about access to file server (I run Plex on the Server) forced me to reconnect the server to the 1G nic and switch.

 

Next week when I get back home I will as you suggest update router firmware and change the DNS server to opendns.

 

I pay for a business class IP service which gives me a fixed IP address that I have known for years.

Edited by Poppapete

Share this post


Link to post
Share on other sites
JackoUK

When these puppies are working I'd find it useful to have some dimensioning guidelines.

In another thread a member is considering a Synology NAS with 4 cores at 2.4GHz and 16GB memory which supposedly maxes out the 10GbE link.

We know that SMB MC can use up to 4 channels per NIC. Questions:

 

1. What is the max we can get out of 1 channel?

2. How does throughput scale with CPU speed. Will a  single 4GHz 8th Gen 8100 core be better than a XEON 2GHz server core? It would be useful to know that, for example, a Ryzen 2200G with 4 cores at 3.5GHz is a capable of driving a 10GbE link, or that a Ryzen 1700 is capable of driving 2 x 10GbE links when dedicated to a file server.

3. How much difference does hyperthreading in a CPU make, if any? 

4. The Cinevate document suggests 32GB RAM, whereas the Synology only uses 16GB. What is the memory consumption of the link in operation? Would 8GB be enough for a dedicated file server machine?

5. No doubt there are other small tweaks to be found.

 

We'll leave RDMA NIC's, RefS and the new Windows for Workstations license for another time. 

 

Share this post


Link to post
Share on other sites
schoondoggy
When these puppies are working I'd find it useful to have some dimensioning guidelines.
In another thread a member is considering a Synology NAS with 4 cores at 2.4GHz and 16GB memory which supposedly maxes out the 10GbE link.
We know that SMB MC can use up to 4 channels per NIC. Questions:
 
1. What is the max we can get out of 1 channel?
2. How does throughput scale with CPU speed. Will a  single 4GHz 8th Gen 8100 core be better than a XEON 2GHz server core? It would be useful to know that, for example, a Ryzen 2200G with 4 cores at 3.5GHz is a capable of driving a 10GbE link, or that a Ryzen 1700 is capable of driving 2 x 10GbE links when dedicated to a file server.
3. How much difference does hyperthreading in a CPU make, if any? 
4. The Cinevate document suggests 32GB RAM, whereas the Synology only uses 16GB. What is the memory consumption of the link in operation? Would 8GB be enough for a dedicated file server machine?
5. No doubt there are other small tweaks to be found.
 
We'll leave RDMA NIC's, RefS and the new Windows for Workstations license for another time. 
 
I look forward to reading your test results!

Share this post


Link to post
Share on other sites
Poppapete

It's Saturday morning here in Australia and I have some time.  Removed 10G nic from server and installed in a W1O box on my network.  Loaded the 10G drivers. Took all of 20 minutes. Transferred a 22GB movie from one computer to the other while still on 1G nics. 104MB/s speed.  Removed 1G connections on 1G switch and connected 10G's to 10 G switch. Both joined the network immediately with 10G LED's.

Transferred 22GB movie 340MB/s.

 

I am pretty sure that either 1.  The Asus drivers does not work correctly in the 2012 server or 2. The server has a DHCP problem of some kind even though I have DNCP disabled.

Share this post


Link to post
Share on other sites
itGeeks

I had a hunch that it may turn out Microsoft servers may not be supported, I was just waiting for your test results. The reason I say this is because unlike the Intel X540/X550 Asus does not list any drivers for the servers only desktop SKU's.. I found an Intel X550t1 on ebay 170.00, It says new and 30 day money back. Found here if you want to give it a try.

Share this post


Link to post
Share on other sites
Poppapete

 

itGeeks,

 

OK but read this:

https://www.virtualizationhowto.com/2018/03/asus-xg-c100c-10g-nic-windows-server-2016-vlans-jumbo-frames/

 

This guy used server 2016 and it just worked. He used the supplied CD for drivers and it shows V 1.40.42.0

I don't think a own a computer with any cd/dvd hardware so I got my drivers from asus and the version is 1.40.37.0 which I DL from asus this morning and seems to be older and dated 11/04/2017

 

I am now going to put the card back into the 2012server and load the drivers from Aquantia which are dated February 2018.

 

They also have Linux drivers and there is few forum threads about guys trying this NIC on linux server and having problems.

 

I bet if I buy a second hand Intel 10G nic and install in my server it will work BUT I have bought the ASUS one!

 

Share this post


Link to post
Share on other sites
itGeeks

@Poppapete

That was an interesting read and further confirms my hunch that its a "Desktop" NIC as Brandon Lee stats in his article working on Windows Server 2016 does not mean it will work on Windows Server 2012. I would not get rid of the box just yet and I hope you purchased it from a seller such as Amazon so you can return it if it fails for you. I wish you the best of luck and hope it works but the fact remains its designed for "desktop" SKU's so its 50/50 chance. Please keep us posted, There is always the Intel NIC I pointed out if you really want to go 10G in a server.

Edited by itGeeks

Share this post


Link to post
Share on other sites
Poppapete

For anyone following my 10G home saga I will relate what transpired.

Having concluded that the Asus 10G card drivers are having problems with Server2012 I decided to retire the server, a supermicro board that has served me well for over 4 years. I have just built a new desktop (X299) and will now repurpose my old X99 W10 machine as the server.  It is more powerful in all respects and has more SATA ports.

Disadvantages:

It will use more power
It cannot run the MS sever backups, but I have found it easier to use Veeam and or Cloudberry for Client backups.
It can’t be a domain server but I skipped that in my instances of Server2012R2E.

Advantages:

10G card will work to give me 10G on my network
I will not have to pay for another OS
I can move Cloudberry, DrivePool and Scanner from Windows Server to W10 “server” at no cost.
I can use ASUS FanXpert on this machine to control those noisy fans

So:

I did a W10 reset on the X99 with the 10G card in place and it just worked on the network at 10G
Gave it a fixed IP on Router
Loaded Plex as Media Server
Loaded Cloudberry to backup the OS (Bare Metal) to both the Cloud and a local D: drive
Loaded StableBit DrivePool and Scanner
Setup Remote Desktop as the new “server” will run headless.

Cannot believe how easy it is to move DrivePool.  I just moved all 8 pooled drives from the old server to the new and when I booted the whole pool was there even with the same drive letters.  I will buy new licenses for the new server as Covecube deserve the support.

  • Like 1

Share this post


Link to post
Share on other sites
JackoUK

Enjoyed following ... gets me off my backside to do some work ... 

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...