Jump to content
RESET Forums (homeservershow.com)
Poppapete

10G Network

Recommended Posts

schoondoggy

Have you updated the firmware on the 10GbE switch? Did you assign an IP address to the 10GbE switch?
It sounds like your 1GbE connections work as you expect them to, have you tried connecting your 1GbE NIC's to the 10GbE switch to see if that works. It may help narrow down if it is a switch or NIC issue.

Share this post


Link to post
Share on other sites
Poppapete

I have just woken up Sunday Morning here in Australia to find useful suggestions.

 

The switch is unmanaged to I have no access.

First thing I tried was the schoondoggy suggestion of connecting the 1G nics to the 10G router.  All works instantly as it should. So the switch works but not at 10G.

Next I connected the 10G nics to the 1G switch. The DT connects fine to the network but I can't see the server. So I suggest the problem lies with the ASUS XG-C100C. I will reload the drivers. Again thanks to all.

Share this post


Link to post
Share on other sites
JackoUK

Might be worth trying a direct 10GbE to 10GbE connection without switches using a single cable.

 

I was considering using Thunderbolt in a similar fashion: replacing my old 'workstation' with an 8 disk machine and 2 of my ancient HP Microservers with an 8 disk data server. In other words just 2 machines and no switching.

However I don't see TB on Ryzen or Threadripper yet ...

... so you've made me think about just using two 10GbE NIC's instead on an AMD system.

Share this post


Link to post
Share on other sites
Poppapete
12 hours ago, JackoUK said:

Might be worth trying a direct 10GbE to 10GbE connection without switches using a single cable.

 

I was considering using Thunderbolt in a similar fashion: replacing my old 'workstation' with an 8 disk machine and 2 of my ancient HP Microservers with an 8 disk data server. In other words just 2 machines and no switching.

However I don't see TB on Ryzen or Threadripper yet ...

... so you've made me think about just using two 10GbE NIC's instead on an AMD system.

I would still need the 1G nics to connect to the network.  Would the the server and DT use the 10G connection in preference to the 1G.  Not sure how to set that up!

Share this post


Link to post
Share on other sites
JackoUK

"I would still need the 1Gb NIC's to connect to the network."

I think not ... but let's try ...

 

1. Connect the 1G machine ports to the 10G switch and leave the 10G switch connected to the router. The 10G switch will be autosensing and should  connect at 1G to the router and obtain IP addresses via DHCP from the router automatically without any configuration (presumably that's what you had before with only the router).

Log in to the 2 machines and check they connect to the internet.

If that  works then the 10G switch is OK.

 

2. Next connect up the 10G ports leaving the 1G out and the switch - router cable in. When you power up the server and the DT they should each talk through the switch to the router and obtain an IP address.

Log in to the server and see if you can connect  to the Internet. If not then that 10G NIC is faulty/incorrectly configured.

Similarly log in to the DT and check for internet connectivity.

If either of the machines gets through then it means its 10G NIC is OK.

 

Edited by JackoUK

Share this post


Link to post
Share on other sites
Poppapete

1.If I connect the 1G ports to the 10G switch with 10G switch connected to the router:
Then I have internet on both machines, access to the files on the server from the DT.  So 10G switch is OK

2.If I connect only the 10G ports to the 10G switch:
The server immediately has no internet, but the DT is connected to internet.
All connections on both 10G switch and the ASUS Nic card in server show green flashing LED’s.  I have latest drivers on both 10G nics and both show working with best drivers in device manager.

Results of an ipconfig /all command on both machines:
Server:
Media State                 Media Disconnected

Description                  ASUS XG-C100C 10G PCI-E
Address                       88-D7 etc
DHCP enabled            Yes
Autoconfiguration      Yes

Desktop:
Connection-specific DNS Suffix  . :
Description . . . . . . . . . . . :     Aquantia 10G Ethernet connection
Physical Address. . . . . . . . . : B0-6E-BF-82-9A-13
DHCP Enabled. . . .. . . :         Yes
Autoconfiguration Enabled . . Yes
Link-local IPv6 Address . . . . fe80::a938:2622:773b:6464%7(Preferred)
IPv4 Address. . . . . . . . . . . :   192.168.0.5(Preferred)
Subnet Mask . . . . . . . . . .       : 255.255.255.0  
Lease Obtained. . . . . . . . .      . : Tuesday, 27 February 2018 7:48:17 AM
Lease Expires . . . . . . . . . . :    Tuesday, 27 February 2018 9:48:16 AM
Default Gateway . . . . . . . .     . : 192.168.0.1
DHCP Server . . . . . . . . . .      . : 192.168.0.1
DHCPv6 IAID . . . . . . . . . . . : 129003199
DHCPv6 Client DUID. . . . .    : 00-01-00-01-21-88-3F-07-B0-6E-BF-82-9A-14
DNS Servers . . . . . . . . . .       . : 203.0.178.191
                                                 203.215.29.191
NetBIOS over Tcpip. .. .         : Enabled

 

Share this post


Link to post
Share on other sites
ShadowPeo
3 hours ago, Poppapete said:


Server:
Media State                 Media Disconnected

Description                  ASUS XG-C100C 10G PCI-E
Address                       88-D7 etc
DHCP enabled            Yes
Autoconfiguration      Yes

 

 

 

The NIC is disconnected here, either that is your issue,, bad cable or something, or can we please get a connected printout

Share this post


Link to post
Share on other sites
Poppapete

continuing saga.

 

So I go to lunch after shuting DT down.

Come home and reboot and get no internet. ( DT still connected to 10G switch)

ip config shows:

Desktop:
Connection-specific DNS Suffix  . :
Description . . . . . . . . . . . :     Aquantia 10G Ethernet connection
Physical Address. . . . . . . . . : B0-6E-BF-82-9A-13
DHCP Enabled. . . .. . . :         Yes
Autoconfiguration Enabled . . Yes
Link-local IPv6 Address . . . . fe80::a938:2622:773b:6464%7(Preferred)
IPv4 Address. . . . . . . . . . . :   169.254.100.100(Preferred)
Subnet Mask . . . . . . . . . .       : 255.255.0.0  
Default Gateway . . . . . . . .     .

 

TOTALLY DIFFERENT IPv4 ADDRESS    !!!???

 

to post this I had to reconnect the 1G nic on the DT.

 

".can we please get a connected printout" - you mean from the server when it is connected to internet via 1G!!!

 

I will try a different cable on the server 10G before I give up!.

 

EDIT:

I tried a CAT6 cable from server to 10G switch. As before all leds light up green (10G speed) but ip config shows nic as disconnected.

 

Really appreciate the time you have spent with me on this but I give up.


 

 

Edited by Poppapete

Share this post


Link to post
Share on other sites
JackoUK

“Lay on, Macduff, and damned be him who first cries ‘Hold! enough!’”

 

You made a lot of progress in those last tests ...

... in particular it sounds like the most expensive item - the 10GbE switch - is working ...

... that at least 1 of the 10GbE cards is working ...

... but the DHCP process is flaky and indeed may be the single cause of all problems.

 

I don't know much about server software or Windows Essentials but I do know the software can act as a DHCP server.

So next 'test' is to see if Essentials 2012 R2 has the DHCP server role enabled.

You don't want it - you want your router to be the only device dishing out and managing local IP addresses.

 

Note that when the DT connected properly you had a completely standard setup:

- the NIC was using DHCP i.e. did not have a manually specified IP address

- it went looking for the DHCP server at 192.168.0.1 ( a totally standard figure, almost certainly the IP of your router)

- the router issued the IP address 192.168.0.5 (again totally standard and in the same range of 192.168.0) to the DT

- similarly the router 192.168.0.1 is the default gateway and provides lookup for names such as www.google.com via DNS

All good.

 

The next test is to swap the 10GbE cards between DT and server.

If the server now connects but the DT doesn't - then I think you have identified a bad NIC.

 

Next test (which I suggested earlier).

Take the router, switch, DHCP, DNS and all that stuff out of the equation by unplugging all networking cables from the DT and server.

Use only 1 cable between the 10GbE ports on the DT and server: direct cable NIC to NIC.

Set the IP addresses manually, say 192.168.0.10 and 192.168.0.11

Now try to ping from one to the other.

 

I have a last resort up my sleeve as well ...

 

Share this post


Link to post
Share on other sites
JackoUK

My help is not entirely altruistic - I'm planning a similar setup myself. Very nice of you to be guinea pig :)

I'm sure other members of the forum are too.

It would be useful to know if the Aquantia silicon is reliable.

For instance I'm considering a motherboard with an Aquantia 10GbE chip and Thunderbolt header on board.

We'd really like to get to the position where 10GbE is expected and working on board as we take 1GbE for granted now.

 

Not sure if its relevant... but in researching Thunderbolt ... Windows software will issue an automatic IP address of: 

169.254.0.0/16

according to Intel's guide.

Looks similar to your 169.254.100.100.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×