Jump to content
RESET Forums (homeservershow.com)
Sign in to follow this  
AlexF

"load balance" outgoing traffic from Windows client with multiple NICs

Recommended Posts

AlexF

For those that are interested....

 

I installed TP-LINK TG-3468 (PCI-E) into my PC, Windows 7; Intel EXPI9301CTBLK (PCI-E) into my N36L, FreeNAS 8.3.0. Now, I have dual connections:

1. PC-built-in NIC (Windows's System Info says) RTL8167 to N36L's built-in NIC, via L2 switch

2. PC TG-3468 NIC (Windows's System Info says) RTL8167 to N36L's EXPI9301CTBLK NIC, directly (cable).

 

On N36L:

[Alex@freenas /etc]$ ifconfig -av
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
        ether 68:05:ca:19:57:9c
        inet 192.168.2.95 netmask 0xffffff00 broadcast 192.168.2.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
        ether b4:99:ba:bf:4a:78
        inet 192.168.1.95 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex,flowcontrol,rxpause,txpause>)
        status: active

 

I rebooted both PC and N36L so as to prevent file-caching affecting result, then did robocopy tests:

 

C:\Windows\system32>robocopy /v "\\192.168.1.95\Media\Movies\DVD\XXX\VIDEO_TS" c:\temp VTS_01_1.VOB

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------

  Started : Sun Jul 14 10:37:52 2013

   Source : \\192.168.1.95\Media\Movies\DVD\XXX\VIDEO_TS\
     Dest : c:\temp\

    Files : VTS_01_1.VOB

  Options : /V /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

                           1    \\192.168.1.95\Media\Movies\DVD\XXX\VIDEO_TS\
100%        New File            1023.9 m        VTS_01_1.VOB

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         1         0         0         0
   Files :         1         1         0         0         0         0
   Bytes : 1023.99 m 1023.99 m         0         0         0         0
   Times :   0:00:13   0:00:13                       0:00:00   0:00:00


   Speed :            80123854 Bytes/sec.
   Speed :            4584.723 MegaBytes/min.

   Ended : Sun Jul 14 10:38:05 2013

C:\Windows\system32>robocopy /v "\\192.168.2.95\Media\Movies\DVD\XXX\VIDEO_TS" c:\temp VTS_01_2.VOB

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------

  Started : Sun Jul 14 10:38:10 2013

   Source : \\192.168.2.95\Media\Movies\DVD\XXX\VIDEO_TS\
     Dest : c:\temp\

    Files : VTS_01_2.VOB

  Options : /V /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

                           1    \\192.168.2.95\Media\Movies\DVD\XXX\VIDEO_TS\
100%        New File            1023.9 m        VTS_01_2.VOB

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         1         0         0         0
   Files :         1         1         0         0         0         0
   Bytes : 1023.99 m 1023.99 m         0         0         0         0
   Times :   0:00:24   0:00:24                       0:00:00   0:00:00


   Speed :            43099577 Bytes/sec.
   Speed :            2466.177 MegaBytes/min.

   Ended : Sun Jul 14 10:38:35 2013

 

As you can see direct connection is significantly slower. Why? I don't know.

Share this post


Link to post
Share on other sites
AlexF

As you can see direct connection is significantly slower. Why? I don't know.

 

The logical conclusion is either:

1. one of the NICs is deficient

2. Window SMB client is deficient

3. FreeNAS SMB server is deficient.

 

To rule out 1, I performed iPerf:

 

d:\temp\jPerf>bin\iperf.exe -c 192.168.2.95 -P 10 -i 10 -p 5001 -f k -t 60
:
[SUM]  0.0-60.0 sec  6931456 KBytes  945673 Kbits/sec

d:\temp\jPerf>bin\iperf.exe -m -c 192.168.1.95 -P 10 -i 10 -p 5001 -f k -t 60
:
[SUM]  0.0-60.1 sec  6930832 KBytes  945343 Kbits/sec

 

As can be seen, performance is approximately identical. So, I guess it must be the other two.

 

[bTW, I did try Jumbo frame - configuring 9000 byte frame size on both PCI-E NICs, with and wihtout setting HKLM\System\CurrentControlSet\Services\LanmanWorkstation\Parameters with key DisableLargeMtu. Iperf showed ~5% increase (with iPerf's -M 3.0K) but SMB was slower.]

Share this post


Link to post
Share on other sites
AlexF

I re-did the robocopy tests, this time, using target file but specifying different target IP. I know that if I specify same target, then either Windows or FreeNAS's cache kicks-in and copy would be extra-ordinarily fast, but it looks like if the target path is different then it defaults back to a dumb copying.

 

 

d:\temp\jPerf>robocopy /v "\\192.168.2.95\Media\Movies\BluRay\YYY\BDMV\STREAM" c:\temp 00800.m2ts

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------

  Started : Sun Jul 14 17:52:25 2013

   Source : \\192.168.2.95\Media\Movies\BluRay\YYY\BDMV\STREAM\
     Dest : c:\temp\

    Files : 00800.m2ts

  Options : /V /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

                           1    \\192.168.2.95\Media\Movies\BluRay\YYY\BDMV\STREAM\
100%        New File              34.0 g        00800.m2ts

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         1         0         0         0
   Files :         1         1         0         0         0         0
   Bytes :  34.032 g  34.032 g         0         0         0         0
   Times :   0:06:58   0:06:58                       0:00:00   0:00:00


   Speed :            87235762 Bytes/sec.
   Speed :            4991.670 MegaBytes/min.

   Ended : Sun Jul 14 17:59:24 2013

d:\temp\jPerf>robocopy /v "\\192.168.1.95\Media\Movies\BluRay\YYY\BDMV\STREAM" c:\temp 00800.m2ts

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------

  Started : Sun Jul 14 18:00:13 2013

   Source : \\192.168.1.95\Media\Movies\BluRay\YYY\BDMV\STREAM\
     Dest : c:\temp\

    Files : 00800.m2ts

  Options : /V /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

                           1    \\192.168.1.95\Media\Movies\BluRay\YYY\BDMV\STREAM\
100%        New File              34.0 g        00800.m2ts

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         1         0         0         0
   Files :         1         1         0         0         0         0
   Bytes :  34.032 g  34.032 g         0         0         0         0
   Times :   0:06:56   0:06:56                       0:00:00   0:00:00


   Speed :            87758469 Bytes/sec.
   Speed :            5021.579 MegaBytes/min.

   Ended : Sun Jul 14 18:07:09 2013 

 

As you see, results are comparable.

 

I then did simultaneous robocopy of 1 GB (VOB) file using two shells:

Test 1: simultaneous robocopy of VTS_02_1.VOB and VTS_02_2.VOB - total bandwidth 99.18MB/s

Test 2: robocopy of VTS_02_3.VOB  - bandwidth 83.09MB/s

Test 3: simultaneous robocopy of VTS_02_4.VOB and VTS_02_5.VOB - total bandwidth 114.00MB/s

 

So, I got 19% and 37% input in throughput.

Edited by AlexF

Share this post


Link to post
Share on other sites
AlexF

> So, I got 19% and 37% input in throughput.

 

Meant, "So, I got 19% and 37% improvement in throughput."

Share this post


Link to post
Share on other sites
ikon

Yeah, kinda figured that :)

Share this post


Link to post
Share on other sites
AlexF

Yeah, kinda figured that :)

 

The summary is: as far as reading is concerned, multiple requests using multiple connections (what I called "application-based" load-balancing) gives a signficant throughput inprovement, proving that SMB and networking (using a single 1G NIC) is the limiting factor, not NAS RAID. Maybe this isn't true for writing to RAID because that may be more NAS CPU or disk intensive - I haven't tested this.

 

As example, I can provide one practical scenario which is relevant to me where above would be useful, (since I own two NASs and sync between then using FreeNAS's built-in rsync client/server): I can have NASs doing a sync via NICs on its second subnet, whilst servicing user requests on the first.

Edited by AlexF

Share this post


Link to post
Share on other sites
ikon

At work, we have all the servers, SAN, and offline storage on a second, fibre LAN from the one the client computers are on. Resyncs, backups, balancing, etc. take place in the background without bothering the users. :)

Share this post


Link to post
Share on other sites
timekills

At work, we have all the servers, SAN, and offline storage on a second, fibre LAN from the one the client computers are on. Resyncs, backups, balancing, etc. take place in the background without bothering the users. :)

Typically you'd create one or more management VLANs for background transfers (or for ESXi/Hyper-V for HA migrations of VMs) and others for user access. Then the NIC assigned to each VLAN is forced by layer 3 to only direct traffic accross that network (unless you implment a router somewhere in the loop and allow cross-network traffic.)

 

This is probably beyond the scope of most home users, especially as it requires a managed switch to allow for VLAN creation anyway. Most NAS boxes that come with multiple NICs allow for on-box layer 2 management that just works without any OS or switch side tweaking. It's when you get into the channel bonding discussion that things get tricky. Separating "user" traffic from management traffic is fairly simple, if perhaps overkill for the home.

Share this post


Link to post
Share on other sites
ikon

Definitely overkill for the home. We have VLANs at work, but implemented the server "shadow LAN" before VLANs became popular. The decision was to simply continue with that since it wasn't broken :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  



×
×
  • Create New...