TCP Windowing Issue

cisco_troopercisco_trooper Too manyPosts: 1,443Member ■■■■□□□□□□
OK guys I've been fighting an issue for a while now that I can't seem to resolve.

Scenario 1:
I download a file from a CIFS share on a SAN and get blazingly fast speed. Wireshark reveals proper TCP Windowing negotiations and everything works as expected.

Scenario 2:
I upload a file to the same CIFS share on a SAN and get the worst F'ing throughput I've ever seen. TCP Windows aren't negotiating to what I would consider normal and get about 50K/s throughput.

I have tried tuning the TCP parameters on both Windows XP and Windows 7 machines to see if I can get the one way throughput issues resolved but I have had ZERO luck and my brain is about to bleed.

It should be noted that I can reproduce this via LAN access as well as WAN access over IPSec VPN.

Any thoughts anyone? Has anyone else run into this problem?

Comments

  • ptilsenptilsen ■■■■■■■■■■ Posts: 2,835Member ■■■■■■■■■■
    Just to clarify, have you verified the problem is specific to CIFS? I don't see in your list of steps that you've downloaded a file directly to local storage on the server hosting the CIFS share.

    Is the SAN storage on a physical machine running Windows, a virtual machine running Windows, or something else? What version of Windows? What SAN protocol? What hardware?

    Edit: Also, any noteworthy software on server or client OS? I've seen AVG Link Scanner, for example, cause some strange captures in Wireshark for CIFS and HTTP traffic.
    Working B.S., Computer Science
    Complete: 55/120 credits SPAN 201, LIT 100, ETHS 200, AP Lang, MATH 120, WRIT 231, ICS 140, MATH 215, ECON 202, ECON 201, ICS 141, MATH 210, LING 111, ICS 240
    In progress: CLEP US GOV,
    Next up: MATH 211, ECON 352, ICS 340
  • cisco_troopercisco_trooper Too many Posts: 1,443Member ■■■■□□□□□□
    The SAN storage in question is CIFS advertised by and EMC NS-120. I am going to try shares off Windows File Servers as well as trying something else like FTP, but I'll have to setup an FTP server or something specifically for that. I am accessing the shares from Windows XP and Windows 7, but the initial report of the issue come in from someone using a MacBook Pro, which I don't have available for testing.

    I have Kaspersky AV on my workstations and servers, but I won't even get into that whole fiasco at the moment as that is outside my control.

    I'll keep this thread updated with details as they become available.

    Thanks.
  • vinbuckvinbuck Posts: 785Member
    I would probably be getting iperf out and setting the TCP settings to match the application. Then you can prove the network is or isn't the problem and either fix it or move on to the OS. Might want to start checking layer 2 and 3 mtu settings along the path also...
    Cisco was my first networking love, but my "other" router is a Mikrotik...
  • SteveO86SteveO86 Posts: 1,423Member
    vinbuck wrote: »
    I would probably be getting iperf out and setting the TCP settings to match the application. Then you can prove the network is or isn't the problem and either fix it or move on to the OS. Might want to start checking layer 2 and 3 mtu settings along the path also...

    Yep that is where I would.

    Any retransmission or or errors in the Packet capture?
    My Networking blog
    Latest blog post: Let's review EIGRP Named Mode
    Currently Studying: CCNP: Wireless - IUWMS
  • cisco_troopercisco_trooper Too many Posts: 1,443Member ■■■■□□□□□□
    Sorry for the delay in update. I was able to get a packet capture going at the remote end of the IPSec Tunnel and prove that packets were being dropped. There were a ton of retransmission originating from that end that were never received on my local end resulting in drops in the TCP Window as well as bandwidth wasted on multiple copies of the same segment. Always do your sanity checks folks before you start getting into the workings of the protocol negotiations. :)

    Thanks all.
  • SteveO86SteveO86 Posts: 1,423Member
    It's easy to diagnose the problem by looking at the packets, the challenging part is finding why the packets are behaving the way they way there.


    (Sadly I've had 3 incidents in the last months forcing me to go through TCPDump output diagnosing some pretty stupid issues so this is fresh in my mind... of course the fun part is when people argue with your findings and the packet capture)
    My Networking blog
    Latest blog post: Let's review EIGRP Named Mode
    Currently Studying: CCNP: Wireless - IUWMS
Sign In or Register to comment.