path-mtu-discovery
How path-mtu-discovery is supposed to work.
Local device checks it's window size i.e. how big its local buffer is, checks its outgoing interface mtu, and whichever is smaller is
chosen as MSS, lets say window size is 16k on both sides, standard 1500 mtu is used, MSS will use 1500 - (ip + tcp header) = 1460 byes.Not taking any optional fields into account in headers i.e. standard 20 bytes each.
All outgoing packets will get marked with the DF =1 i.e. don't fragment. Let's say a core device has an MTU of 800, if we try and send a packet greater than 800 it will be dropped, and an icmp fragmentation needed is received at the originator, the pmtud protocol will lower the MSS and try again, eventually working out an optimal MTU size of 760bytes.
Below are the results i've seen. BGP has path-mtu-discovery enabled by default, as per below, it didn't work until i enabled
"ip tcp path-mtu-discovery" globally on both peer routers.Strange results were seen depending on combination used.
I'm not sure on the implications on other tcp protocols originating or terminating on the router, just wondering has anyone else seen this? If i'm asked to verify pmtud is functioning on the lab, is this the right path? or is there something im missing?
1)
path-mtu-discovery is enabled on bgp peers(default)
path-mtu-discovery disabled on both neighbors(default)
SYN
IP flag 0x02 "Dont fragement"
MSS 536
SYN ACK
IP flag 0x02 "Dont fragement"
MSS 536
ACK,BGP OPEN, BGP Keepalives all "Dont Fragment"
2)
path-mtu-discovery is disabled on bgp peers
path-mtu-discovery disabled on both neighbors(default)
SYN,SYN ACK
IP flag 0x00 "fragment"
MSS 536
ACK,BGP OPEN, BGP Keepalives all "Fragment"
3)
path-mtu-discovery is disabled on bgp peers
path-mtu-discovery enabled on tcp on both neighbors
SYN
IP flag 0x00 "fragment"
MSS 536
SYN ACK
IP flag 0x02 "Dont fragement"
MSS 536
ACK,BGP OPEN, BGP Keepalives all "Fragment"
4)
path-mtu-discovery is enabled on bgp peers(default)
path-mtu-discovery enabled on tcp on both neighbors
SYN,SYN ACK
IP flag 0x02 "Dont fragement"
MSS 1460
ACK,BGP OPEN, BGP Keepalives all "Dont Fragment"
Local device checks it's window size i.e. how big its local buffer is, checks its outgoing interface mtu, and whichever is smaller is
chosen as MSS, lets say window size is 16k on both sides, standard 1500 mtu is used, MSS will use 1500 - (ip + tcp header) = 1460 byes.Not taking any optional fields into account in headers i.e. standard 20 bytes each.
All outgoing packets will get marked with the DF =1 i.e. don't fragment. Let's say a core device has an MTU of 800, if we try and send a packet greater than 800 it will be dropped, and an icmp fragmentation needed is received at the originator, the pmtud protocol will lower the MSS and try again, eventually working out an optimal MTU size of 760bytes.
Below are the results i've seen. BGP has path-mtu-discovery enabled by default, as per below, it didn't work until i enabled
"ip tcp path-mtu-discovery" globally on both peer routers.Strange results were seen depending on combination used.
I'm not sure on the implications on other tcp protocols originating or terminating on the router, just wondering has anyone else seen this? If i'm asked to verify pmtud is functioning on the lab, is this the right path? or is there something im missing?
1)
path-mtu-discovery is enabled on bgp peers(default)
path-mtu-discovery disabled on both neighbors(default)
SYN
IP flag 0x02 "Dont fragement"
MSS 536
SYN ACK
IP flag 0x02 "Dont fragement"
MSS 536
ACK,BGP OPEN, BGP Keepalives all "Dont Fragment"
2)
path-mtu-discovery is disabled on bgp peers
path-mtu-discovery disabled on both neighbors(default)
SYN,SYN ACK
IP flag 0x00 "fragment"
MSS 536
ACK,BGP OPEN, BGP Keepalives all "Fragment"
3)
path-mtu-discovery is disabled on bgp peers
path-mtu-discovery enabled on tcp on both neighbors
SYN
IP flag 0x00 "fragment"
MSS 536
SYN ACK
IP flag 0x02 "Dont fragement"
MSS 536
ACK,BGP OPEN, BGP Keepalives all "Fragment"
4)
path-mtu-discovery is enabled on bgp peers(default)
path-mtu-discovery enabled on tcp on both neighbors
SYN,SYN ACK
IP flag 0x02 "Dont fragement"
MSS 1460
ACK,BGP OPEN, BGP Keepalives all "Dont Fragment"
Networking, sometimes i love it, mostly i hate it.Its all about the $$$$
Comments
To verify the path mtu for a given session, "show ip bgp neighbor" will list the max data segment. If you're in the lab and you've configured pmtud and cleared your sessions, but still see a low mtu.. check for ICMP filtering somewhere in the network.
neighbor x.x.x.x transport path-mtu-disable, so its enabled implicitly.
Every device should support a max mtu of 576, when you minus the overhead 40, you get the 536 for the max segment which should be supported by TCP.Cisco defaults to use 536.
The problem i've seen is BGP is choosing a max segment size of 536 even with PMTUD implicitly enabled n bgp peers.The PMTUD never kicks in, it's supposed to choose the MTU of the outgoing interface minus overhead.It only works as expected when "ip tcp path-mtu-discovery" is configured globally.Just wondering if this is what everyone else has seen? but maybe most have just seen the capability in the neighbor and took for grated that it works.
I'm peering between 2 loopbacks, no filtering and i'm capturing all sent and received packets.
It makes sense... since this way, you just have to issue a single command to get pmtud running on all neighbor sessions. Yet, you can still disable on a per neighbor basis.
Thanks..