QoS is killing me.

cisco_troopercisco_trooper Member Posts: 1,441 ■■■■□□□□□□
in CCNP Service Provider
Ladies and Gentleman, can you give your attention to me:

Getting ready to go smooth off here. I'm trying to implement and End-to-End QoS solution beginning with classifying voice bearer and signalling traffic from both hard and soft phones. The voice traffic is originating from about 40 Cisco 2950-SI access switches. The Cisco 2950s are uplinked to Cisco 6513s. I'm using OmniPeek to capture packets off a couple of the VLANs to inspect the traffic and make sure it is getting marked and it looks like a lot of it is getting marked, but a lot of it isn't. Inspecting the packets that aren't getting marked reveals nothing useful. They all match the conditions I have specified in my traffic classification and marking ACLs and Policy Maps. I'm at a loss as to why some of this traffic is not getting marked, unless a 6513 just can't keep up with it (yeah right, CPU is barely over 20% on this thing).

Any input you all can provide would be appreciated. Thanks.
· Share on FacebookShare on Twitter

Comments

  • xwesleyxwillisxxwesleyxwillisx Member Posts: 158
    My first question would be what end devices are you using? Are they Cisco phones? In that case, they should automatically be setting their L3/L2 markings to 5 (ef).

    Have you noticed if the unmarked traffic is limited to the soft clients? Are these IPCommunicator clients? I don't have much experience with it but I would think it would automatically be marked as well.

    What configuration do you have for the voice ports and trunks to your core? They should be set to trust the markings and not overwrite them.

    These were my initial thoughts...
    · Share on FacebookShare on Twitter
  • cisco_troopercisco_trooper Member Posts: 1,441 ■■■■□□□□□□
    Might as well throw this inconsistent output in here:

    show policy-map interface

    GigabitEthernet1/8
    Service-policy input: LAN_QOS_INGRESS
    class-map: VOICE-RTP (match-all)
    Match: access-group name QOS_VOICE_RTP
    set dscp 46:
    class-map: VOICE-SIGNALLING (match-all)
    Match: access-group name QOS_VOICE_SIGNALLING
    set dscp 26:
    class-map: class-default (match-any)
    Match: any
    set dscp 0:

    Gig1/8 has no Earl, though there is plenty of traffic.

    GigabitEthernet1/10
    Service-policy input: LAN_QOS_INGRESS
    class-map: VOICE-RTP (match-all)
    Match: access-group name QOS_VOICE_RTP
    set dscp 46:
    Earl in slot 7 :
    31747164 bytes
    5 minute offered rate 201304 bps
    aggregate-forwarded 31747164 bytes
    class-map: VOICE-SIGNALLING (match-all)
    Match: access-group name QOS_VOICE_SIGNALLING
    set dscp 26:
    Earl in slot 7 :
    20476 bytes
    5 minute offered rate 112 bps
    aggregate-forwarded 20476 bytes
    class-map: class-default (match-any)
    Match: any
    set dscp 0:
    Earl in slot 7 :
    14302757 bytes
    5 minute offered rate 97800 bps
    aggregate-forwarded 14302757 bytes

    show int gi1/8
    GigabitEthernet1/8 is up, line protocol is up (connected)
    Hardware is C6k 1000Mb 802.3, address is 0018.19bd.e687 (bia 0018.19bd.e687)
    Description: L1N-1X06Z:Gi0/1:172.16.1.12
    MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
    reliability 255/255, txload 1/255, rxload 1/255
    Encapsulation ARPA, loopback not set
    Keepalive set (10 sec)
    Full-duplex, 1000Mb/s
    input flow-control is off, output flow-control is off
    Clock mode is auto
    ARP type: ARPA, ARP Timeout 04:00:00
    Last input 00:00:25, output 00:00:47, output hang never
    Last clearing of "show interface" counters 01:43:49
    Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
    Queueing strategy: fifo
    Output queue: 0/40 (size/max)
    5 minute input rate 180000 bits/sec, 108 packets/sec
    5 minute output rate 244000 bits/sec, 147 packets/sec
    817508 packets input, 172516805 bytes, 0 no buffer
    Received 1467 broadcasts (1352 multicasts)
    0 runts, 0 giants, 0 throttles
    0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
    0 watchdog, 0 multicast, 0 pause input
    0 input packets with dribble condition detected
    1091779 packets output, 231924909 bytes, 0 underruns
    0 output errors, 0 collisions, 0 interface resets
    0 babbles, 0 late collision, 0 deferred
    0 lost carrier, 0 no carrier, 0 PAUSE output
    0 output buffer failures, 0 output buffers swapped out

    show int gi1/10
    GigabitEthernet1/10 is up, line protocol is up (connected)
    Hardware is C6k 1000Mb 802.3, address is 0018.19bd.e689 (bia 0018.19bd.e689)
    Description: L1N-6X072:Gi0/1:172.16.1.16
    MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
    reliability 255/255, txload 1/255, rxload 1/255
    Encapsulation ARPA, loopback not set
    Keepalive set (10 sec)
    Full-duplex, 1000Mb/s
    input flow-control is off, output flow-control is off
    Clock mode is auto
    ARP type: ARPA, ARP Timeout 04:00:00
    Last input 00:00:57, output 00:00:24, output hang never
    Last clearing of "show interface" counters 01:44:27
    Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
    Queueing strategy: fifo
    Output queue: 0/40 (size/max)
    5 minute input rate 288000 bits/sec, 175 packets/sec
    5 minute output rate 337000 bits/sec, 209 packets/sec
    2267257 packets input, 494454247 bytes, 0 no buffer
    Received 1824 broadcasts (1360 multicasts)
    0 runts, 0 giants, 0 throttles
    0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
    0 watchdog, 0 multicast, 0 pause input
    0 input packets with dribble condition detected
    2460044 packets output, 526336813 bytes, 0 underruns
    0 output errors, 0 collisions, 0 interface resets
    0 babbles, 0 late collision, 0 deferred
    0 lost carrier, 0 no carrier, 0 PAUSE output
    0 output buffer failures, 0 output buffers swapped out

    show tcam interface gi1/8 qos type1 ip detail
    +-+
    +
    +
    +
    +
    +
    +---+----+-+---+--+---+---+
    |T|Index| Dest Ip Addr | Source Ip Addr| DPort | SPort | TCP-F |Pro|MRFM|X|TOS|TN|COD|F-P|
    +-+
    +
    +
    +
    +
    +
    +---+----+-+---+--+---+---+
    V 36257 0.0.0.0 0.0.0.0 P=0 P=0
    0 ---- 0 0 -- --- 0-0 <-
    M 36260 0.0.0.0 0.0.0.0 0 0 0 ---- 0 0 <-
    R rslt: 3 <-
    V 36828 0.0.0.0 0.0.0.0 P=0 P=0
    0 ---- 0 0 -- --- 0-0 <-
    M 36836 0.0.0.0 0.0.0.0 0 0 0 X--- 0 0 <-
    R rslt: 0 <-
    V 36829 0.0.0.0 0.0.0.0 P=0 P=0
    0 M--- 0 0 -- --- 0-0
    M 36836 0.0.0.0 0.0.0.0 0 0 0 X--- 0 0
    R rslt: 3

    show tcam interface gi1/10 qos type1 ip detail
    +-+
    +
    +
    +
    +
    +
    +---+----+-+---+--+---+---+
    |T|Index| Dest Ip Addr | Source Ip Addr| DPort | SPort | TCP-F |Pro|MRFM|X|TOS|TN|COD|F-P|
    +-+
    +
    +
    +
    +
    +
    +---+----+-+---+--+---+---+
    V 36182 10.100.6.0 10.255.152.0 10000<P<20000 P=0
    17 ---- 0 0 -- --- 0-0 <-
    M 36188 255.255.254.0 255.255.254.0 0 255 X-X- 0 0 <-
    R rslt: B80F00 <-
    V 36191 10.255.152.0 10.100.6.0 P=0 10000<P<20000
    17 ---- 0 0 -- --- 0-0 <-
    M 36197 255.255.254.0 255.255.254.0 0 255 X-X- 0 0 <-
    R rslt: B80F00 <-
    V 36202 10.100.6.0 10.255.152.0 P=5060 P=0
    17 ---- 0 0 -- --- 0-0 <-
    M 36206 255.255.254.0 255.255.254.0 65535 0 255 X-X- 0 0 <-
    R rslt: 681000 <-
    V 36203 10.255.152.0 10.100.6.0 P=5060 P=0
    17 ---- 0 0 -- --- 0-0
    M 36206 255.255.254.0 255.255.254.0 65535 0 255 X-X- 0 0
    R rslt: 681000
    V 36227 0.0.0.0 0.0.0.0 P=0 P=0
    0 ---- 0 0 -- --- 0-0 <-
    M 36233 0.0.0.0 0.0.0.0 0 0 0 X--- 0 0 <-
    R rslt: 1100 <-
    V 36301 0.0.0.0 0.0.0.0 P=0 P=0
    0 ---- 0 0 -- --- 0-0
    M 36305 0.0.0.0 0.0.0.0 0 0 0 ---- 0 0
    R rslt: 1100
    V 36828 0.0.0.0 0.0.0.0 P=0 P=0
    0 ---- 0 0 -- --- 0-0 <-
    M 36836 0.0.0.0 0.0.0.0 0 0 0 X--- 0 0 <-
    R rslt: 0 <-
    V 36829 0.0.0.0 0.0.0.0 P=0 P=0
    0 M--- 0 0 -- --- 0-0
    M 36836 0.0.0.0 0.0.0.0 0 0 0 X--- 0 0
    R rslt: 3


    THESE INTERFACE ARE CONFIGURED EXACTLY THE SAME, YET TCAM DOES NOT APPEAR TO BUILD FOR GI1/8. BOTH INTERFACES HAVE PLENTY OF TRAFFIC GOING THROUGH THEM. TCAM RESOURCES ARE PLENTIFUL.

    show tcam counts
    Used Free Percent Used Reserved
    ---- ----
    Labelsicon_sad.gifin) 26 4070 0
    Labelsicon_sad.gifeg) 4 4092 0
    ACL_TCAM
    Masks: 21 4075 0 72
    Entries: 102 32666 0 576
    QOS_TCAM
    Masks: 39 4057 0 18
    Entries: 261 32507 0 144
    LOU: 4 124 3
    ANDOR: 0 16 0
    ORAND: 0 16 0
    ADJ: 4 2044 0

    ANYWAY, SLIGHTLY OFF TOPIC, BUT BOTHERSOME NONETHELESS. ALL OTHER INTERFACES APPEAR AS GI1/10 DOES, AS EXPECTED.
    · Share on FacebookShare on Twitter
  • cisco_troopercisco_trooper Member Posts: 1,441 ■■■■□□□□□□
    xwesleyxwillisx wrote: »
    My first question would be what end devices are you using? Are they Cisco phones? In that case, they should automatically be setting their L3/L2 markings to 5 (ef).
    The phones attached to the 2950s are marking COS 5, DSCP 46.
    xwesleyxwillisx wrote: »
    Have you noticed if the unmarked traffic is limited to the soft clients? Are these IPCommunicator clients? I don't have much experience with it but I would think it would automatically be marked as well.
    I haven't been able to confirm or deny this as a possibility so far. BUT, since the trust on the 6513 is not working as expected, I don't expect to be able to mark the traffic at the softphone and have it succeed. This was what I was going to do originally.
    xwesleyxwillisx wrote: »
    What configuration do you have for the voice ports and trunks to your core? They should be set to trust the markings and not overwrite them.
    The 6513 trunks WERE set to trust DSCP, but sniffing packets revealed that all values were 0. This was excessively bothersome, so I tried trust COS, with the same results. So now, rather than trusting the markings from the uplink, I am classifying on ingress with Policy-Maps using ACLs. This appears to be working other than some of the traffic that matches the ACLs for some reason doesn't get marked, (or perhaps its slipping past classification entirely).
    · Share on FacebookShare on Twitter
  • cisco_troopercisco_trooper Member Posts: 1,441 ■■■■□□□□□□
    OK. Back to tracing back unmarked QoS traffic. I've been sniffing and tracing packets back to their sources most of the day. The packets that aren't being marked are packets that are coming in on interfaces where QoS TCAM is not built (w t f with that?!!?). Has anyone experienced situations like this before. I'm beginning think I'm going to need a TAC call, but I really want to avoid that if possible. These are interfaces that appear the same as Gi1/8 above.
    · Share on FacebookShare on Twitter
  • ilcram19-2ilcram19-2 Banned Posts: 436
    do this on your switch for each port that you have the phone hooked up to
    interface renge x-x
    mls trust cos
    switchport voice vlan x
    switchport priority extended trust

    you shoudl be all set, the packets shoudl be mark all the way to the phone with the right cos value and any other packet (in case you have a pc hooked up to the phone) should have a diferrent value
    · Share on FacebookShare on Twitter
  • cisco_troopercisco_trooper Member Posts: 1,441 ■■■■□□□□□□
    Just an FYI. This blade actually required a reset not too long ago because it was not even forwarding traffic. Since then I have confirmed with Cisco TAC that the blade is bad due to bad TCAM.
    · Share on FacebookShare on Twitter
  • networker050184networker050184 Mod Posts: 11,962 Mod
    cisco_trooper wrote: »
    Just an FYI. This blade actually required a reset not too long ago because it was not even forwarding traffic. Since then I have confirmed with Cisco TAC that the blade is bad due to bad TCAM.

    Glad you got it sorted out. Its always nice to find out it wasn't something you did wrong.
    An expert is a man who has made all the mistakes which can be made.
    · Share on FacebookShare on Twitter
Sign In or Register to comment.