HSRP and %SW_MATM-4-MACFLAP_NOTIF

ZblaJhaNi

Hi,

I am trying to resolve the problem with HSRP.

DST1 is the STP root and HSRP active.The only port that is blocking is the marked one on DST2 (HSRP Standby) fr.
When i am trying to ping the HSRP addressom the PC , i start to receiving the %SW_MATM-4-MACFLAP_NOTIF notifications on all over the place

any suggestions?


BR

DevilWAH

what are your hello timmers on hsrp ? less than 150msec?

ZblaJhaNi

DevilWAH wrote: »

what are your hello timmers on hsrp ? less than 150msec?

Hi,

I changed the timers (hello1, holdtime3), back to the default`s, but it does not help.
Just a hint:
If i establish the link between "distribution" switches everything looks ok.


Thanks for help

DevilWAH

try a show logg

and show standby (on both the core switches running hsrp)

see how often the active and stand by are swapping. and how many times they have.

ZblaJhaNi

DevilWAH wrote: »

try a show logg

and show standby (on both the core switches running hsrp)

see how often the active and stand by are swapping. and how many times they have.

The HSRP seems stable, active& standby router (active is C3550 and standby is C1712 - is maybe problem here.....?!), does not change their roles.

I have two vlans configured:

Vlan 13- SVI on every swtich (the remain to are C2960) for management purposes.
Vlan 33- SVI`s on C3550 and C1712


My PC is attached in access vlan 33....

Trunks seems ok, so do STP.....

As i said, with the active connection between HSRP routers everything seems ok....

I am a liitle bit confused

Right now only the switch where the PC is attached receiving notifications:

04:15:50: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/2
04:15:51: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.c2dd.7e98 in vlan 33 is flapping between port Fa0/2 and port Fa0/13
04:16:19: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/2
04:17:37: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/2
04:18:06: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/2
04:18:35: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/2
04:19:27: %SW_MATM-4-MACFLAP_NOTIF: Host 0014.c2dd.7e98 in vlan 33 is flapping between port Fa0/2 and port Fa0/13
04:19:53: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/2
04:20:22: %SW_MATM-4-MACFLAP_NOTIF: Host 0024.519f.97c1 in vlan 13 is flapping between port Fa0/1 and port Fa0/

qplayed

who owns which mac? trace it out

DevilWAH

qplayed wrote: »

who owns which mac? trace it out

MAC_Find: Vendor/Ethernet/Bluetooth MAC Address Lookup and Search handy litle tool, pas the mac in there and it will tell you the manafacture of the device.

so the 0024.51xx.xxxx is a cisco devices (first 6 are the manafacture code)

find out what switch this is comming from

you can use

#show mac-address table

to see waht port a mac address is on to trace it back.

#show mac address table | in 0014.c2dd.7e98

will filter the results to only that one mac

qplayed

no need to go that far, just find out where ti is being sourced from. It will most likely be your "PC". I came across this same issue doing an HSRP lab also. Its mostly likely a loop if I'm not mistaken. Unfortunately I didn't document my findings I wish I did... sorry I'm not much help...

ZblaJhaNi

Hi,

problem found:

C890_Branch#sh spanning-tree vlan 33

VLAN33 is executing the ieee compatible Spanning Tree protocol
Bridge Identifier has priority 32768, address 0026.0b5b.65a0
Configured hello time 2, max age 20, forward delay 15
!!!!!!!!Current root has priority 24609, address 000c.3164.e680!!!!!!!!
!!!!!!!!command spanning-tree vlan 33 root primary configured on DST1!!!!!!!
Root port is 3 (FastEthernet2), cost of root path is 38
Topology change flag set, detected flag not set
Number of topology changes 5 last change occurred 00:05:59 ago
from FastEthernet1
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0, aging 300

Port 2 (FastEthernet1) of VLAN33 is blocking
Port path cost 19, Port priority 128, Port Identifier 128.2.
Designated root has priority 24609, address 000c.3164.e680
Designated bridge has priority 32801, address 0024.519f.9780
Designated port id is 128.1, designated path cost 19
Timers: message age 15, forward delay 0, hold 0
Number of transitions to forwarding state: 4
BPDU: sent 357, received 67

Port 3 (FastEthernet2) of VLAN33 is forwarding
Port path cost 19, Port priority 128, Port Identifier 128.3.
Designated root has priority 24609, address 000c.3164.e680
Designated bridge has priority 32801, address 001e.1446.3980
Designated port id is 128.2, designated path cost 19
Timers: message age 16, forward delay 0, hold 0
Number of transitions to forwarding state: 1
BPDU: sent 462, received 67

command executed a few seconds later

C890_Branch#sh spanning-tree vlan 33

VLAN33 is executing the ieee compatible Spanning Tree protocol
Bridge Identifier has priority 32768, address 0026.0b5b.65a0
Configured hello time 2, max age 20, forward delay 15
!!!!!!!We are the root of the spanning tree!!!!!!!!!!
Topology change flag set, detected flag set
Number of topology changes 5 last change occurred 00:06:05 ago
from FastEthernet1
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 1, topology change 34, notification 0, aging 300

Port 2 (FastEthernet1) of VLAN33 is listening
Port path cost 19, Port priority 128, Port Identifier 128.2.
Designated root has priority 32768, address 0026.0b5b.65a0
Designated bridge has priority 32768, address 0026.0b5b.65a0
Designated port id is 128.2, designated path cost 0
Timers: message age 0, forward delay 14, hold 0
Number of transitions to forwarding state: 4
BPDU: sent 358, received 67

Port 3 (FastEthernet2) of VLAN33 is forwarding
Port path cost 19, Port priority 128, Port Identifier 128.3.
Designated root has priority 32768, address 0026.0b5b.65a0
Designated bridge has priority 32768, address 0026.0b5b.65a0
Designated port id is 128.3, designated path cost 0
Timers: message age 0, forward delay 0, hold 0
Number of transitions to forwarding state: 1
BPDU: sent 463, received 67

Btw: I replaced C1712 with C890.

Ok, i use debbuging and i found:
Spanning Tree event debugging is on

*Apr 29 12:16:15.283: STP: VLAN33 new root port Fa1, cost 38
*Apr 29 12:16:15.283: STP: VLAN33 Fa1 -> listening
*Apr 29 12:16:15.283: STP: VLAN33 we are the spanning tree root
C890_Branch#
*Apr 29 12:16:30.283: STP: VLAN33 Fa1 -> learning
C890_Branch#
*Apr 29 12:16:43.467: STP: VLAN33 heard root 24609-000c.3164.e680 on Fa1
*Apr 29 12:16:43.467: current Root has 49152-0026.0b5b.65a0
*Apr 29 12:16:43.467: supersedes 49152-0026.0b5b.65a0
*Apr 29 12:16:43.467: STP: VLAN33 new root is 24609, 000c.3164.e680 on port Fa1, cost 38
*Apr 29 12:16:43.467: STP: VLAN33 sent Topology Change Notice on Fa1
*Apr 29 12:16:43.467: STP: VLAN33 new root port Fa2, cost 38
*Apr 29 12:16:43.471: STP: VLAN33 Fa1 -> blocking
C890_Branch#
*Apr 29 12:16:45.467: STP: VLAN33 sent Topology Change Notice on Fa2
C890_Branch#
*Apr 29 12:17:15.283: STP: VLAN33 new root port Fa1, cost 38
*Apr 29 12:17:15.283: STP: VLAN33 Fa1 -> listening
*Apr 29 12:17:15.283: STP: VLAN33 we are the spanning tree root
C890_Branch#
*Apr 29 12:17:30.283: STP: VLAN33 Fa1 -> learning
C890_Branch#
*Apr 29 12:17:45.283: STP: VLAN33 Fa1 -> forwarding

I am also receiving unknown protocol drops on both interfaces:


C890_Branch#sh interfaces fastEthernet 1
FastEthernet1 is up, line protocol is up
Hardware is Fast Ethernet, address is 0026.0b5b.65a0 (bia 0026.0b5b.65a0)
Description: ACC2_Fa0/1
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:18, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/2228/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 1 packets/sec
3167 packets input, 245830 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
7199 packets output, 594370 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
27 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
C890_Branch#sh interfaces fastEthernet 2
FastEthernet2 is up, line protocol is up
Hardware is Fast Ethernet, address is 0026.0b5b.65a1 (bia 0026.0b5b.65a1)
Description: ACC1_Fa0/2
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:23, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/2265/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1000 bits/sec, 1 packets/sec
5 minute output rate 1000 bits/sec, 2 packets/sec
3291 packets input, 255362 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
7693 packets output, 634592 bytes, 0 underruns
0 output errors, 0 collisions, 2 interface resets
37 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out

BR

ZblaJhaNi

hi,

It is possible that i just hit a Cisco bug? I just turned off CDP on all devices, adn STP and HSRP works like a charm

DevilWAH

its strange casue when i first set up hsrp i got an error lik this, and after I rebooted one of the switch it all worked fine?

togo

Hey Guys... hello .... wel this happen to me once ... here is my issue and solution.

---

Issue: Vlan Flapping for a specific MAC between switches / Vlan going UP/Down in a router (Standby by in HSRP) / Standby for HSRP going from Init to standby state.

---

First thing to remember is that when you use a FHRP (HSRP/VRRP/GLBP) it is a best practice recommendation to use the Active/Master device as the STP root.

In my case I track the MAC in my distribution switches and it was my router, HSRP config was ok on both routers for my vlan but when I checked the STP for that vlan the root was a user switch (people who set the network before did not change STP default values) in that switch there was a pc working as a bridge and sending BPDU with a "BEST" mac for STP root election, when the computer was connected or disconnected STP tried to determine who the STP root was and the flapping message came into action.

---

Solution in this case: change configuration of access ports to stop bpdu messages and reset the switch were the PC was detected to reconfigure STP

---

Additional changes to avoid future problems

1. Verify STP configuration to be sure that yout core routers will be STP root, remember that best practice is to use same device as STP root and FHRP main device.
2. Configure access swtiches to stop BPDUs

Hope this help, as I said this was my particular issue, some forums will say you need to check CPU overload, HSRP/VRRP timers and many other suggestions, I will say do not forget about STP xD !!!

ZblaJhaNi, you already found the issue and solved, but as I mentioned in my case it was a PC and a previous configuration mistake (use of STP default and access ports not configured correctly) the cause of the problem.

Cheers !!!

togo ...

tanix

togo wrote: »

Additional changes to avoid future problems

1. Verify STP configuration to be sure that yout core routers will be STP root, remember that best practice is to use same device as STP root and FHRP main device.

2. Configure access swtiches to stop BPDUs

That one is a big one. In the foundations and the official cert, this is stated over and over again. I assume it is because of this nature of problem why it is consistently mentioned.

fly351

togo wrote: »

1. Verify STP configuration to be sure that yout core routers will be STP root, remember that best practice is to use same device as STP root and FHRP main device.

hmm I understood this as the exact opposite... that is, VLAN traffic should stop at the Distribution layer and not reach the Core. But if you configure the STP root bridge at the Core layer then you are essentially saying that VLAN traffic will go all the way to the Core.

Unless you are talking about a collapsed Core design?

tanix

fly351 wrote: »

hmm I understood this as the exact opposite... that is, VLAN traffic should stop at the Distribution layer and not reach the Core. But if you configure the STP root bridge at the Core layer then you are essentially saying that VLAN traffic will go all the way to the Core.

Unless you are talking about a collapsed Core design?

You are correct I believe, I was focusing more on the aspect of the STP configuration in how it concerns FHRP:

remember that best practice is to use same device as STP root and FHRP main device

As a best practice you want to contain your traffic to local vlans and not have them traverse the core (though there are some situations where end-to-end might be necessary due to a local vlan limitations concerning various business requirements and function).

In a collapsed core design, I don't think it is much of an issue. Core implementation is for the need of a large infrastructure to which needs high bandwidth backbones to connect the infrastructure which is why vlans as well as various controls and packet inspection are relegated to the distribution and access switches in order to limit traffic and decision functions to as relevant as an area as possible (you normally don't want broadcast traffic and the like taking up bandwidth with no purpose across the core).

The key in this issue I believe concerning the OP's problem (and the relevance to my mention) is issue with having your STP root other than your FHRP active or master router as STP has no active element to understand FHRP process and so acts independently which could result in sub-optimal paths among other issues.