Network loop

-prophet-

I have just started at a new company and am getting to grips with their Foundry switched network.

Last week, the servers started getting high pings and then the entire network went down. We searched for some time to find the source of the problem:

Someone had plugged their ethernet cable from the PC back into another port in the wall, therefore creating a loop...

I was under the impression STP takes control of this?
How can this problem occur?

Cheers,
prophet

macdude

Is STP enabled? Can prevent this problem sometimes.

Paul Boz

Spanning tree would (hopefully) prevent a loop if you plugged in a bridge/switch/other intermediary device.

What's happening in your situation is this:

Rather than plugging in an intermediary device, someone is literally putting a loop between the two ports on your switch. After all, the data outlet installed in people's cubicle is just an ethernet cable terminated to a wall jack. By connecting the two wall jacks that person is closing the circuit between two ports. You can simulate this by taking a patch cable and running it between two ports. Spanning tree won't take action because nothing within spanning tree is designed to block a port if the BPDU is originating on the switch itself.

It's a handy technique for testing interfaces. Loop tests are used throughout the telecom industry to do exactly this

-prophet-

Paul Boz wrote:

What's happening in your situation is this:

Rather than plugging in an intermediary device, someone is literally putting a loop between the two ports on your switch. After all, the data outlet installed in people's cubicle is just an ethernet cable terminated to a wall jack. By connecting the two wall jacks that person is closing the circuit between two ports. You can simulate this by taking a patch cable and running it between two ports.

Thanks Paul!

Is there a method to combating this situation?

Cheers,
prophet

Paul Boz

-prophet- wrote:

Paul Boz wrote:

What's happening in your situation is this:

Rather than plugging in an intermediary device, someone is literally putting a loop between the two ports on your switch. After all, the data outlet installed in people's cubicle is just an ethernet cable terminated to a wall jack. By connecting the two wall jacks that person is closing the circuit between two ports. You can simulate this by taking a patch cable and running it between two ports.

Thanks Paul!

Is there a method to combating this situation?

Cheers,
prophet

There are several methods to combat the problem that I can think of.

I don't know if Foundry switches support port security, but on Cisco switches you can statically bind a MAC address to a port. If you did this, whenever someone plugged in a loop in your network the port would detect the MAC address of the now-connected switchport and error-disable because it's the wrong MAC address for the port.

Do the employees need multiple data outlets or did the person connect the cable between two cubicles or something? If it's the former, make some labels and place them over the data jacks with instructions not to bridge the connections. If it's the latter, it sounds like a little mischief and a stern talking to will prevent it from happening again.

-prophet-

Paul Boz wrote:

Do the employees need multiple data outlets or did the person connect the cable between two cubicles or something?

We have meeting rooms with ports available in the floor for employees to access the network when doing presentations. The problem here is that it is a different computer every time. It would be nice to have a fool proof system as it is too easy to bring the whole network to it's feet.

Cheers,
prophet

marlon23

The only reasonable solution is to buy Cisco switches nextime

However, the features you are looking for are : Cisco's BPDU guard or BPDU filter. BPDU guard, when enabled on port, will put the port down when there is BPDU received. BPDU filter will just filter it.
Try to ask Foundry guys if they have some similar feature.
Overally, I think that it is just not very smart way of spanning-tree implementation by Foundry, which is causing your problems (For example Cisco's switches would detect & adapt to such a situation).

Good luck

kryolla

edit

marlon23

kryolla:

""Pings and STP are two different protocols. Pings work at layer 3 and STP (BPDU) works at layer 2. STP prevents broadcast storms from happening since there is no way to stop it. Where as at layer 3 there is a TTL field that stop loops.""

I dont agree with you, broadcast storm is just general term used to describe a state when there is huge amount of broadcasts running over network. STP is just creating loop-free logical topology, even with STP you can have broadcast storm, simplest one is just ping flood. STP is not stopping this, there is a feature called storm-control, which is designed to fight broadcast storm, not STP.

kryolla

edit

darkuser

first you must understand a bridging loop BEFORE
you delve into spanning tree.
i know this is cisco info but the topics are automomous
ever heard of a infinate loop in programming ?
well spanning tree is based on an algorythmic process.
the rocket scientist that cabled two jacks together
gave you a wonderful lab experiment in what never to do.


http://www.cisco.com/en/US/docs/internetworking/technology/handbook/Transparent-Bridging.html

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/sw_ntman/cwsimain/cwsi2/cwsiug2/vlan2/stpapp.htm

kryolla

edit

marlon23

kryolla:

You should look also outside the certification books during study. If they mention broadcast
storm in same chapter with STP it doesnt mean that they absolutely rely on each other.
Yes, ICMP and STP are USING the same layer, it is layer 2 and protocol is called Ethernet.
Imagine I'll run command "#ping -f 10.255.255.255" from a PC connected via GigE, tell me how does STP stop this broadcast storm ? Or imagine 10 vired PC's doing the same thing on the same segment.
STP is used to fight one of many causes of broadcast storms.
Using storm control is not a poor design, it actually very advanced & smart design. What kind of today's host applications needs to send 0.1-1-10-100-1000Mb of L2 broadcast traffic ???
Just think out of the box or just google it.

btw: "Pings and STP are two different protocols" . - Ping is a program, not protocol.

kryolla

edit

marlon23

Thanks, Good luck on ICSW, ONT, DESGN, ARCH, CCIE written & finding your self a job.

kryolla

edit

-prophet-

Wow, a lot of info and a little heat as well...

Thanks for the tips guys.

Cheers,
prophet

marlon23

Kryolla:

"To the OP STP wont stop a layer 2 DOS (broadcast storm) so you need to find a way to isolate your server from this type of attack perhaps put it on a different segment since routers dont forward broadcast. Cisco has storm control you can use but not sure about Foundry."

Seem like you get it finally

kryolla

edit

malcybood

on the matter in hand not the bickering..........

we had a network loop similar to this on one of our nortel lan switch stacks in head office - 350 users in the building.

a user plugged a live ip phone ethernet cable (that should have only been plugged into a mobile user laptop) into a wall point that was patched up for a hot desk which caused a network loop on the switch stack, bringing 100 users on the 2nd floor to a halt.

we narrowed it down to stp being disabled on all 8 switches in the 2nd floor stack which was rather bewildering as stp was configured and tested during rollout........

on further investigation a few days after the event we discovered that when we created an additional vlan on the switch stack to separate our voice traffic from data traffic onto searate vlans when setting the switch config, there was a bug that disabled stp....a great nortel 'feature' lol

it was weird when it was happening and took about 8 hours to troubleshoot as when we powered down the stack and powered up the switches one at a time it wasn't until switch 6 was powered on things started really grinding to a halt although the loop was actually in switch 2.

fortunately management seen sense and we have some good network monitoring software that alerts us of this type of stuff now!

was a good learning exercise although not what I said at the time....

another thing to look for that can cause similar issues is if you have dual homed servers setup with ip and MAC teaming.

yes, I have also had this issue where somebody plugged a dual homed nic teamed server into one of our core l3 data center switches, which made it go crazy! wasn't stp that time that caused the issue, but it just shows you not always what you would first assume!

gojericho0

kryolla wrote:

Yeah after much thought there are different types of broadcast storm. 1 caused by layer 2 loop and 2 caused by Denial of service attacks. This is why I like open forums

Check out the smurf attack as well. It causes its own DoS and loop by spoofing the host address

http://www.nordu.net/articles/smurf.html

You can really cause havoc if you make the source address look like another network within the internetwork