Unable to ping Default Gateway

So i feel like a complete noob not being able to figure this one out. Here is the scenario.

I am testing out a Layer 3 switch, Cisco 3550. I have two of the interfaces configured as routed ports and traffic is able to get where it needs to but there is a strange issue that I cannot figure out and was wondering if any of you Cisco gurus out there have seen such a thing.

I have a layer 2 switch connected to fa0/2 of the Cisco 3550. Fa0/1 connects to our main router that has connections to the rest of the network. The IP on the 3550 fa0/2 is 192.168.251.1 and the vlan 1 on the layer 2 switch is 192.168.251.2. If i power on the 3550 I can ping the Default Gateway for that interface (192.168.251.1) but after some amount of time elapses I can no longer ping this IP from behind that routed port (a machine connected to the layer 2 switch). I am able to get responses via other subnets but it is something from behind this Layer 3 switch that does not allow the ping responses to be sent back. I do not have any access-lists created to block any traffic.

The real kicker is that the machine connected to the Layer 3 switch is able to get to the internet and ping other gateways in the network so its not like the default gateway is down but it won't ping, and if i ever needed to troubleshoot this device I would definately get hung up on this because I simply can't ping the gateway telling me that i only have Layer 2 network connectivity.

Here is a basic topology to get an idea

InternetRouter<

FA0/1>Cisco3550<Fa0/2

>Cisco2900<---->laptop

Any ideas on this? If any one needs more info let me know.

Thanks in advance.

Cisco3550.pdf

Cisco 2900.pdf

Find more posts tagged with

Save $250 on 2025 certification boot camps from Infosec!

Book now with code EOY2025

Button

Comments

danielno8

busines4u wrote: »

L3Internal#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is 192.168.250.1 to network 0.0.0.0

C 192.168.251.0/24 is directly connected, Vlan1
C 192.168.250.0/24 is directly connected, FastEthernet0/1
S* 0.0.0.0/0 [1/0] via 192.168.250.1

If anyone needs any more information or some additional show commands feel free to ask.

Should this not say Fa0/2?

busines4u

The way that the switch is currently setup is to use vlan1 rather than having fa0/2 as a routed port. Ideally I would like to have it as a routed port, but in order to test i switched it back to use vlan1 as the default gateway to see if i received the same results, which i did. I am going to adjust this once I get back to the office since that is the way I want and need it setup anyways. Good catch though!

I will definitely fire up wireshark and do some packet captures to check out source/destination headers. Hopefully it gives me some good information as to where the hell this traffic is going.

I will also hard code these two interfaces just to rule out that as a possibility, Thanks!

vinbuck

In addition to all the other advice on here, check the following for your intermittent ping connectivity problems.

1.) Physical/Cabling - Ive seen weird things happen when cabling/SFPs/etc go bad, are damaged or incorrectly terminated. I would try different cables and ports on the switch.

2.) IP Address conflict or overlap. Make sure this subnet/IP hasn't been used elsewhere in the network. I would try a subnet that you know isn't used anywhere in the network and see if the issue persists.

3.) Sonicwalls are the bane of my existence

...I've seen more problems with sonicwalls causing unintentional havoc on networks (do an nslookup on webcfs00.com - it's a domain that sonicwall owns and runs their content filtering system - it returns a private address to route to on the Internet). I would definitely wireshark the traffic between the Sonicwall and the 3550. Also, try disconnecting the link between the 3550 and the Sonicwall temporarily while you have a ping running to see if that changes anything.

4.) boot a different IOS. Could be an issue with what you're running or oftentimes connecting equipment from different vendors results in undesirable side effects that shouldn't happen but frequently do.

5.) If you have a different 3550 or better yet a different L3 switch, swap it in with the same config and see what happens...That alone will rule out a bunch of stuff.

busines4u

Great advice! Along the lines of your issue #1. Whenever i first cabled these devices i thought that i would need to use a straight through between the L2 switch and the L3 switch (since the fa0/2 at that time was a routed port) However the link would never come up and would only do so if i used a cross-over cable. That was a bit confusing to me but none the less it did work when using the crossover.

Unfortunately i think this is the only L3 switch we have as well as the only IOS that i have for it. Since its not under contract i don't think i will even be able to download one.

I will definitely test out these other recommendations tomorrow when i get back to the office and will report my findings!

Thanks to all of you offering advice I greatly appreciate it. (I know i've repeated this several times but its great that everyone is friendly and willing to help on TE!)

vinbuck

Another thing I would check is your ARP entries at each piece of gear: Laptop, L2 Switch, L3 Switch and Sonicwall. Make sure the MAC associated with your gateway IP matches the MAC on the interface you've got your gateway IP on.

busines4u

MississippiGuardsman wrote: »

Another thing I would check is your ARP entries at each piece of gear: Laptop, L2 Switch, L3 Switch and Sonicwall. Make sure the MAC associated with your gateway IP matches the MAC on the interface you've got your gateway IP on.

This was one of the first things that i checked. On second thought I did not check the ARP on the SonicWALL since everything from the L3 switch to the SonicWALL and to the rest of the network worked as it should. I will double check though.

vinbuck

busines4u wrote: »

This was one of the first things that i checked. On second thought I did not check the ARP on the SonicWALL since everything from the L3 switch to the SonicWALL and to the rest of the network worked as it should. I will double check though.

In theory the sonicwall shouldn't intefere with any communication on that IP since it sits after the 3550 and not between your host and the 3550. Having said that however, I've been bit by a sonicwall more than once and something that "just shouldn't happen" did

Heero

Is there ANY other device plugged into either the layer 2 switch, or the 3550 (other than port fa0/1)?

busines4u

I agree with this statement. I recently encountered our SonicWALL sweeping an entire subnet and poising the switches MAC table with every IP translating to the SonicWALL's MAC address. As you can image every machine on this subnet received the duplicate IP address error.

busines4u

Heero wrote: »

Is there ANY other device plugged into either the layer 2 switch, or the 3550 (other than port fa0/1)?

Nope These are the only two devices connected the the L3 switch.

vinbuck

busines4u wrote: »

I agree with this statement. I recently encountered our SonicWALL sweeping an entire subnet and poising the switches MAC table with every IP translating to the SonicWALL's MAC address. As you can image every machine on this subnet received the duplicate IP address error.

Have you started a continuous ping to see if there is a pattern when it goes to timeout? Does the ping ever start back without intervention (i.e. power cycling the switch) ?

Take a look at this...

http://packetlife.net/wiki/well-known-intervals/

busines4u

MississippiGuardsman wrote: »

Have you started a continuous ping to see if there is a pattern when it goes to timeout? Does the ping ever start back without intervention (i.e. power cycling the switch) ?

Take a look at this...

Well-known Intervals - Packet Life

After switching from VLAN1 routing the traffic to fa0/2 being a routed port i started a continuous ping and will keep a close eye on it to see if i can put a time stamp on when the pings stop. Once i stop receiving replies i plan on capturing some traffic to get some detailed information.

busines4u

Well i came in this morning and was no longer receiving replies. I don't have exact time stamps as to when they stopped but the sequence number was 47042. I restarted the device and the damn thing replied so i let it run throughout the day and was only able to reach about half of that number (47042). So if it turns out that the pings stop after 47042 again then there would be a good indication of something going on there, just not sure what.

I did capture some packets and both the packets sent that did not receive a reply and that did receive a reply looked identical. The src & dst MAC addresses were accurate, the CAM tables on both switches were golden. I could ping the L2 switch from the Laptop connected directly to the L2 switch and I could ping the L3 switch still from my laptop located elsewhere on the network.

So it appears that there is something on that L3 switch preventing this. I did remove the DHCP thinking that that could be the problem since that was the only other configuration that i had on the L3 switch. If i check on the laptop tomorrow and it is still replying than something was going on with DHCP. Not sure what but at least it is finally something definite to point the finger at. I will update as time allows and as this issue progresses.

I really wish i had another IOS or L3 switch to swap.....

Forsaken_GA

do you have another device you could attach to the port? I had a friend who recently had almost exactly the same issue (except it was an ESXi host), and the problem turned out to the be the host. Once he reinstalled it, it worked fine.

If you can possibly put another device on the vlan and see if it gets the same results, that will tell you for sure whether it's a host issue or a network issue. I think we may all have been committing a cardinal sin by assuming it was the network when that hasn't been established for sure. (there's a reason why very senior network engineers default to the response 'ITS NOT THE NETWORK!')

burbankmarc

What type of host is it? I've had linux installs with bad intel drivers where traffic would just stop.

busines4u

Forsaken_GA wrote: »

do you have another device you could attach to the port? I had a friend who recently had almost exactly the same issue (except it was an ESXi host), and the problem turned out to the be the host. Once he reinstalled it, it worked fine.

If you can possibly put another device on the vlan and see if it gets the same results, that will tell you for sure whether it's a host issue or a network issue. I think we may all have been committing a cardinal sin by assuming it was the network when that hasn't been established for sure. (there's a reason why very senior network engineers default to the response 'ITS NOT THE NETWORK!')

The host is a MacBook running 10.6. I have the L2 switch connected to it and it is unable to receive any replies either. Once the L3 switch stopped responding i did hook up a windows laptop to it but received the same result (no replies)

I checked on this laptop this morning and the pings stopped again. This was after removing DHCP scope from the L3 switch. The pings made it much further this time compared to last but they still stopped. Once the L3 switch stopped responding from that side then I tested with other parts of the network and received replies.

Is there any default security mechanism placed in these devices to prevent such things as DoS attacks. Maybe the L3 switch is doing this for this subnet only or something?

What i decided to do what start a continuous ping from my laptop (located elsewhere on the network) to see how long it would take before the L3 switch stopped responding. This will at least tell me if it is something located behind the L3 switch or not since i am coming in on a different interface.

busines4u

Well the ping that was sourcing from elsewhere in the network is still going. I connected to the L2 switch and tried to ping the same exact interface but do not receive any replies. So it looks like it is strictly related to that subnet. Any ideas anyone?

instant000

busines4u wrote: »

Well the ping that was sourcing from elsewhere in the network is still going. I connected to the L2 switch and tried to ping the same exact interface but do not receive any replies. So it looks like it is strictly related to that subnet. Any ideas anyone?

On the switch, do a "sh log" you should get a hint there, I'm hoping.

busines4u

instant000 wrote: »

On the switch, do a "sh log" you should get a hint there, I'm hoping.

I did a show log but do not see anything that would point me in the right direction. Below is the output of it in case i am over looking something.

L3Internal#sh log
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
Console logging: level debugging, 11 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 11 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Trap logging: level informational, 14 message lines logged

Log Buffer (4096 bytes):

00:00:37: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
00:00:39: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
00:00:39: %SYS-5-CONFIG_I: Configured from memory by console
00:00:40: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C3550 Software (C3550-IPBASE-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 19-Jul-07 21:08 by nachen
00:00:42: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up
00:00:43: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
00:00:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
00:00:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
00:00:47: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
00:00:49: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
00:00:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up

busines4u

I decided to see what type of results I would get if i let a Windows Machine ping the interface. The ping has been running successfully for the past couple of days, which tells me that there is something up with the Mac Client that causes the interface to stop replying to icmp's. I tried to move the windows machine over to this same port but the port still refuses to respond. Not too sure what the Mac OS does to cause this type of behavior but i'm hoping its just some sort of bug in the IOS.

instant000

Is that what you see in the log, from both before and after the machine stops responding?

Let me be clear:

Please check what is in the log, from both before and after that machine stops responding.

If there is something that changes about the switch, it would more than likely be logged.

busines4u

Well i compared the logs and didn't see anything that was too helpful for me. I am going to paste the output below to have another set of eyes take a look at it. One thing that is interesting is that it says that there is 1 message rate-limited. I'm not too sure on what show command i need to run to give me some more information. I did do "show int rate-limit" but it does not have any information.

One interesting thing here is that I had a Ubuntu, Windows and Mac machine ping the L3 switch throughout the whole weekend. Whenever i checked on it this morning the only machine that was not receiving replies was the Mac. Windows and Ubuntu both were receiving replies. The only difference between this is that the Windows machine and the Ubuntu machine are located elsewhere on the network. I am planning on moving them to connect directly to the L3 switch along with a windows machine to see if i see similar or different results. Just wanted to update. Here is the show log output.

Switch is not responding:

Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
Console logging: level debugging, 17 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 17 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Trap logging: level debugging, 20 message lines logged
Logging to 192.168.1.38, 20 message lines logged, xml disabled,
filtering disabled

Log Buffer (4096 bytes):

00:00:37: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
00:00:39: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
00:00:39: %SYS-6-CLOCKUPDATE: System clock has been updated from 00:00:39 UTC Mon Mar 1 1993 to 19:00:39 EST Sun Feb 28 1993, configured from console by console.
*Feb 28 19:00:39.483: PSECURE: Assert failure: 0: ../switch/psecure/psecure_ifc.c: 291: psecure_get_vlanid
*Feb 28 19:00:39.483: -Traceback= 378DF4 37D7B4 376BFC 37AFB8 3159E8 325B4C 23B7CC 675A94 68F39C 68F7D4 780344 77A908 772FFC 77D18C 77D3F4 6D20AC
00:00:39: %SYS-5-CONFIG_I: Configured from memory by console
00:00:40: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C3550 Software (C3550-IPBASE-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 19-Jul-07 21:08 by nachen
00:00:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
00:00:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
00:00:48: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up
00:00:49: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
00:00:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
00:00:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
00:11:12: %SYS-5-CONFIG_I: Configured from console by vty0 (192.168.1.14

00:13:17: %SYS-6-CLOCKUPDATE: System clock has been updated from 19:13:17 EST Sun Feb 28 1993 to 19:13:17 EST Sun Feb 28 1993, configured from console by vty0 (192.168.1.14

.
00:13:19: %SYS-5-CONFIG_I: Configured from console by vty0 (192.168.1.14

00:13:36: %SYS-6-CLOCKUPDATE: System clock has been updated from 19:13:36 EST Sun Feb 28 1993 to 10:50:00 EST Fri Apr 15 2011, configured from console by vty0 (192.168.1.14

.

Switch is responding:

Reloaded switch show log int is pinging:
L3Internal#sh log
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes, 0 overruns, xml disabled, filtering disabled)
Console logging: level debugging, 13 messages logged, xml disabled,
filtering disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 13 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Trap logging: level debugging, 16 message lines logged
Logging to 192.168.1.38, 16 message lines logged, xml disabled,
filtering disabled

Log Buffer (4096 bytes):

00:00:37: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
00:00:39: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
00:00:39: %SYS-6-CLOCKUPDATE: System clock has been updated from 00:00:39 UTC Mon Mar 1 1993 to 19:00:39 EST Sun Feb 28 1993, configured from console by console.
*Feb 28 19:00:39.483: PSECURE: Assert failure: 0: ../switch/psecure/psecure_ifc.c: 291: psecure_get_vlanid
*Feb 28 19:00:39.483: -Traceback= 378DF4 37D7B4 376BFC 37AFB8 3159E8 325B4C 23B7CC 675A94 68F39C 68F7D4 780344 77A908 772FFC 77D18C 77D3F4 6D20AC
00:00:39: %SYS-5-CONFIG_I: Configured from memory by console
00:00:40: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C3550 Software (C3550-IPBASE-M), Version 12.2(35)SE5, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 19-Jul-07 21:08 by nachen
00:00:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
00:00:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
00:00:48: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up
00:00:49: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
00:00:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
00:00:50: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up

os73355

i am stuck in a issue! unable to ping the SVI
i am design a small network for a office.
1 router 2811
1 switch 3750-e

Router is connected to the mpls cloud with ospf.

here re the config.

Router#
int fa0/0
ip 10.10.10.1 255.255.255.252
des connected to switch
no shut
!
int fa0/1
ip 20.20.20.1 255.255.255.252
des connected to MPLS cloud
no shut
!

SWITCH#config t
Vlan 201
exit
int vln 201
ip address 22.0.68.251 255.255.255.0
des USER VLAN
no shut
int fa1/0/48
no switchport
ip address 10.10.10.2 255.255.255.252
no shut
!
ip routing
ip route 0.0.0.0 0.0.0.0 10.10.10.1
!
!
!
int fa1/0/1
switchport
switchport mode access
switchport access vlan 201
no shut
+++++++++++++++++++++++++++++++
SVI VLAN201 is UP
i connect my laptop and give ip 22.0.68.1 255.255.255.0 and default gateway 22.0.68.251
but can not ping SVI VLAN 201 (22.0.68.251) ?

and from the SWITCH i can not ping the 20.20.20.2?

please help.