Packet loss w/ Static Routes

advanex1

Curious if anyone knows the specific reason for this. For some reason, while doing static routing between 5 routers and I'm pinging from every router to every network, I'll hit a point in which packets are dropped within the ping. A typical ping in these spots would give me 3-4 packets out of 5. All of the other spots typically are 5 of 5. I double checked all of my routes and the routers have multiple ways to access each router (2 ways since 4 routers are in a circle).

Anyone else run into this issue? I can't seem to see what's going on here, this isn't necessarily for the studying, but now I'm curious. When I do RIPv2 and setup the networks, I have no packet loss or delay.

I guess the very curious part about it is that the pings are going through... just that the packets are dropping.

I am doing this on Packet Tracer 5.3 by the way (all routers are 2620xm). I only have 1 router in my lab.. go figure. 1 router and 3 switches, the opposite of what I need right now

Thanks.

hiddenknight821

Try running the traceroute command to do some further troubleshooting. I know it has nothing to do with static route configuration as long as it is configured properly.

Maybe this is just another PT bug, but did you say your topology is 5 routers connected in daisy chained linear toplogy? I'm not suggesting this can be the cause.

advanex1

Hidden, the way they are setup is as such..

1 WAN Router connected to R1 via s0/0
R1 to R2 via FE
R2 to R3 via FE
R3 to SW1 via FE
SW1 to R4 via FE
R4 to R1 via FE

I threw a switch in there to change it up a bit.

The trace route completes sometimes and drops other times, that's why I'm confused. I tried running that as well.

hiddenknight821

advanex1 wrote: »

I threw a switch in there to change it up a bit.

Did it work perfectly if you take out the switch?

Monkerz

Can you post your configs?

advanex1

I'll post the configs in the morning (I'm in Iraq) and see if you guys can make anything of it, also, I'll try removing the switch as well. Headed home to start my reading portion of studying today. Test scheduled for the 19th, we will see how I do.

pham0329

could be the arp process used to resolve the MAC address

advanex1

Found out the issue was in regards to the administrative distance of the static routes. Since they are by default 1, when I changed the admin distance to 2 for the second static route (redundant route) the problem was fixed. No more dropped packets or issues, when I took one route down and it switched to the second static route, no issues there either.

I was talking to a buddy of mine at work (Just got his CCNA and passed his CCNP Route) and he told me the issue had to do with admin distance AND load balancing, was a good conversation and information. Above my skill level (for now), but I love that knowledge.

rickinyorkshire

advanex1 wrote: »

Found out the issue was in regards to the administrative distance of the static routes. Since they are by default 1, when I changed the admin distance to 2 for the second static route (redundant route) the problem was fixed. No more dropped packets or issues, when I took one route down and it switched to the second static route, no issues there either.

I was talking to a buddy of mine at work (Just got his CCNA and passed his CCNP Route) and he told me the issue had to do with admin distance AND load balancing, was a good conversation and information. Above my skill level (for now), but I love that knowledge.

Excellent. Good work.

hiddenknight821

advanex1 wrote: »

Found out the issue was in regards to the administrative distance of the static routes. Since they are by default 1, when I changed the admin distance to 2 for the second static route (redundant route) the problem was fixed. No more dropped packets or issues, when I took one route down and it switched to the second static route, no issues there either.

I was talking to a buddy of mine at work (Just got his CCNA and passed his CCNP Route) and he told me the issue had to do with admin distance AND load balancing, was a good conversation and information. Above my skill level (for now), but I love that knowledge.

Interesting to know this. I would have to give this a try since I thought the switch was the problem. I still don't understand what you mean by the second static route. I was just reading some stuff on routing protocol theory last night, and I thought you are describing routing loop problem, but not sure if that's the case. I hope I can crack this case when I finish my CCNA. If not, then it looks like I would have to go for CCNP-level to understand this.

Monkerz

He had two static routes on each router for each network in the topology. Essentially load balancing the traffic across both interfaces rather than just one. One route exiting the fa0/0 interface and the other exiting the fa0/1 interface.

By changing the AD of one of the static routes higher than default of 1, he created a floating static route. The higher AD would purge the route from the routing table, because the routing table only contains the best routes to each network. This leaves just one path for traffic to take for each destination.

He probably could have messed with CEF and received the same result.

advanex1

Monkerz wrote: »

He had two static routes on each router for each network in the topology. Essentially load balancing the traffic across both interfaces rather than just one. One route exiting the fa0/0 interface and the other exiting the fa0/1 interface.

By changing the AD of one of the static routes higher than default of 1, he created a floating static route. The higher AD would purge the route from the RIB, because the RIB only contains the best routes to each network. This leaves just one path for traffic to take for each destination.

He probably could have messed with CEF and received the same result.

Precisely. I suppose I should have said "floating static route".

When I set multiple routes to the same destination through different hops the show ip route would show something like this..

192.168.3.0/24 [1/0] via 192.168.2.2
[1/0] via 192.168.4.2

As opposed to when you change the administrative distance...

So I have both routes input

S 192.168.3.0/24 [1/0] via 192.168.2.2

S 192.168.3.0/24 [2/0] via 192.168.4.2

But when you do a show ip route, with both of them configured, you will only see 192.168.3.0/24 [1/0] via 192.168.2.2

Hope this helped to explain it a bit. Neat huh?

hiddenknight821

advanex1 wrote: »

Hope this helped to explain it a bit. Neat huh?

It was definitely mind-blowing! Now I got it, especially when Monkerz broke it down for me (+1). I'm still gonna try this at home to see how it plays out when I throw in a bunch of things.

Forsaken_GA

advanex1 wrote: »

Found out the issue was in regards to the administrative distance of the static routes. Since they are by default 1, when I changed the admin distance to 2 for the second static route (redundant route) the problem was fixed. No more dropped packets or issues, when I took one route down and it switched to the second static route, no issues there either.

I was talking to a buddy of mine at work (Just got his CCNA and passed his CCNP Route) and he told me the issue had to do with admin distance AND load balancing, was a good conversation and information. Above my skill level (for now), but I love that knowledge.

I'd be willing to bet that ARP was the culprit more than anything else, you will typically lose packets if someone on the path has to wait on an ARP reply. Having two equal cost routes is not exactly a bad thing.

advanex1

I'm curious then, if you could break it down for me, how ARP would delay the packets if the arp cache was already built? If the ARP cache is built, why would there be a need to wait for the reply? It was wierd to me at first when running traceroute that the route was flip flopping every time I did it until I was explained load balancing.

Curiosity.. I love learning about this stuff.

Forsaken_GA

advanex1 wrote: »

I'm curious then, if you could break it down for me, how ARP would delay the packets if the arp cache was already built? If the ARP cache is built, why would there be a need to wait for the reply? It was wierd to me at first when running traceroute that the route was flip flopping every time I did it until I was explained load balancing.

Curiosity.. I love learning about this stuff.

Well, it all depends on timing. If all you're doing is static routing, then there's no traffic flowing across those links until you put it there, so the ARP cache isn't refreshing constantly, it will time out. You don't have that problem with a routing protocol, because there's always some form of traffic going across the links to keep the arp cache fresh.

The other possibility is you might have screwed up some of your routes, or something down the line was missing a route. Or it could be PT misbehaving.

Consistent packet loss due to equal cost load sharing is not typical as a regular state of operation, and should be investigated for root cause. You should not be carrying the idea that packet loss due to load balancing is normal, it's not.

advanex1

Forsaken_GA wrote: »

Well, it all depends on timing. If all you're doing is static routing, then there's no traffic flowing across those links until you put it there, so the ARP cache isn't refreshing constantly, it will time out. You don't have that problem with a routing protocol, because there's always some form of traffic going across the links to keep the arp cache fresh.

The other possibility is you might have screwed up some of your routes, or something down the line was missing a route. Or it could be PT misbehaving.

Consistent packet loss due to equal cost load sharing is not typical as a regular state of operation, and should be investigated for root cause. You should not be carrying the idea that packet loss due to load balancing is normal, it's not.

It very well could have been user error. God knows I've made plenty of errors. I guess the part in which I didn't assume it's an input error is that when I remove one of the routes, the other works perfectly. To me, that means that the route both ways is configured correctly, just not working well together.

I was pinging, tracerouting, and putting packets on the line all within very close proximity and none of it longer than 180 seconds.. I don't know, I'm new to all of this, but it didn't seem to dawn on me that it could even be ARP under those conditions.

I'll read more into load balancing so that I don't have a poisoned view of it so to speak, I do appreciate the input. After all, like you said, it could have just been PT as well.

Thanks again!

Forsaken_GA

advanex1 wrote: »

It very well could have been user error. God knows I've made plenty of errors. I guess the part in which I didn't assume it's an input error is that when I remove one of the routes, the other works perfectly. To me, that means that the route both ways is configured correctly, just not working well together.

Well, when you used RIP, I'm assuming you also had the equal cost routes in place, and you mentioned that it worked fine. The router isn't going to treat your equal cost routes differently because they're RIP routes instead of static routes, it makes it's forwarding decisions based on what's in the routing table, regardless of how it got there. So you were looking at the same situation with RIP routes as you were with static routes, the only difference is that the RIP routes have an AD of 120.

That's why I suspect if it wasn't arp, it'd be user error, as static routes are more prone to error because it's a human manually determining where the traffic goes. I've done it plenty of times where I forgot to include a route out, or a route back. Generally speaking, in a lab environment, when you see packet loss consistently, it's almost always a missing route. You won't put enough traffic through a lab environment for it to be anything like saturation of the link. In a virtualized setup, you also take out factors like bad cables (but of course you also introduce factors like code errors!)

What I would do is repeat the scenario and see if you get different results. If you get the same results, then start checking your routing tables all over the place and see if you can find anything missing. If you can't validate that, then change the interfaces out from ethernet to serial point to point links (no layer 2 resolution issues then). If you can make the static routes work with serial links with no issues, then I'd put my money back on an arp problem.

advanex1

Forsaken,

Going to try the serial interfaces today and see how that works. I'll let you know. As practice for my exam on friday, I'm practicing RIP v2 setup right now anyways.

Thanks again!

hiddenknight821

I can't believe I fell for it. I actually trusted all the responses to be correct before Forsaken arrived here. Now I dunno which CCNP to trust! I guess I'm better off trusting myself as I build up my skill progressively over the years. Well, knowing that Forsaken is a beast at this game, I'm gonna have to side with him on this since I initially thought the switch (ARP) was the cause too.

advanex1

Interestingly enough, without the switch, I'm still arriving at the same issue even using fastE. I've been reading chapters for now, did my dynamic routing earlier and will change them all over to serials here shortly.To be continued..

instant000

The easiest way to narrow this down:

1. post screenshots of your setup
2. post your configs

==================

Without knowing any of that really, we're left with what you have told us:
You had two routes of equal distance, configured via static routing. After fixing this, you alleviated your "packet loss" problem.

Based on all the data presented, your problems were due to how you had the static routing configured, and your "packet loss" was simply the need to refresh your arp caches when the traffic had to go on an alternate route.

As you previously stated, you did not get this packet loss issue when running a routing protocol, as the routing protocol would keep the arp caches fresh, so no need to have a "lost packet due to the need to arp" when you pinged.

However, you could eliminate all doubts by posting pictures and configs

advanex1

Will do it tonight, cannot access a picture posting website on a government computer. I had them all screen shotted and such and I can't post them now heh, go figure.

Anyways, I didn't think it was an ARP update due to the fact that I was running the trace and pings back to back to back with the same result. If they don't update or stay updated from continuous use... that's odd, heh.

Then again, when I do input the username * privilege 15 password * if I do not somewhere "?" the command line before I enter that command, then the privilege 15 doesn't take on packet tracer..

We will find out tonight! Going to email my file over to my civilian email and post it when I get to the chu. Thanks again!

networker050184

Sounds like a possible packet tracer issue to me.