OSPF routes being dropped every few seconds

mattsthe2

Heres the topology,

ISP
/ \
R1 R2
| |
\ /
R3


R1 and R2 are iBGP peers and R1 and R2 are BGP Peers to the ISP Router
R1 and R2 are doing two way redistribution of BGP and OSPF
R3 runs only OSPF

When i do a sh ip route on R3. I see the ISP routes which show as OSPF E2 routes however if i continuously do a sh ip route those OPSF E2 routes will be removed after about 6 seconds and come back a few seconds later.

I have noticed that if i remove the iBGP Peer between R1 and R2 the OPSF E2 routes will stick.


What gives?

networker050184

Post the configs. I can't think of anything that would cause that with a straight up simple config.

mikej412

Without any configuration information, I'd guess you failed at loop prevention with your two way redistribution on two different routers.

ColbyG

Does your OSPF or BGP relationship die? Have you done any debugs?

mattsthe2

R1 -

interface FastEthernet0/0
description To-6K
ip address 172.17.50.16 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/1
description to ISP
ip address 192.168.2.2 255.255.255.0
duplex auto
speed auto
!
router ospf 1
log-adjacency-changes
redistribute bgp 6400 metric 100 subnets
network 172.17.50.0 0.0.0.255 area 0
!
router bgp 6400
bgp log-neighbor-changes
neighbor 172.17.51.16 remote-as 6400
neighbor 192.168.2.1 remote-as 6500
neighbor 192.168.2.1 password cisco
!
address-family ipv4
redistribute ospf 1 metric 50 match external 1 external 2
neighbor 172.17.51.16 activate
neighbor 172.17.51.16 next-hop-self
neighbor 192.168.2.1 activate
neighbor 192.168.2.1 next-hop-self
neighbor 192.168.2.1 soft-reconfiguration inbound
no auto-summary
no synchronization
network 172.17.50.0 mask 255.255.255.0
exit-address-family





R2 -

interface FastEthernet0/0
ip address 172.17.51.16 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/1
description To-ISP
ip address 192.168.1.2 255.255.255.0
shutdown
duplex auto
speed auto
!
router ospf 1
log-adjacency-changes
redistribute bgp 6400 metric 40 subnets
network 172.17.51.0 0.0.0.255 area 0
!
router bgp 6400
bgp log-neighbor-changes
neighbor 172.17.50.16 remote-as 6400
neighbor 192.168.1.1 remote-as 6500
neighbor 192.168.1.1 password cisco
!
address-family ipv4
redistribute ospf 1 metric 60 match external 1 external 2
neighbor 172.17.50.16 activate
neighbor 192.168.1.1 activate
neighbor 192.168.1.1 next-hop-self
neighbor 192.168.1.1 soft-reconfiguration inbound
no auto-summary
no synchronization
network 172.17.51.0 mask 255.255.255.0
exit-address-family





ISP -

interface FastEthernet0/0
description To-3845-1
ip address 192.168.2.1 255.255.255.0
duplex auto
speed auto
!
interface FastEthernet0/1
description To-3845-2
ip address 192.168.1.1 255.255.255.0
duplex auto
speed auto
!
router bgp 6500
no synchronization
bgp log-neighbor-changes
network 172.17.11.0 mask 255.255.255.0
network 172.17.14.0 mask 255.255.255.0
network 172.17.32.0 mask 255.255.255.0
network 172.17.48.0 mask 255.255.255.0
neighbor 192.168.1.2 remote-as 6400
neighbor 192.168.1.2 password cisco
neighbor 192.168.2.2 remote-as 6400
neighbor 192.168.2.2 password cisco
neighbor 192.168.2.2 next-hop-self
no auto-summary




R3 -

interface FastEthernet0/0
ip address 172.17.50.2 255.255.255.0
duplex auto
speed auto
standby 1 ip 172.17.50.1
!
interface FastEthernet0/1
ip address 172.17.51.2 255.255.255.0
duplex auto
speed auto
!
router ospf 1
log-adjacency-changes
redistribute connected subnets
network 172.17.50.0 0.0.0.255 area 0
network 172.17.51.0 0.0.0.255 area 0

mattsthe2

mikej412 wrote: »

Without any configuration information, I'd guess you failed at loop prevention with your two way redistribution on two different routers.

Mike that's the exact problem.

On R1, routes from the ISP are being advertised one minute by the ISP the next they are being advertised by R2 inside address. Whats the best way to stop this without breaking the iBGP relationship?


#Sh ip route

172.17.0.0/16 is variably subnetted, 8 subnets, 2 masks
O 172.17.51.0/24 [110/20] via 172.17.50.2, 00:09:23, FastEthernet0/0
C 172.17.50.0/24 is directly connected, FastEthernet0/0
O E2 172.17.43.0/24 [110/20] via 172.17.50.2, 00:09:23, FastEthernet0/0
O E2 172.17.42.0/24 [110/20] via 172.17.50.2, 00:09:23, FastEthernet0/0
B 172.17.32.0/24 [20/0] via 192.168.2.1, 00:00:04
B 172.17.14.0/24 [20/0] via 192.168.2.1, 00:00:04
B 172.17.11.0/24 [20/0] via 192.168.2.1, 00:00:04
O E2 172.17.0.0/21 [110/20] via 172.17.50.2, 00:09:23, FastEthernet0/0
O E2 172.20.0.0/16 [110/20] via 172.17.50.2, 00:09:23, FastEthernet0/0
C 192.168.2.0/24 is directly connected, FastEthernet0/1


#Sh ip route

172.17.0.0/16 is variably subnetted, 8 subnets, 2 masks
O 172.17.51.0/24 [110/20] via 172.17.50.2, 00:09:25, FastEthernet0/0
C 172.17.50.0/24 is directly connected, FastEthernet0/0
O E2 172.17.43.0/24 [110/20] via 172.17.50.2, 00:09:25, FastEthernet0/0
O E2 172.17.42.0/24 [110/20] via 172.17.50.2, 00:09:25, FastEthernet0/0
B 172.17.32.0/24 [200/60] via 172.17.51.16, 00:00:01
B 172.17.14.0/24 [200/60] via 172.17.51.16, 00:00:01
B 172.17.11.0/24 [200/60] via 172.17.51.16, 00:00:01
O E2 172.17.0.0/21 [110/20] via 172.17.50.2, 00:09:25, FastEthernet0/0
O E2 172.20.0.0/16 [110/20] via 172.17.50.2, 00:09:25, FastEthernet0/0
C 192.168.2.0/24 is directly connected, FastEthernet0/1

networker050184

You are learning your internal OSPF routes back via eBGP. You need to implement some filtering. This is why mutual redistribution without filtering is a bad idea.

mikej412

mattsthe2 wrote: »

Whats the best way to stop this without breaking the iBGP relationship?

Well, without having a clue why you're doing this or what you're trying to accomplish -- I'd guess that just redistributing OSPF INTERNAL routes into BGP might kill the loop.

mattsthe2

mikej412 wrote: »

Well, without having a clue why you're doing this or what you're trying to accomplish -- I'd guess that just redistributing OSPF INTERNAL routes into BGP might kill the loop.

Cool that worked.

I am putting in a redundant connection on R2 to our ISP.
This was all mocked up on Dynamips.

cisco_trooper

mikej412 wrote: »

Without any configuration information, I'd guess you failed at loop prevention with your two way redistribution on two different routers.

I agree. Let's see some configs, this should be easy to hammer out.


EDIT:
Okay, next time I real the entire post before I put my two cents in.

mattsthe2

mikej412 wrote: »

Well, without having a clue why you're doing this or what you're trying to accomplish -- I'd guess that just redistributing OSPF INTERNAL routes into BGP might kill the loop.

Ok i was quick to jump the gun, although the loop was fixed.
My R3 OSPF routes are not making it up to the ISP now after making this change per Mike on R1 & R2


R2
router bgp 6400
redistribute ospf 1 metric 60

R1
router bgp 6400
redistribute ospf 1

mikej412

mattsthe2 wrote: »

This was all mocked up on Dynamips.

YEAH!!

I was afraid to ask if this was in a lab or production

mattsthe2

mikej412 wrote: »

YEAH!!

I was afraid to ask if this was in a lab or production

Nope,

So could i enable the external redistribution into BGP like i had before and apply like a distribution list to solve my issue?

networker050184

You shouldn't need to redistribute external OSPF routes unless those networks from R3 or beyond are already external on R1 and R2.

Is there a reason you need the ISP to know about all your OSPF routes? I'd use some route maps to filter and only send/receive what you need. Your set up is ok for a lab, but I wouldn't leave a session that wide open in the real world.

Also what kind of routes are you going to be getting from the ISP? You don't need to put a whole or even partial table into OSPF. A default route pointed towards your border routers should do the trick.

kryolla

why are you redistro connected on R3 just put a network statement or figure out what's causing the loop. You are only redistro internal OSPF routes into BGP but in production do you have any externals. Can you get the same routing table entries into your lab environment which will give you a better picture on what happens when you add that redundant link and setup iBGP peering. Also isn't your ISP doing any filtering.

mattsthe2

ok fair play guys heres some more information.
I try not to load up to much information into the post to overwhelm people.

The ISP is really my MPLS provider router and im using this to simulate our WAN network.
That is the reason why im sending all the routes too and from the ISP router.

On R3 i setup a bunch of Loopbacks to simulate SVI's.
When i removed the external from the redistrubed statement on R1 and R2 i noticed that the ISP router did not get my Loopback addresses.

But as i stated above when i put that in i get the Routing Loop situation.

cisco_trooper

mattsthe2 wrote: »

ok fair play guys heres some more information.
I try not to load up to much information into the post to overwhelm people.

The ISP is really my MPLS provider router and im using this to simulate our WAN network.
That is the reason why im sending all the routes too and from the ISP router.

On R3 i setup a bunch of Loopbacks to simulate SVI's.
When i removed the external from the redistrubed statement on R1 and R2 i noticed that the ISP router did not get my Loopback addresses.

But as i stated above when i put that in i get the Routing Loop situation.

You are going to have to use some filters. If you are going to do two way redistribution there is no way around distribute-lists or route-maps in order to prevent re-learning about OSPF networks from eBGP.

networker050184

cisco_trooper wrote: »

You are going to have to use some filters. If you are going to do two way redistribution there is no way around distribute-lists or route-maps in order to prevent re-learning about OSPF networks from eBGP.

I agree. I also don't see why you need to redistribute BGP into OSPF. If both routers are going to your one MPLS router then a default should do the trick. No matter where your traffic is going it has to go out that router so having all the information isn't really doing you any good. The BGP session in a MPLS VPN is mostly for you to advertise your routes to the provider. You don't really need to receive anything from them besides a default unless you have more than one exit point. Remember more complicated doesn't always mean better.

Is there more to this that you aren't telling us still?

mattsthe2

Thanks for sticking with me on this guys!

The ISP router in the diagram is a router that in production I would not have control over, its the MPLS Providers router and both R1 and R2 most likely will connect to two different ISP routers, I just created one for ease to mock it up in the lab.

I need to redistrib from BGP to OSPF because also on R1 and R2 i have public internet connections so i use a default route for those.

So filtering that you both have suggested what would i be filtering and what router/interface would i be filtering on?

kryolla

networker050184 wrote: »

Is there more to this that you aren't telling us still?

mattsthe2 are you really in charge of getting this done or are you just curious because this seems over your head

cisco_trooper

mattsthe2 wrote: »

So filtering that you both have suggested what would i be filtering and what router/interface would i be filtering on?

These are route filters. They are used to filter which routes can be learned/advertised via your routing protocols. They will be defined under your routing protocol configurations. You can use a distribution list or a route-map.

Check out chapters 11 and 12 in the BSCI if I remember right...

cisco_trooper

If you're really curious check out Chapters 11 thru 14 in Routing TCP/IP Volume I, Second Edition. If you're not straight after that then you got problems

APA

You need to read BGP Design & Implementation by Randy Zhang....

Some of the chapters go through the exact route oscillation that you are experiencing...

It's to do with the issues of redistribution and why it's a dangerous game occasionally when you have redundancy and you've got it wrong.

It's pretty much to do with the BGP scanner running and realising that there is a better path to the IGP route because you aren't filtering the redistribution of OSPF back into BGP, then a few minutes later running again and realising no there is another better route via iBGP...... and so begins the oscillation that you are seeing.

If you have loopbacks on R3 and they are advertised into the OSPF topology.... why not just use network statements on R1\R2 to advertise the networks cleanly to the ISP..... Avoid redistributing OSPF into BGP....

Remember that BGP only needs the route to be in the IP routing table for a network statement to be injected into iBGP\eBGP....

mattsthe2

APA good suggestions, and thanks everyone for the Help. I think I'm armed to solve this problem.

kryolla wrote: »

mattsthe2 are you really in charge of getting this done or are you just curious because this seems over your head

kryolla - Yes, I'm really in charge of getting this done, and it is a bit out of my skillset, but I'm trying and I have someone checking my work before it ever hits production. Man i really feel like adding to your reputation for your helpful and uplifting response. Cheers!

ColbyG

This really is for production? Can you get some assistance from anyone other than the internet? This is a pretty big deal to just wing it.

burbankmarc

I tried recreating this issue. I thought it'd be interesting to see.

I tried, but I couldn't get it to work. I copied the configs you posted, and just added the loop backs on the ISP router so it would inject the routes into BGP.

My R1 and R2 prefferred all external routes to eBGP, and all internal routes to OSPF. Which makes sense given the ADs. Did I miss a step though, since I couldn't reacreate the issue?