BGP hub and spoke over MPLS VPN

I was re-reading the MPLS Fundamentals book and decided I would lab up the BGP hub and spoke topology thats in the book and modify it a little to fit into my existing BGP/MPLS GNS3 study topology.

Anyway, it took me a lot longer than expected (seeing as I am only about 4 months out from my RS lab), and definately reinforces the point to consider the basics before trying to use your "CCIE ninja tricks" to solve the problem.

Because it might be a bit of fun, I'll post a picture of my topology BEFORE I changed a few things around to eventually solve the problem, and walk through the steps up until the problem occurs. If anyone is interested to have a crack at trying to fix the problem by looking only at the topology and some partial configs, please post here!

Here is a picture of the topology that is currently NOT working. Remember our goal is to create a hub and spoke topology where BGP spoke sites advertise their routes to the hub site, and the hub site will advertise them back to the other spoke sites. Therefore, spoke-to-spoke communication occurs through the hub (otherwise we could just set the import RT of one site to the export RT of the other sites and be done with it).

Now for the steps up until the problem occurs.
1. CE router R6 injects network 6.6.6.6/32 into BGP
2. R6 advertises the IPv4 prefix to PE router R4 over EBGP with an AS_PATH of [236].
4. R4 slaps an RD of 230:6 onto the route to create a VPNv4 prefix of 230:6:6.6.6/32 and advertises this to PE router R1 over IBGP. The export RT of 230:6 is also slapped onto the route.
5. R1 recieves the VPNv4 route and looks for any VRFs importing RT 230:6. VRF 230 is. R4 removes the RD of 230:6 and puts it into the BGP VPN table for VRF 230.
6. R1 then advertises the IPv4 prefix over EBGP to CE routers R2 and R3 with the AS_PATH of [14 236]
7. R2 and R3 receive the route, perform bestpath selection, and then advertise the route to each other via IBGP.

After step 7, a few problem occurs. R1 does not process the update received from the CE routers. This is the first problem and is pretty easy to fix

Once the first problem is fixed, R1 still does not send the VPNv4 route to R4 with the hub-to-spoke RT on it, meaning that the spokes never receive each other's routes through the hub. This is the second problem.

TLDR: Look at topology, look at the partial config below, identify the two problems.

Note that all links between routers have the third octet identifying the routers on the link (e.g. R1-R4 link = 10.0.14.0/24). The host address on all links is simply set to the number of the router (e.g. R6 = .6)

Partial config on R1:

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config)#ip vrf 230
R1(config-vrf)#rd 230:1
R1(config-vrf)#route-target import 230:5
R1(config-vrf)#route-target import 230:6
R1(config-vrf)#route-target export 230:5
R1(config-vrf)#route-target export 230:6[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]
[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config)#router bgp 14
R1(config-router)#address
R1(config-router)#address-family ipv4 vrf 230
R1(config-router-af)#neighbor 10.0.12.2 remote-as 230[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config-router-af)#neighbor 10.0.13.3 remote-as 230[/FONT]
[/FONT][/COLOR]

Partial config on R4:

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R4(config)#ip vrf 235
R4(config-vrf)#rd 230:5
R4(config-vrf)#route-target both 230:5

[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R4(config)#ip vrf 236
R4(config-vrf)#rd 230:6
R4(config-vrf)#route-target both 230:6
[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]
[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R4(config)#router bgp 14
R4(config-router)#address-family ipv4 vrf 235
R4(config-router-af)#neighbor 10.0.45.5 remote-as 235
R4(config-router)#address-family ipv4 vrf 236
R4(config-router-af)#neighbor 10.0.46.6 remote-as 236[/FONT]
[/FONT][/COLOR]

Note that the problem only requires fixing on R1 and R4 so don't worry about the other routers. Feel free to ask questions and I will give you more hints.

The end goal is to have this:

R5#show ip bgp
BGP table version is 28, local router ID is 10.0.45.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2.2.2.2/32       10.0.45.4                              0 14 230 i
*> 3.3.3.3/32       10.0.45.4                              0 14 230 i
*> 5.5.5.5/32       0.0.0.0                  0         32768 i
*> 6.6.6.6/32       10.0.45.4                              0 14 230 14 236 i
*> 10.0.23.0/24     10.0.45.4                              0 14 230 i

BGP_hubspoke.jpg

Find more posts tagged with

Comments

Jackace

One question for you.

"R1 does not process the update received from the CE routers" Is that all CE routers? or just specific routers?

Edit - My first initial response to the issue is it looks like your issues are at least partially caused by BGP loop prevention mechanisms.

The first things I would try are using allowas-in on the peering sessions with R2 and R3, and I would change the RTs you are using. From Routers 5 and 6 you can import using the same RT, but you need to export 2 different RTs one for each site. On R1 you need to import both the RTs from R5 and R6, but you need to export just the one RT. For example:

R4 vrf 235
rd 230:5
Import 230:1
Export 230:5

R4 vrf 236
rd 230:6
Import 230:1
Export 230:6

R1 vrf 230
rd 230:1
Import 230:5
Import 230:6
Export 230:1

lrb

Indeed the first issue is caused by not having allowas-in on the PE-CE neighbour statements. Nice pickup.

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]
R1(config-router-af)#neighbor 10.0.12.2 allowas-in[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config-router-af)#neighbor 10.0.13.3 allowas-in[/FONT]
[/FONT][/COLOR]

And yes you are sort of on the right track with the RTs, I was trying to attach two export RTs on R1 and perform some filtering. In the end I did use only a single export RT to get it to work though but not exactly in the way you mentioned.

So now that the route is coming back into the VRF BGP table on R1 from R2 and R3, will it ever be considered the BGP bestpath and be sent to the spokes with the new RTs? It is infact the same VPNv4 route because the RD is the same for both ingress and egress right so the route through R4 (the other PE router) would surely always be better? Remember that the RD keeps a prefix unique within BGP, so we would need some way of getting around this to make this topology work.

I will provide some more configuration on R1 from the working solution.

[COLOR=#000000][FONT=Courier New]
R1(config)#no ip vrf 230[/FONT][/COLOR][COLOR=#000000][FONT=Courier New]
R1(config)#ip vrf 230[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]R1(config-vrf)#rd 230:1[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]R1(config-vrf)#route-target [B]import [/B]230:5[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]R1(config-vrf)#route-target [B]import [/B]230:6[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]R1(config-vrf)#exit[/FONT][/COLOR]

I left the exit on the configuration above so that you guys knew I wasn't adding extra config that just wasn't pasted

lrb

Okay so there were no takers but the answer to the problem is as follows.

You need to create two VRFs on PE router R1: One VRF forroutes being advertised from the spokes to the hub and another VRF for routes being advertised from the hub to the spoke, and also having unique RDs. In practice this means that one VRF is used for imporing routes from BGP to the hub', and another VRF is used for exporting routes from the hub to the spokes. You could do this using 2 links to each CE router (tunnel, 802.1Q, or physical) or just make one router able to receive routes and the other CE router able to send routes to the spokes, which is what i did.

The VRF configuration is as follows on PE router R1:

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config)#ip vrf 230
R1(config-vrf)#rd 230:1
R1(config-vrf)#route-target import 230:5
R1(config-vrf)#route-target import 230:6

[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config)#ip vrf 233
R1(config-vrf)#rd 230:3
R1(config-vrf)#route-target export 230:3[/FONT]
[/FONT][/COLOR]

Change the VRF membership of the link to CE router R3.

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config)#inter fa1/0[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config-if)#ip vrf forwarding 233[/FONT]
[FONT=Courier New]R1(config-if)#ip address 10.0.13.1 255.255.255.0[/FONT]
[/FONT][/COLOR]

Create a new IPv4 AF for VRF 233 (the VRF responsible for hub to spoke advertisement)

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config)#router bgp 14
R1(config-router)#no synchronization
R1(config-router-af)#address-family ipv4 vrf 230[/FONT][/FONT][/COLOR]
[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1(config-router-af)#no neighbor 10.0.13.3[/FONT]
[FONT=Courier New]
[/FONT]
[FONT=Courier New]R1(config-router)#address-family ipv4 vrf 233[/FONT]
[FONT=Courier New]R1(config-router-af)#neighbor 10.0.13.3 remote-as 230[/FONT]

[/FONT][/COLOR]

And then update the RTs on the VRFs on PE router R4 so that the hub's routes can be imported into the VRF.

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R4(config)#ip vrf 235
R4(config-vrf)#no route-target import 230:5
R4(config-vrf)#route-target import 230:3[/FONT]
[/FONT][/COLOR]
[FONT=Courier New]
R4(config)#ip vrf 236
R4(config-vrf)#no route-target import 230:6
R4(config-vrf)#route-target import 230:3[/FONT]

You can see the effects of these changes on PE router R1 in BGP.

[COLOR=#000000][FONT=Courier New]R1#show ip bgp vpnv4 vrf 230[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]BGP table version is 57, local router ID is 1.1.1.1[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]              r RIB-failure, S Stale[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]Origin codes: i - IGP, e - EGP, ? - incomplete[/FONT][/COLOR]

[COLOR=#000000][FONT=Courier New]   Network          Next Hop            Metric LocPrf Weight Path[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]Route Distinguisher: 230:1 (default for vrf 230)[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 2.2.2.2/32       10.0.12.2                0             0 230 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*>i5.5.5.5/32       4.4.4.4                  0    100      0 235 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*>i6.6.6.6/32       4.4.4.4                  0    100      0 236 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 10.0.23.0/24     10.0.12.2                0             0 230 i[/FONT][/COLOR]

[COLOR=#000000][FONT=Tahoma][FONT=Courier New]R1#show ip bgp vpnv4 vrf 233
BGP table version is 60, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 230:3 (default for vrf 233)
*> 2.2.2.2/32       10.0.13.3                              0 230 i
*> 3.3.3.3/32       10.0.13.3                0             0 230 i
[COLOR=#000000][FONT=Courier New]*> 5.5.5.5/32       [COLOR=#000000][FONT=Tahoma][FONT=Courier New]10.0.13.3[/FONT][/FONT][/COLOR]                0             0 230 14 235 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 6.6.6.6/32       [COLOR=#000000][FONT=Tahoma][FONT=Courier New]10.0.13.3[/FONT][/FONT][/COLOR]                0             0 [/FONT][/COLOR][COLOR=#000000][FONT=Tahoma][FONT=Courier New][COLOR=#000000][FONT=Courier New]230 14 236 i[/FONT][/COLOR][/FONT][/FONT][/COLOR][COLOR=#000000][FONT=Courier New][/FONT][/COLOR]
*> 10.0.23.0/24     10.0.13.3                0             0 230 i[/FONT]
[/FONT][/COLOR]

Because the routes are made unique by way of the unique RDs, BGP can successfully perform best path selection for routes going to the spokes from the hub, and then advertise these through BGP to PE router R4 (who will in turn import these routes into the VRFs).

Confirm control plane on CE router R5.

[COLOR=#000000][FONT=Courier New]R5#show ip bgp[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]BGP table version is 28, local router ID is 10.0.45.5[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]              r RIB-failure, S Stale[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]Origin codes: i - IGP, e - EGP, ? - incomplete[/FONT][/COLOR]

[COLOR=#000000][FONT=Courier New]   Network          Next Hop            Metric LocPrf Weight Path[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 2.2.2.2/32       10.0.45.4                              0 14 230 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 3.3.3.3/32       10.0.45.4                              0 14 230 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 5.5.5.5/32       0.0.0.0                  0         32768 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 6.6.6.6/32       10.0.45.4                              0 [/FONT][/COLOR][COLOR=#328712][FONT=Courier New]14[/FONT][/COLOR][COLOR=#000000][FONT=Courier New] 230 [/FONT][/COLOR][COLOR=#328712][FONT=Courier New]14[/FONT][/COLOR][COLOR=#000000][FONT=Courier New] 236 i[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]*> 10.0.23.0/24     10.0.45.4                              0 14 230 i[/FONT][/COLOR]

Confirm data plane on CE router R5 by tracerouting to the other spoke.

[COLOR=#000000][FONT=Courier New]R5#traceroute 6.6.6.6 source loop0 numeric[/FONT][/COLOR]

[COLOR=#000000][FONT=Courier New]Type escape sequence to abort.[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]Tracing the route to 6.6.6.6[/FONT][/COLOR]

[COLOR=#000000][FONT=Courier New]  1 10.0.45.4 164 msec 92 msec 104 msec[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]  2 10.0.13.1 [MPLS: Label 20 Exp 0] 384 msec 388 msec 380 msec[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]  3 10.0.13.3 268 msec 324 msec 428 msec[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]  4 10.0.23.2 [AS 230] 372 msec 316 msec 288 msec[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]  5 10.0.12.1 344 msec 256 msec 196 msec[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]  6 10.0.46.4 [MPLS: Label 20 Exp 0] 740 msec 772 msec 604 msec[/FONT][/COLOR]
[COLOR=#000000][FONT=Courier New]  7 10.0.46.6 776 msec 988 msec 700 msec[/FONT][/COLOR]

Hope that helps somebody in their MPLS studies for the lab

jamesp1983

Great tutorial! Thanks.

lrb

Thanks James! Hope the study is going well for you too

jamesp1983

lrb wrote: »

Thanks James! Hope the study is going well for you too

I'm still at it.