Add redundancy to ISP facing router

mzinzmzinz Member Posts: 328
I attached a diagram.

Previously had 1 router connecting between service provider (Ethernet hand-off to MPLS cloud) and network core. We are now adding a second router for redundancy. Using EIGRP internally to choose a path to MPLS network.

What is typically done for the connection to the MPLS network? Right now I just have a single /30 IP address from the provider.
_______LAB________
2x 2950
2x 3550
2x 2650XM
2x 3640
1x 2801

Comments

  • networker050184networker050184 Mod Posts: 11,962 Mod
    Unless the provider is going to give you another handoff your second edge router isn't really going to provide much redundancy for you in this scenario. You'd probably be better off just keeping it as a spare.
    An expert is a man who has made all the mistakes which can be made.
  • shodownshodown Member Posts: 2,271
    for your setup you would run HSRP between the 2 routers, but like it was said above you have a much larger chance of failure on the ISP site than 2 internal routers. If they give you another hand off you can run IP SLA(option 1) and you won't have to get too fancy with the configs.

    Option 2
    You can get 2 hand off's and run BGP with 1 provider, I actually set this up a few days back. You will have your own private AS and run IBGP between your 2 devices and bring in a default route and set local pref to take the preferred path out.


    Option 3
    If you decide to go to another provider for redundancy you will have to obtain your own AS and public IP space to run BGP with the providers.


    I may have made a mistake as I'm typing this out fast, but someone here will back me up in my thinking
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
  • mzinzmzinz Member Posts: 328
    shodown wrote: »
    for your setup you would run HSRP between the 2 routers, but like it was said above you have a much larger chance of failure on the ISP site than 2 internal routers. If they give you another hand off you can run IP SLA(option 1) and you won't have to get too fancy with the configs.

    Option 2
    You can get 2 hand off's and run BGP with 1 provider, I actually set this up a few days back. You will have your own private AS and run IBGP between your 2 devices and bring in a default route and set local pref to take the preferred path out.


    Option 3
    If you decide to go to another provider for redundancy you will have to obtain your own AS and public IP space to run BGP with the providers.


    I may have made a mistake as I'm typing this out fast, but someone here will back me up in my thinking

    Thanks for the replies.

    As for option 1, are you proposing that I use HSRP on the internal side only, and IP SLA to failover if the provider hand-off goes down? Or HSRP on both internal and external?
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • mzinzmzinz Member Posts: 328
    Also - why would it make more sense to run HSRP instead of letting a routing protocol (EIGRP) decide the path?

    I had chosen EIGRP so that I could keep the connection between the switch and routers /30
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • shodownshodown Member Posts: 2,271
    Ok I see what you are doing. You are running EIGRP on your core switch. For some reason I thought you were doing something else(reading is fundamental showdown,lol). So yes you can just run EIGRP there. You might not even have to run it below. in the other setup I made. My next question for you is why are you guys using 2 routers.
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
  • jason_lundejason_lunde Member Posts: 567
    mzinz wrote: »
    Also - why would it make more sense to run HSRP instead of letting a routing protocol (EIGRP) decide the path?

    I had chosen EIGRP so that I could keep the connection between the switch and routers /30

    Are you talking about best path to an outside network? or best path from the inside to the closest gateway? Those routers really have no metrics to hand eigrp for outside destinations since the are both connected to the same external provider (Assuming these are geographically in the same location). you are really looking at first hop redundancy from what I am looking at.
  • mzinzmzinz Member Posts: 328
    shodown wrote:
    Ok I see what you are doing. You are running EIGRP on your core switch. For some reason I thought you were doing something else(reading is fundamental showdown,lol). So yes you can just run EIGRP there. You might not even have to run it below. in the other setup I made. My next question for you is why are you guys using 2 routers.

    Haha - no worries :) The two router question is a great point. It would have made more sense to use switches since it's a Ethernet hand-off, but the hardware was already purchased when I entered the contract. Also, I was keeping the diagram simple, it is handling some other jobs too, (VPN, QoS).

    The purpose of having the second router is purely failover, in case the first one dies.
    Are you talking about best path to an outside network? or best path from the inside to the closest gateway? Those routers really have no metrics to hand eigrp for outside destinations since the are both connected to the same external provider (Assuming these are geographically in the same location). you are really looking at first hop redundancy from what I am looking at.

    Assuming I get a second hand-off from the provider, the paths to the MPLS cloud will be equal. The only reason I'm using EIGRP is in case one router goes down, we have a backup path via the second router. This may not be the best way to do this - so if not, then I'm all ears for suggestions. But it's not so much about choosing the best path (since they're the same), it's about having a backup route (feasible successor).
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • jason_lundejason_lunde Member Posts: 567
    Do you get any routes from your MPLS provider? What is your MPLS used for? i.e. inter-site connectivity, internet access, or both?
  • shodownshodown Member Posts: 2,271
    1 thing I'm going to add in my experience of doing this is that i have seen more circuits die than Redundant routers. I guess you already got the gear so might as well do something with it, unless you can get your partner to credit u for the gear and get something else you guys need. We do it all the time for customers.
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
  • networker050184networker050184 Mod Posts: 11,962 Mod
    I think you will just be adding more complexity without much gain by adding a second router there. If you have one ISP hand off then the second router really only provides redundancy for one cable (the one from the core to the existing router). Not worth it IMO.
    An expert is a man who has made all the mistakes which can be made.
  • jason_lundejason_lunde Member Posts: 567
    I think you will just be adding more complexity without much gain by adding a second router there. If you have one ISP hand off then the second router really only provides redundancy for one cable (the one from the core to the existing router). Not worth it IMO.

    This is where i was headed with this conversation as well.
  • shodownshodown Member Posts: 2,271
    I think you will just be adding more complexity without much gain by adding a second router there. If you have one ISP hand off then the second router really only provides redundancy for one cable (the one from the core to the existing router). Not worth it IMO.


    Thanks for making it simple. I was trying to say this couldn't quite type it out.
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
  • mzinzmzinz Member Posts: 328
    Thanks for the replies guys.

    We don't get routes from the MPLS provider. We have about 200 remote branches connected, all within the same contiguous class B subnet. (So we currently have a static route for something like 192.168.0.0/16, with each site having a /24 internal LAN).

    The reason they wanted a second router is because of the "what if it dies scenario". We would be completely down, as would all of our remote offices, for who knows how long. This way, even if something happens to the circuit, it isn't our fault.

    I do understand what you're saying with the complexity and risk/reward issues. What would be a more suitable substitute? Maybe a couple stacked 3560s? It probably would be possible to get a credit for the gear.

    I guess I thought that HSRP was for scenarios just like this? Would it just make more sense if we had a second provider, as opposed to two circuits from the same single provider?

    EDIT: Or, alternatively, we could have the provider connect straight to our 3750 stack core, and rule out an intermediary device completely. I suppose this entire post is more of a fundamental design question than anything.
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • shodownshodown Member Posts: 2,271
    if you have that many sites that are relying on you I would get a connection from 2 providers(make sure you get path diversity) and run BGP and have your own Private AS inside. That way you can always be seen by the remote sites. I can get more into detail when I get home as I have a few fires I'm putting out now at work.
    Currently Reading

    CUCM SRND 9x/10, UCCX SRND 10x, QOS SRND, SIP Trunking Guide, anything contact center related
  • networker050184networker050184 Mod Posts: 11,962 Mod
    mzinz wrote: »
    The reason they wanted a second router is because of the "what if it dies scenario". We would be completely down, as would all of our remote offices, for who knows how long. This way, even if something happens to the circuit, it isn't our fault.

    I'd just keep a hot spare which will do you just as much good. If the first router dies and it is the only one with a link to the ISP you are SOL anyway.
    mzinz wrote: »
    I do understand what you're saying with the complexity and risk/reward issues. What would be a more suitable substitute? Maybe a couple stacked 3560s? It probably would be possible to get a credit for the gear.

    That could work out as long as you don't need any features of the router.
    mzinz wrote: »
    I guess I thought that HSRP was for scenarios just like this?

    Not really. HSRP is suited to provide first hop redundancy for hosts. It allows you to not have to go changing the default gateway on devices when a router/switch dies. You really don't get anything from it when trying to do router to router redundancy. A routing protocol with fail over is what you would be looking for in this scenario as next hops are updated automatically anyway. It could help somewhat if you have static routes, but you want to avoid those if you can use a routing protocol instead.
    mzinz wrote: »
    Would it just make more sense if we had a second provider, as opposed to two circuits from the same single provider?

    That would be my choice. Unless you specifically get a special design, another circuit will most likely ride the same equipment in your market at some point and fail the same time the other would if there was a cable cut or something.
    mzinz wrote: »
    EDIT: Or, alternatively, we could have the provider connect straight to our 3750 stack core, and rule out an intermediary device completely. I suppose this entire post is more of a fundamental design question than anything.

    Again, if you don't need any router features (NAT, VPN etc.) this would work out pretty well for you. You mentioned VPN so thats probably not the best fit for you though. You could always hang the VPN router off the core and use it like a VPN on stick scenario if you really wanted to though.
    An expert is a man who has made all the mistakes which can be made.
  • mzinzmzinz Member Posts: 328
    shodown wrote: »
    if you have that many sites that are relying on you I would get a connection from 2 providers(make sure you get path diversity) and run BGP and have your own Private AS inside. That way you can always be seen by the remote sites. I can get more into detail when I get home as I have a few fires I'm putting out now at work.

    Thanks for your help - more info would be great whenever you have a chance.
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • mzinzmzinz Member Posts: 328
    Again, if you don't need any router features (NAT, VPN etc.) this would work out pretty well for you. You mentioned VPN so thats probably not the best fit for you though. You could always hang the VPN router off the core and use it like a VPN on stick scenario if you really wanted to though.

    The router does have other roles, but they aren't necesarily for the remote hosts. Doing something like VPN on a stick would work fine.

    I would like to have some device between the 3750 stack and the provider hand-off for traffic policies, though. Maybe having two 3560's in a stack, running EIGRP in case of failover, would be the best option. I can apply ACL's on the L3 interfaces from the provider.

    Functionally, it would be doing the same thing as the router, but I could probably get the 2 3560's for about the price of a single 3845.

    Edit: If possible, I would get another circuit from another provider - I'm just not sure that is possible, budget wise. So I'm going for the "next best" option, really.
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • hasitha257hasitha257 Member Posts: 25 ■□□□□□□□□□
    Originally Posted by mzinz
    Would it just make more sense if we had a second provider, as opposed to two circuits from the same single provider?

    Guys, aren't we talking about MPLS redundancy here? How is it possible to go with two ISPs for your MPLS redundancy?

    Even if you had a redundant link to another ISP , how are you routing the traffic over to the remote sites?

    I am just curious............appreciate any responses!!!
  • networker050184networker050184 Mod Posts: 11,962 Mod
    hasitha257 wrote: »
    Originally Posted by mzinz
    Would it just make more sense if we had a second provider, as opposed to two circuits from the same single provider?

    Guys, aren't we talking about MPLS redundancy here? How is it possible to go with two ISPs for your MPLS redundancy?

    Even if you had a redundant link to another ISP , how are you routing the traffic over to the remote sites?

    I am just curious............appreciate any responses!!!

    Good catch. While you can get MPLS redundancy with two providers, it is a more complex scenario that involves cooperation between carriers. They would need to have an inter-AS VPN with each other.
    An expert is a man who has made all the mistakes which can be made.
  • ConstantlyLearningConstantlyLearning Member Posts: 445
    mzinz wrote: »
    Edit: If possible, I would get another circuit from another provider - I'm just not sure that is possible, budget wise. So I'm going for the "next best" option, really.

    You should go with the solution that work's best. If that's getting a second provider, then plan that out and see if the budget allows for it.
    "There are 3 types of people in this world, those who can count and those who can't"
  • mzinzmzinz Member Posts: 328
    Good catch. While you can get MPLS redundancy with two providers, it is a more complex scenario that involves cooperation between carriers. They would need to have an inter-AS VPN with each other.

    Can you expand on this some, or provide a link if you have one? I don't have experience with that.
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • networker050184networker050184 Mod Posts: 11,962 Mod
    There are quite a few flavors (options) for an Inter-AS MPLS VPN, but here is some info on option AB.

    MPLS VPN - Inter-AS Option AB [Cisco IOS and NX-OS Software] - Cisco Systems

    Its nothing you would have to worry about as the customer. The second provider you get a circuit from would need to already have an arrangement with your primary provider for this to be set up though.

    We have a few set up with other providers, but honestly I'm not quite sure how popular these are so I have no clue what the chances are that your provider has something like this in place.

    Another option is a backup IPSEC tunnel or DMVPN over the internet.
    An expert is a man who has made all the mistakes which can be made.
  • mzinzmzinz Member Posts: 328
    There are quite a few flavors (options) for an Inter-AS MPLS VPN, but here is some info on option AB.

    MPLS VPN - Inter-AS Option AB [Cisco IOS and NX-OS Software] - Cisco Systems

    Its nothing you would have to worry about as the customer. The second provider you get a circuit from would need to already have an arrangement with your primary provider for this to be set up though.

    We have a few set up with other providers, but honestly I'm not quite sure how popular these are so I have no clue what the chances are that your provider has something like this in place.

    Another option is a backup IPSEC tunnel or DMVPN over the internet.

    Interesting. Thanks for the information. I think I'll call my provider and ask if it's something they would support.

    For the second option, are you suggesting that the second router has a connection to the internet, and we have DMVPN set up on all remote branches, and that it uses the VPN link only in cases of failover? (ie: Use VPN if MPLS unaccessible)
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • networker050184networker050184 Mod Posts: 11,962 Mod
    mzinz wrote: »
    For the second option, are you suggesting that the second router has a connection to the internet, and we have DMVPN set up on all remote branches, and that it uses the VPN link only in cases of failover? (ie: Use VPN if MPLS unaccessible)

    Yes, exactly. Depending on your network needs, something as simple as business DSL could provide a backup.
    An expert is a man who has made all the mistakes which can be made.
  • mzinzmzinz Member Posts: 328
    Yes, exactly. Depending on your network needs, something as simple as business DSL could provide a backup.

    This would be an excellent solution... unfortunately we don't have any internet connections at the remote branches - just the MPLS.
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • networker050184networker050184 Mod Posts: 11,962 Mod
    Your provider should be able to get you internet access over those same MPLS circuits without much trouble. I'm sure you'd have to pay some sort of MACD fee, but it could be minimal.

    DSL isn't overly expensive to get into your branches either. If they want redundancy its going to cost something besides just one extra router to sit there doing nothing :D
    An expert is a man who has made all the mistakes which can be made.
  • mzinzmzinz Member Posts: 328
    Your provider should be able to get you internet access over those same MPLS circuits without much trouble. I'm sure you'd have to pay some sort of MACD fee, but it could be minimal.

    DSL isn't overly expensive to get into your branches either. If they want redundancy its going to cost something besides just one extra router to sit there doing nothing :D

    Fair enough :)

    It seems like having a connection into the same MPLS cloud from a different provider would really be the best move, if possible. We don't need absolute redundancy at each and every site. Having a secondary router in case the main circuit goes down would be great though.
    _______LAB________
    2x 2950
    2x 3550
    2x 2650XM
    2x 3640
    1x 2801
  • sides14sides14 Member Posts: 113
    The greatest thing to remember about redundancy is that it needs to be truly isolated from the original path. Many providers will sell you a redundant link, but when you drill down to the detail (or have an outage), you find out pretty quick that the path is not redundant.

    Story about redundancy. Had to "redundant" GigE circuits from a provider. Wild fire caused both links to go down. After further investigation, the circuits were on two distinct sets of fibers, but the first and second sets were OPGW on the same 250KV line - not very redundant or protected.
Sign In or Register to comment.