Best design for hub & spoke topology?

TesseracT · June 2011

Hey guys I'm not sure if this is the right forum for this but it is a design question + I'm studying for the DA and DP as we speak.

We're using a basic hub & spoke topology / star topology here at work and I'm wondering if there's any way to improve it.

At the core is an ASA which all the satellite offices connect to (around 10 remote offices). These offices all connect back via an IPSEC site to site VPN and in most cases the satellite offices can communicate with eachother. This is all done via static routes

Is there a better way of doing this? MPLS would be nice but way too expensive. Also, these offices are literally all around the globe each in a different country.

DMVPN would be nice but not available for the ASA as far as I'm aware.

Can an IGP be used here in the place of static routes? Considering the topology and the fact that site to site tunnels are used here I'm not really convinced that it's an option.

What would you guys do in this situation? Everything works ok at the moment but wondering how scalable this solution is if we hit some fast growth...

Forsaken_GA · June 2011

Unless there's actually a problem you need to solve, leave it be.

You could use an IGP to handle your routing needs, but there's a tradeoff - circuits which connect to the public internet can be a fickle *****, and if one of them starts flapping, it can screw with the routing performance of the entire company. With 10 sites, I wouldn't bother, that's still a small enough amount that administrating static routes is not a huge burden.

What problems are you trying to solve? Is there a specific performance issue that can be attributed to the network design? Are your traffic levels high enough that aggregating back to the central site is causing packet loss?

Basically, don't ever go mucking about with a network that is performing what it's asked to do just for the sake of doing it, or 'doing it better.' The butterfly effect is quite alive and well in the network world, and I've made changes that invoked the Law of Unintended Consequences far more often than I'm willing to admit.

If there's a problem that needs immediately solving, then you should be looking for options, but if you're just trying to reinvent the wheel, don't waste your time. The best strategy is to generally look at what you need, incorporate that into the design goals, and then slowly migrate portions of the network to the design after you've tested it, validated it, broken the lab/pilot on purpose to prove you can fix it, and sacrificed a goat and/or virgin to Nidhoggr, the Net Serpent to prevent him from devouring your packets.

Any deviation from that path leads to you either being a big hero, or having a Resume Generating Event (talk to Vegas for the odds on both)

Turgon · June 2011

TesseracT wrote: »

Hey guys I'm not sure if this is the right forum for this but it is a design question + I'm studying for the DA and DP as we speak.

We're using a basic hub & spoke topology / star topology here at work and I'm wondering if there's any way to improve it.

At the core is an ASA which all the satellite offices connect to (around 10 remote offices). These offices all connect back via an IPSEC site to site VPN and in most cases the satellite offices can communicate with eachother. This is all done via static routes

Is there a better way of doing this? MPLS would be nice but way too expensive. Also, these offices are literally all around the globe each in a different country.

DMVPN would be nice but not available for the ASA as far as I'm aware.

Can an IGP be used here in the place of static routes? Considering the topology and the fact that site to site tunnels are used here I'm not really convinced that it's an option.

What would you guys do in this situation? Everything works ok at the moment but wondering how scalable this solution is if we hit some fast growth...

You could do some feasibility work and consider the options. But as the previous poster said I would be inclined to let it be. If you plan on running routing protocols look to do that over private circuits as opposed to IPSec tunnels over the internet. The next time you have a satellite site to light up you could try a routing protocol as opposed to static routes to see if it flies. Sticks static routes in as a fallback with a higher admin distance.

TesseracT · June 2011

Thanks for bringing me back to reality guys

I think I'm reading too much design stuff at the moment so I feel the need to over-engineer everything. The network is running fine as well, I just thought there was maybe a way to clean up the config on the ASA. I guess it's still pretty tidy though, compared to how many site-to-site tunnels you can create on these things. I guess just after doing the CCNP and studying for the CCDP I wan't to put my knowledge to the test... I just guess that my production network isn't the place for it

must. resist. urge. to. break. network.

Forsaken_GA · June 2011

TesseracT wrote: »

Thanks for bringing me back to reality guys

I think I'm reading too much design stuff at the moment so I feel the need to over-engineer everything. The network is running fine as well, I just thought there was maybe a way to clean up the config on the ASA. I guess it's still pretty tidy though, compared to how many site-to-site tunnels you can create on these things. I guess just after doing the CCNP and studying for the CCDP I wan't to put my knowledge to the test... I just guess that my production network isn't the place for it

must. resist. urge. to. break. network.

Understand completely. I have to rein myself in when it comes to play with new tech, or new concepts as well. I got into some heated arguments because I wanted to implement 802.1x network wide. I thought it was a good idea, and I wanted to see it actually work (that was not my argument of course, security, authentication, the lifeblood of the company, blah blah blah. I thought it was cool. I wanted to do it. My entire motivation, regadless of what I said to those above my paygrade).

Well when I was told no, I decided to go ahead and implement it in my own lab.

After the pain in the ass of making RADIUS work with LDAP (the x509 certs are, comparatively, painless), I'm rather glad they told me no. It's not a trivial project, and has far reaching consequences.

That's the one thing about reading all these books on tech. They never tell you about the real world side effects of all this crap, save for Network Warrior. A few years of experience, and you start looking at the texts in a different light - as you're reading, instead of thinking 'wow, that sounds cool!', you start thinking 'what kind of crack are they on, if I did this, it'd break this, this and this' or 'great, more cisco marketing drivel, skip!'

Turgon · June 2011

Forsaken_GA wrote: »

A few years of experience, and you start looking at the texts in a different light - as you're reading, instead of thinking 'wow, that sounds cool!', you start thinking 'what kind of crack are they on, if I did this, it'd break this, this and this' or 'great, more cisco marketing drivel, skip!'

Correct. Lack of recent operations experience by writers is often prevalent and sometimes there is overuse of the crack pipe. Just because we can do something doesn't always mean we should do it.

TesseracT · June 2011

yeah, some of the design 'best practices' have me scratching my head a bit. I'm not sure if it was in the CCDA or CCDP guide but I remember reading that you should confine 1 VLAN per access layer switch rather than spanning multiple VLANs across them... what? How the hell are you meant to separate voice and data traffic then? I can see it might be useful in some rare cases but nearly all the networks I've worked on need multiple VLANs accross the access layer.

Anyway I had a fun little problem here at work today. For no apparant reason we started getting calls from irate users that weren't able to log in via VPN (ipsec or ssl). At first I thought it was our RSA server but doing a bit of testing showed me that the authentication was successful - just that when a user logged in they would be logged straight back out immediately.

After scratching the noggin for a bit I stumbled onto a bug specific to our software version 8.2.1. When the CPU gets pegged it will completely screw up the VPNs. I checked the memory and found it had stayed consistenly at 80% the whole day, no variation at all which caused a few alarm bells to ring. Cisco's solution - install more memory or reboot the device.

Fair enough, rebooted the ASA and everything is back to normal. I immediately disabled netflow to give us some breathing room until some more ram can be sourced. Fun times...

Looks like Cisco even subscribe to the old 'have you tried turning it off and on again?'

Forsaken_GA · June 2011

TesseracT wrote: »

yeah, some of the design 'best practices' have me scratching my head a bit. I'm not sure if it was in the CCDA or CCDP guide but I remember reading that you should confine 1 VLAN per access layer switch rather than spanning multiple VLANs across them... what? How the hell are you meant to separate voice and data traffic then? I can see it might be useful in some rare cases but nearly all the networks I've worked on need multiple VLANs accross the access layer.

Take the CCDA/DP material with a grain of salt. I am not a fan of the Design track (and I *am* a CCDP), because they do far too much cheerleading for Cisco solutions. As I was reading through the material, I was sitting and thinking of other solutions to the problems they were presenting. Non-cisco solutions. Papa John does not always know best.

In particular, I had serious issues with their firewall and security solutions. Most of their security solutions are EOL, and given the transitory nature of Cisco security products, you have to be absolutely brain dead or getting one hell of a kick back to try and sell your management on it. I'm also not a fan of their load balancing solutions. I can do much better with F5's, or even a linux box.

The design material has some good information in it, but learning how to seperate it from the marketing bullshit is a challenge. For anything outside of routing and switching, always always ALWAYS look for solutions that will cover your needs that may not be imprinted with the Cisco logo.

Forsaken_GA · June 2011

Oh, and as far as the VLAN thing goes....

These days, networks contain alot of broadcast traffic. If you extend a vlan to every switch in the company, that's alot of extra traffic. If possible, you should consolidate a vlan to as few switches as possible. The other problem is that, since spanning tree recovergence's happen on a per vlan basis these days (lord I HOPE you're not running just CST), a vlan spanned across every single switch in the network means that every single switch experiences a reconvergence. Theoretically, this only occurs on the vlans themselves, but i've seen some interesting side effects of reconvergences interfering with other vlans on the switch.

As well, if you confine vlans to one switch, that means you can confine subnets to one switch. That means you can take advantage of those lovely layer 3 features and route your traffic instead of switch it (hint, if you design that way, you can use both of your uplinks in a redudant switch setup).

However, your design is going to be dictated by your needs. Some applications require layer 2 adjacency (hello virtualization trend!), so layer 3 seperation isn't always the best way to get things done.

Turgon · June 2011

Forsaken_GA wrote: »

Take the CCDA/DP material with a grain of salt. I am not a fan of the Design track (and I *am* a CCDP), because they do far too much cheerleading for Cisco solutions. As I was reading through the material, I was sitting and thinking of other solutions to the problems they were presenting. Non-cisco solutions. Papa John does not always know best.

In particular, I had serious issues with their firewall and security solutions. Most of their security solutions are EOL, and given the transitory nature of Cisco security products, you have to be absolutely brain dead or getting one hell of a kick back to try and sell your management on it. I'm also not a fan of their load balancing solutions. I can do much better with F5's, or even a linux box.

The design material has some good information in it, but learning how to seperate it from the marketing bullshit is a challenge. For anything outside of routing and switching, always always ALWAYS look for solutions that will cover your needs that may not be imprinted with the Cisco logo.

There are sometimes non commercial reasons for staying with Cisco security solutions. Moving from an FWSM to an ASA solution can make a lot of sense as opposed to trying Checkpoint for example, as you leverage years of operations experience with your existing security products. All the vendor tracks suffer from fanboy syndrome in my opinion. It was ever so back in the late nineties when I studied for my MCSE and CNE. Rarely does a study track offer a fair taxonomy of options across vendors so it's important to be open minded as well as critical. Sometimes something *else* is the correct decision. At the same time while usually there are *better* devices out there for individual tasks, the cost of being mixed are not insignificant so sometimes the correct decision is to invest in something that works that may not be optimal but allows you to keep a more or less vanilla portfolio of products in production. This helps with standardization and leverage with suppliers. Again these are all things that you learn by actually doing proper consultancy and design, and it's not something the books can teach you.

shodown · June 2011

Forsaken_GA wrote: »

Understand completely. I have to rein myself in when it comes to play with new tech, or new concepts as well. I got into some heated arguments because I wanted to implement 802.1x network wide. I thought it was a good idea, and I wanted to see it actually work (that was not my argument of course, security, authentication, the lifeblood of the company, blah blah blah. I thought it was cool. I wanted to do it. My entire motivation, regadless of what I said to those above my paygrade).

Well when I was told no, I decided to go ahead and implement it in my own lab.

After the pain in the ass of making RADIUS work with LDAP (the x509 certs are, comparatively, painless), I'm rather glad they told me no. It's not a trivial project, and has far reaching consequences.

That's the one thing about reading all these books on tech. They never tell you about the real world side effects of all this crap, save for Network Warrior. A few years of experience, and you start looking at the texts in a different light - as you're reading, instead of thinking 'wow, that sounds cool!', you start thinking 'what kind of crack are they on, if I did this, it'd break this, this and this' or 'great, more cisco marketing drivel, skip!'

i would have wanted to strangle you. A site I was working a VOIP install, they wanted to implement it for 2500 users they maybe had 2 cisco guys in the entire building on top of supporting 2 networks, with wireless for both and around 5 remote sites (1 of them 500) users some security person got the idea to do this. As a contractor doing voip I even had to get involved in the push back. She eventually backed down, but she had strong support for a while, but that many changes on the switches, plus all moving around that is normally done there would have made that place a nightmare.

Forsaken_GA · June 2011

shodown wrote: »

i would have wanted to strangle you. A site I was working a VOIP install, they wanted to implement it for 2500 users they maybe had cisco guys in the entire building on top of supporting 2 networks, with wireless for both and around 5 remote sites (1 of them 500) users some security person got the idea to do this. As a contractor doing voip I even had to get involved in the push back. She eventually backed down, but she had strong support for a while, but that many changes on the switches, plus all moving around that is normally done there would have made that place a nightmare.

Yup. If I ever have the opportunity to build out from scratch, I'll implement it then. It's alot easier to get done if you design it into the base specification. And I'll push for it whenever we add any new sections to the network, or any sections which get migrated to a new design.

There's no way in hell I'd go to the trouble of integrating it into an existing design.

This is also why people who are gainfully employed, not just those who are hoping to become so, should employ their own labs. It's much better to learn the pitfalls of an implementation on something that you don't care about.

Right now, I'm converting my home lab to native ipv6, and I'm going to put an ipv6 proxy/gateway on the border to handle anything that can't be sent natively (converting the network to ipv6 is easy. Getting the gateway working properly is proving to be a much bigger pain in the ass than I'd anticipated, so right now, it's still dual stack except for my test section. There are several companies I want to strangle for leaving bad AAAA records in their DNS zones, or worse, not providing glue records).

I'm doing this, again, because I think it's cool, but this project is a little more pragmatic - I think this is the path most companies are going to take for the ipv6 migration, and I think there's going to be a crapload of money to be made when folks do decide to pull the trigger. So I'm trying to get ahead of the game.

Panzer919 · June 2011

When I am looking at a design, after I've drawn it up I usually go back and make sure it follows the KISS mentality - Keep it simple stupid. This usually makes me think about how if A goes berserk what is it going to effect. If it is going to be a beast to implement and it does not NEED to be there then its not going there.

Example: We have an admin that wanted to use our 20Mb internet circuit to ALSO run our backups over in addition to our 40Mb direct connection at night. I said I would look into it and found time based ACLs and Policy routing. After consulting with someone he said that while it is doable, when it goes wrong, it goes VERY wrong. So I told them its not going to be done and why. He didn't like it but it wouldn't be his rear end getting reamed when it happens.

Best design for hub & spoke topology?

Comments