Routing Architecture

sputnic68

I'm in the process of designing a routing architecture for our new data center and wanted to share my ideas with everyone. My goal is that everyone could benefit from me sharing these ideas and scenarios, and obtain everyone else's opinion on the design and implementation of the routing architecture. Let me know what everyone thinks! If it's a go then i'll start posting the diagram and explaining my situation.




Nick

keenon

holla at me aim, or msn

johnnynodough

Im game to see some of the magic happen, probably not much for input, but I would certainly like to see some of your developed solutions and reasonings, as to help me in my pursuit of CCNP goodness

Webmaster

sputnic68 wrote:

If it's a go then i'll start posting the diagram and explaining my situation.

Go, should be interesting. Feel free to mail me the diagram (webmaster 'at' techexams.net) and I'll put it on our server and add it to your post.

QUIX0TIC

That is so awesome of you to do that. That is extremely altruistic for any person to take the time and share what his/her networking environment would lool like or be developed. Plus, it may help you so others can add their 2 cents in to help you come up with other ideas that you might have missed.

Thanks

sputnic68

I sent off the routing diagram. As soon as it gets posted I'll explain what I'm trying to accomplish.

Webmaster

I added the diagram to your initial post above.

keenon

i think its a great design, alot of redundancy, you could collapse the server switch blocks down to the core 6509s to remove a level. using 48 port 10/100/1000 blades and if that doesn't offer enough ports. i would go to the 16 port fiber blades with the switch blocks... but these are just my thoughts. it also depends on the requirements that are set for scope of project

sputnic68

I'll post my routing design thoughts a little later this evening. I'm swamped right now laying out the server rack design.

sputnic68

The reason for breaking the access layer off from the distribution/core was because of flexibility, scalability and cost. We’re talking somewhere of 1600 to 2000 servers with 3 connections each. Using the access blades on the 6509 would get quite expensive and be less scalable.

For routing, basically what I need to do is separate the two networks (TMP & ORAN) at layer 3 internally in the 6509. I want to run OSPF internally in the data center and redistribute the other routing protocols into it. The networks need to be separated at layer 3 to adhere to the policies of our business. I will be creating the IP addressing scheme for the network also. The plan is to allocate certain subnets for the ORAN network and certain subnets for the TMP network. I will create VLANs that will aid in the separation too. Such as, vlan 5 is routing with TMP and vlan 10 is routing with ORAN. The goal is that nodes associated with ORAN can route out the ORAN for external traffic and can route to other nodes associated within the data center that are part of the ORAN network. The nodes associated with TMP will be routing through the firewall and then out TMP for external traffic and can route to other nodes associated within the data center that are part of the TMP network. I don’t want the ORAN nodes inside the data center to be able to communicate to the TMP nodes and vice versa for TMP. Also, in the 6509 I don’t want ORAN to know any routes associated with TMP and vice versa with TMP. So, basically keeping these networks separate internally in the 6509.

Routing to external for TMP will be OSPF internally to firewall. The firewall will be static routs and TMP will be EIGRP 25. Routing external for ORAN will be OSPF internally, then redistributed into EIGRP 61. EIGRP 25 and 61 cannot communicate to each other and cannot be routed to each other (separate routed networks). Both EIGRP AS’s and OSPF as will be separate within the 6509 with the exception of redistribution.

I hope this makes sense to everyone and if it doesn’t just say so and I’ll try to explain it in more depth. Also, if anyone has any input if there needs to be changes, or if you have input on the actual implementation following the design and policies, please let me know.

keenon

i see, that is alot of servers so having that seperate area is necessary. trusted and untrusted networks are always fun. don't mind me asking but it sounds like from all you have described you have a hippa compliance to meet.

sputnic68

Nope! No hippa. Just some political boundaries that I have to work around.

sputnic68

Anyone's input would be greatly appreciated!!!

johnnynodough

Based on my current skillset (CCNA), all I can really say is it looks really cool

Getting down to the servers, there I could chime in, thats my current forte'

I have decided to get my CCDA first instead of the CCNP, since I can keep the CCNA and CCDA current as long as I take a 642 exam once every three years

darkuser

well , you don't reveal too much about your layer 2 design other than showing the physical interconnections.
I'd implement rapid spanning tree / mst over pvst when given the chance.
what about security , vlans, dmz's ... etc ?
internal or perimiter security ?
wan connectivity ?
my other question is why choose to go multiprotocol if you have a choice ? unless you like redistribution ?
I'd go with either eigrp or ospf.

i'd have a look at the safe whitepaper to validate or make changes to the design.

rossonieri#1

hello,
i've spotted a classic network here.
2 6500s to provide failover redundancy - 2 FWs, but are those 2FW doing hot standby also?
the newer network design use the term CLEAN and DIRTY side - with the help those 2 FW in the middle of 2(existing switches) + 2 more 6500s.
like this maybe :
TMP ORAN
| |
|

---

2 FWs/failover

---

|
| |
6500 6500
|

---

2FWs/failover

---

|
| |
6500 6500
|

---

|
| servers+servers+servers |

i think those little switches you should change with couple of bigger switch to provide eficiency of administration/installation - if you need... ; )
and are those 60 racks of servers run GB NIC?

sputnic68

I'm left with no choice but to incorporate EIGRP AS25 and AS61. If I where to use an EIGRP AS26 for the data center, I would have to still do some redistribution. I like OSPF for many reasons and for a data center network environment you can't go wrong with OSPF. Ultimately I want the best routing architecture that will be easy to configure and manage, but also provide to best performance. If anyone has any ideas that would prove me wrong please step up. I'm also discussing this architecture and implementation with a few CCIE's. I'll keep everyone updated.

-Nick

rossonieri#1

sputnic68 wrote:

I'm left with no choice but to incorporate EIGRP AS25 and AS61. If I where to use an EIGRP AS26 for the data center, I would have to still do some redistribution. I like OSPF for many reasons and for a data center network environment you can't go wrong with OSPF. Ultimately I want the best routing architecture that will be easy to configure and manage, but also provide to best performance. If anyone has any ideas that would prove me wrong please step up. I'm also discussing this architecture and implementation with a few CCIE's. I'll keep everyone updated.

-Nick

well, if your data center have couple of non-cisco equipment : there are only 2 things left for routing -> using standard routing prots or static routing. does your future network using full intranet (like frame-relay) or also including the vpn??

sputnic68

This will be all Cisco Equipment!

keenon

if the network is going to be all cisco run with eigrp...i also think that for that many servers i would have to agree with the other post of using a bigger distribution switches 3750s are good but if possible put something between them and the 6500 like a pair of 4500s then uplink to the 6500s..

sputnic68

We're not talking about a huge amount of traffic. The most I've seen is maybe 100mbs on an interface from the data center that is migrating to this one.

keenon

is that from the access port or the uplink port?

sputnic68

Both uplink and access have not exceeded 100mbs

Yankee

darkuser wrote:

my other question is why choose to go multiprotocol if you have a choice ? unless you like redistribution ?
I'd go with either eigrp or ospf.

I have to agree with Darthee. This is a relatively small network that based on limited info seems to be overly complicated by 2 ASs and a second routing protocol. Perhaps I missed the needs for this or maybe they weren't stated but I would be making every effort to avoid this if possible.

Yankee

EdTheLad

I think in order for this thread to progress correctly Sputnic86 needs to add a
list of constraints.Maybe even 2 lists, one soft list which is kind of flexible and another hard list where theres no flexibility.
I think most people here can give a list of best practices as in 1 AS but sometimes due to higher powers these arnt always feasible.

sputnic68

I'll make some list up tonight to help everyone out.

sputnic68

Here are the restrictions:

I cannot allow routes learned and routing to take place between AS25(TMP) and AS61(ORAN).

There is some flexibility with what routing protocol I can use internally in the data center.

Routing internal in the data center is restricted by the two AS's (devices associated with AS61 cannot be routed within the data center to devices in AS25). If they must communicat they have to be routed to the internet and then back again. Kindof a stupid setup but there is a lot of politics involved with this. The reason for using the two 6509's for all the routing is because of money constraints.

I can only secure TMP with the ASA's because the ORAN is a 10gbs optical ring and would crash the ASA with the amount of traffic.

I have to use EIGRP AS25 for TMP and EIGRP AS61 for ORAN. The only requirement is that I separate those internally in the 6509. There are political boundaries associated with each AS.

I think that I've covered all the constraints.

Give me your input on how I should implement the routing? EIGRP, OSPF? How should I separate the AS's? Policy routing, distribute lists? How should I implement the redistribution. I'd like to work up a higher level of routing and then get into the implementation of it so everyone could benefit from the routing architecture.

-Nick

Yankee

If the two networks (groups) don't trust each other, never talk to each other and don't share hardware then separate them with a firewall. In that case there is no harm in different ASs as there would be no redistribution.
I believe where possible firewalls should be used rather then routing tricks as it is much more secure design.
If untrusted traffic must transit the same router you can look at VRFs but that gets pretty complicated. We are beginnig to use that process and its a pain changing my old habits...

Yankee

keenon

definitely..firewall

sputnic68

I am using a firewall for the TMP network but I'm unable to use one for the ORAN network because of the amount of traffic that traverses the 10gbs link would crash any firewall. I can still use IOS firewall for that network though. But the issue still comes down to that I will have two AS's in one 6509 so there will have to be some routing tricks done.

Does anyone see this different?

keenon

have yoy looked for any of the high end firewalls? does it have to be cisco or can it be another vendor