Network Design Questions

Eildor

For my final year project I will be designing and implementing a network for a University with over 1000 IP-enabled devices.
I have decided that I would like to implement a routed access layer; this is something which I haven't done before so I'm not too sure about a few things, namely:

1. Subnetting. How should this be done? For example, if I have 800 student computers spread out between 17 access switches (800 student computers / 48 ports per switch) should I simply assign each access switch "block" a /26 subnet?

2. VLANs. Should each access switch "block" have its own VLAN assignment? Meaning, for 800 student computers over 17 access switches there will be 17 different VLANs... if so, how in the real world would the VLANs be named in order to ensure ease of manageability? I'm thinking the most logical naming convention would be to simply name the VLAN after the switch, followed by group, for example, SW1-STUDENT, SW2-STUDENT and so on.

3. STP, VTP, Trunking... am I right in saying that there is no real use for these technologies in a routed design?

4. FHRP. I want to make use of a FHRP, namely VRRP. I guess the only place I can implement a FHRP now is for hosts or servers connecting to multiple L3 access switches for redundancy purposes?

5. For a network this size is there a need to use a three layer model and make use of blocks? I was thinking of simply having 2 distribution switches connecting to each of the 24 or so access switches via Gigabit Ethernet. That would give each access switch (consisting of 48 hosts) 2 Gigabit uplinks. Is this design valid? Is there any reason why 2 distribution switches couldn't handle 1152 hosts (CPU usage, address tables etc.)?

6. OSPF. Is there a need to divide the network into separate areas? I'm not sure what the limit is in regards to OSPF neighbours per area.

7. What would be the purpose of interconnecting the 2 distribution switches? Bearing in mind all access switches are connecting to each of the distribution switches, and will load balance between them.

Many thanks.


Kind regards,

Eildor

SteveO86

If you are running a L3 Access Layer you shouldn't need an FHRP, just dual home the access to the distributtion and lower the IGP timers for sub-second convergence.

Why stick with 48 port switches, when you have that many users you might want to look into 4500/6500 family with 48port modules and other special modules like FWSM

There are advantages to interconnecting the distributation over a L3 link, but I'll let you research that one.

Check the Cisco Design Zone, it's got great resources. Maybe grab some of the following books:

Optimal Routing Design
Building Resilent IP Networks
Top Down Network Design 3rd Edition
CCDA/CCDP material

Eildor

If you are running a L3 Access Layer you shouldn't need an FHRP, just dual home the access to the distributtion and lower the IGP timers for sub-second convergence.

Presumably I would need a FHRP if I were to dual home hosts or servers to access switches, right?

Why stick with 48 port switches, when you have that many users you might want to look into 4500/6500 family with 48port modules and other special modules like FWSM

I think doing that might make the network look too simple (remember it's a final year project, needs to at least look a bit complicated)... plus I'm not familiar with all of the different 4500/6500 modules.

There are advantages to interconnecting the distributation over a L3 link, but I'll let you research that one.

I have been looking into it, however I haven't found an answer as of yet; at least not one which I can understand.

Thanks.

m3zilla

Final year project as in this is a school assignment? Either way, you should be striving for the best design, not what's most complicated.

Presumably I would need a FHRP if I were to dual home hosts or servers to access switches, right?

If you're going to do that, why not just stack the switches and configure an etherchannel between the the switches and servers/hosts? I don't think I've ever seen a NIC team on an end user machine...sounds like a waste of ports, not to mention most machine plugs into a phone, and not directly to a switch.

Eildor

m3zilla wrote: »

Final year project as in this is a school assignment? Either way, you should be striving for the best design, not what's most complicated.



If you're going to do that, why not just stack the switches and configure an etherchannel between the the switches and servers/hosts? I don't think I've ever seen a NIC team on an end user machine...sounds like a waste of ports, not to mention most machine plugs into a phone, and not directly to a switch.

I wouldn't call it an assignment, it's a project on whatever I choose; therefore I'm expected to carry out research, ask questions, etc. Right now I am trying to create the proposal for the project, but I can't do that until I know what it is exactly I'm going to be implementing.

I thought it was common practice to dual home critical servers to access switches, how else would you achieve redundancy? I'm not going to dual home the end hosts, I was just using it as an example.

m3zilla

Yes, but you were talking about end users machine, not just servers. I see no need to have 2 connections coming from Bob's or Susan's PC.

Your server will typically reside in a separate block on the network, and not with the campus users.

SteveO86

Take from me, a complicated design is not always the best. A scalable design however is. This is your big project for the class you want to take the best of what learned and combine it to make a picture perfect design. Complexity leads to a tough troubleshooting process, which you might be graded on. Management of the network also needs to be considered when designing a network.

Running 4500/6500s and multiple OSPF areas with summarization. (Maybe NSSA's for the access and server core modules) is a simple yet complex design, you need to account for IP addressing, which can be complex in itself.

Dual homeing your clients to different blades within the chassis relying on the server OS usually has both NICs on the same subnet/VLAN so you don't need the FHRP. Or Check out 6500s running VSS for chassis redundancy, very expensive design.

Campus Network Design Fundamentals is also a great book by Diane Teare (She did some CCDA/CCDP books) I can't beleive I forgot about that one. I'd also consider it a good primer for the CCDP.

it_consultant

Consider why you would do certain things. For example, why subnet at all? One large subnet of 1000 devices will not create nearly enough ARP traffic to have a discernible impact on performance. The trend, nowadays, is to flatten networks where possible. Maybe your access switches are physically separated, like in a campus environment. You probably don't want to have ARP traffic from building one to building to another. In some cases, like mine, the remote site is 10KM away but it is a flat link. It is flat because I need my virtual machines to boot on the same network in a failover situation.

There are lots of things to consider. For example, VLANs, why bother? Perhaps you want to apply different priorities to certain network traffic than others. Or, you need different ACLs for different people, where those people might be situated on the same access switch. The use of the network should drive our design.

One slight quibble, your last bullet point, switches only load balance if you have them in a LAG group, and that load balance only occurs on that LAG. In order to actually load balance the traffic you will need to run a TRILL (Transparent interconnection of lots of links) or SPB (shortest path bridging) style switch. This is actually a good idea for you, since this is a class project and you are being tested on your ability to design a network. We would use TRILL in a case where we might anticipate running a converged data and storage network, commonly referred to as FCOE or FCIP.

Eildor

SteveO86 wrote: »

Take from me, a complicated design is not always the best. A scalable design however is. This is your big project for the class you want to take the best of what learned and combine it to make a picture perfect design. Complexity leads to a tough troubleshooting process, which you might be graded on. Management of the network also needs to be considered when designing a network.

Running 4500/6500s and multiple OSPF areas with summarization. (Maybe NSSA's for the access and server core modules) is a simple yet complex design, you need to account for IP addressing, which can be complex in itself.

Dual homeing your clients to different blades within the chassis relying on the server OS usually has both NICs on the same subnet/VLAN so you don't need the FHRP. Or Check out 6500s running VSS for chassis redundancy, very expensive design.

Campus Network Design Fundamentals is also a great book by Diane Teare (She did some CCDA/CCDP books) I can't beleive I forgot about that one. I'd also consider it a good primer for the CCDP.

I appreciate what you are saying, but I don't think it would be appreciated. Even if I, for example, were trying to be clever and use /31 addresses between access to distribution links, I wouldn't be confident that the person marking the paper would know whether or not the design is valid or not.

I have had a look at Campus Network Design Fundamentals as well as Top Down Network Design and I can't seem to find any information regarding routed access designs.

Eildor

it_consultant wrote: »

One slight quibble, your last bullet point, switches only load balance if you have them in a LAG group, and that load balance only occurs on that LAG. In order to actually load balance the traffic you will need to run a TRILL (Transparent interconnection of lots of links) or SPB (shortest path bridging) style switch. This is actually a good idea for you, since this is a class project and you are being tested on your ability to design a network. We would use TRILL in a case where we might anticipate running a converged data and storage network, commonly referred to as FCOE or FCIP.

Why would the switches not load balance if they are enabled for routing and connect to the distribution layer via 2 equal cost L3 links? Please explain.

it_consultant

If you have two switches and you want a redundant and load balanced connection, you use a LAG to plug the switches together. Having each switch a separate routing domain is way too complicated and will be slower than bridging your switches.

A LAG will turn 2 1GB links into 1 2GB link with failover protection. An equal cost route will round robin, but this does not bond the connections. Your devices will only get 1GB through that link.

If there is any one thing I have learned in networking it is this, switch when you can, route when you have too.

Trifidw

it_consultant wrote: »

A LAG will turn 2 1GB links into 1 2GB link with failover protection. An equal cost route will round robin, but this does not bond the connections. Your devices will only get 1GB through that link.

If there is any one thing I have learned in networking it is this, switch when you can, route when you have too.

Remember a etherchannel will send traffic down a single switchport based on its predefined criteria (mac/ip address/port) so you still do only have 2x 1GB ports.

That saying is very old and doesn't fully apply to todays networks with CEF etc. L3 to the edge is a valid design.

DevilWAH

it_consultant wrote: »

The use of the network should drive our design.

I so agree, so often I hear people talk about there net work like its out of a text book. I don't believe there is such a thing as right and wrong ways to design a network. But there are defiantly good and bad ways, it gets interesting when whats good for one network is bad for another.

While I agree 1000 devices on a sub net is not an proformance issue, it can be a management issue. I don't as a rule run vlans between buildings or floor. Really so I can pin down the location of IP address, and makes setting up network monitoring more straight forward. The other reason I would subnet is for data control (like security or qos). Once you have a subnet it makes controlling the data easier, filtering, routing, ACL's and such.

Every configuration on a network should be there for a reason, the argument "because it the standard" or "that's how it says to do it " is not a good reason. The questions you need to ask are. Is it secure? Does it do what I need it to do? Is it the most efficient way to do it? And I think that is the order in which you need to ask the questions. There is always a better way to do things, like with any thing technology moves on and ideas change. You can spend a lot of time worrying about "is this the right way to do it" rather than worrying about "is this doing the right thing for what I require".

it_consultant

Trifidw wrote: »

Remember a etherchannel will send traffic down a single switchport based on its predefined criteria (mac/ip address/port) so you still do only have 2x 1GB ports.

That saying is very old and doesn't fully apply to todays networks with CEF etc. L3 to the edge is a valid design.

Your right, depending on the type of LAG you set up. This is definitely true of the hash method of LAG but the port/address method is more of a round robin, so your one server or NIC will get better throughput on a rounded robin bond. I am talking about the 802.3 standard LAG, I am not sure if EtherChannel is standards compliant. Even the hash method of bonding is OK because even though your server is effectively maxed out at the speed of one of the links in the bond, server #2, 3, etc will also be able to get the same amount of throughput (at maximum) which effectively creates an aggregate bond.

it_consultant

DevilWAH wrote: »

I so agree, so often I hear people talk about there net work like its out of a text book. I don't believe there is such a thing as right and wrong ways to design a network. But there are defiantly good and bad ways, it gets interesting when whats good for one network is bad for another.

While I agree 1000 devices on a sub net is not an proformance issue, it can be a management issue. I don't as a rule run vlans between buildings or floor. Really so I can pin down the location of IP address, and makes setting up network monitoring more straight forward. The other reason I would subnet is for data control (like security or qos). Once you have a subnet it makes controlling the data easier, filtering, routing, ACL's and such.

Every configuration on a network should be there for a reason, the argument "because it the standard" or "that's how it says to do it " is not a good reason. The questions you need to ask are. Is it secure? Does it do what I need it to do? Is it the most efficient way to do it? And I think that is the order in which you need to ask the questions. There is always a better way to do things, like with any thing technology moves on and ideas change. You can spend a lot of time worrying about "is this the right way to do it" rather than worrying about "is this doing the right thing for what I require".

Agreed, in OP's environment, I would use VLAN as a management tool over a performance/redundancy tool. Depending on what the management needs are (which we don't know) the VLAN design should follow. As an example, a public WIFI or something. You would want to ACL those devices from the rest of your LAN.

networker050184

Since this is for a school project why not go with an internal VPLS design? Cool, somewhat complicated and plenty of stuff to mess around with. Sounds like the perfect school project to me!

it_consultant

networker050184 wrote: »

Since this is for a school project why not go with an internal VPLS design? Cool, somewhat complicated and plenty of stuff to mess around with. Sounds like the perfect school project to me!

Personally, I would go with an ethernet fabric. Complex, but simple at the same time. Mix the fabric with a virtualization and a storage over ethernet strategy - that should get the instructor's attention.

networker050184

Agreed it_consultant, go with some out of the box thinking and wow the professor! I'm sure if you go in there with some switches and a few VLANs it will look just like everyone's design.

Eildor

networker050184 wrote: »

Since this is for a school project why not go with an internal VPLS design? Cool, somewhat complicated and plenty of stuff to mess around with. Sounds like the perfect school project to me!

I don't know about VPLS, and since I have to write up a 15,000 word report I want to do something which I am somewhat confident with. I also have work to do for other modules, so I don't want to get into something I'm not familiar with.

Eildor

No one has really contradicted what I have said, so I am going to assume it is correct.

As there are 800 students, and because in a routed access layer design VLANs cannot span access switches, I need to have 17 /26 subnets (one subnet for each access switch).

it_consultant

It will work, we think it is a bad idea. If I were to come across this network on a consulting gig, I would immediately re-engineer it if it were in my scope. You are displaying good conceptual understanding of routing, but you are not understanding when not to route and why we disagree with the premise that each access switch should have its own VLAN; that is as clumsy as home running each "VLAN" into an open port on a firewall (seen it done) from a daisy chain of access switches below.

Eildor

it_consultant wrote: »

It will work, we think it is a bad idea. If I were to come across this network on a consulting gig, I would immediately re-engineer it if it were in my scope. You are displaying good conceptual understanding of routing, but you are not understanding when not to route and why we disagree with the premise that each access switch should have its own VLAN; that is as clumsy as home running each "VLAN" into an open port on a firewall (seen it done) from a daisy chain of access switches below.

but why would you re-engineer it? i'm still not clear on why it is such a bad idea. one of the requirements i gave myself is that the network should be reliable, users shouldn't be able to detect a failure in the network... i need to have a routed access layer in order to achieve that kind of reliability - no? and as a result of having l3 in the access layer each access switch is therefore going to need its own vlan. is this not correct? sorry about grammar, accessing the forum on mobile seems to have issues with uppercase. thanks.

networker050184

I don't necessarily think its a bad idea. I'm actually a pretty big fan of routing to the access layer when its applicable.

DevilWAH

I agree nothing wrong with routing to access layer, not so long ago it was all the rage to put in as much routing as possible, in the last year or two things have stared moving back to more flat access layers where vlans span multi switches back to the core.

Generally routing proformance is more costly than switching, which is another good reason to only use it when necessary.

A question.. What are the benefits of using routing over switching?? and what are the advantages of switching over routing? There nothing that says one is better to use than the other You can use either at any layer you wish. What you must do is be consistence! Don't use routing in one area for one purpose and then switching in another for the same purpose. Keep it consistence, even if its not perfect, it will be more straightforward to amend it later.

SteveO86

Not sure where the design is suppused to stop at but schools are typically big on WLANs for students, and content filtering, and security between student and faculty access.

it_consultant

Eildor wrote: »

but why would you re-engineer it? i'm still not clear on why it is such a bad idea. one of the requirements i gave myself is that the network should be reliable, users shouldn't be able to detect a failure in the network... i need to have a routed access layer in order to achieve that kind of reliability - no? and as a result of having l3 in the access layer each access switch is therefore going to need its own vlan. is this not correct? sorry about grammar, accessing the forum on mobile seems to have issues with uppercase. thanks.

No, you don't need a routed access layer in order to have reliable links that, if they fail, users won't be able to detect. We do this regularly with link aggregate port groups, what I call a LAG. In fact, the failover will be more seemless with a LAG than an equal cost route link. I have run a continuous ping over a LAG while we failed out ports in the LAG and nary a packet dropped. With VRRP there will be an interruption, all be it a small one.

Link aggregation - Wikipedia, the free encyclopedia

It isn't necessarily a bad thing to have routing, but to have a separate network for each access switch is totally unnecessary. You could have an access layer which is routed to the core; you would use VLAN for that. That way you have the flexibility of adding switches without needing another network plus, if your requirements change throughout the life of the network, your addressing isn't married to the switch's physical location.

You can get really elaborate with LAGs. Traditionally switch A and switch B would be connected by one link. Lets say I deploy a traditional LAG. I plug another link between A and B and I get a bond. What happens if switch A or B fails. Those links don't amount to a hill of beans. So, I take switch A1 and A2, then B1 and B2, plug A1 and A2 together in a stacking configuration (40GB stacking cables) and do the same for B1 and B2. Then I take A1 and plug into B2 and A2 into B1. Since I have a stack those ports bond over a LAG, even though they are in different switches. This is the old picture Cisco uses to demonstrate spanning tree protocol, the one where we have to tell which ports will forward and which ones will block to prevent a loop. Except, in a LAG a loop will not occur (remember to set up the LAG BEFORE you plug the switches together). In a LAG, all ports will be forwarding.

Not only do you get good throughput using LAGs, you can also isolate a switch failure to the individual switch at the access, distribution, and core layers.

m3zilla

DevilWAH wrote: »

Generally routing proformance is more costly than switching, which is another good reason to only use it when necessary.

Do you any articles or studies to back this up? I'd be interested to see the performance between routing vs switching the access layer.

A question.. What are the benefits of using routing over switching?? and what are the advantages of switching over routing?

Unless you know your L2 topology inside out, troubleshooting routing is much easier than troubleshooting an STP issue. It's also much easier to predict behavior when a failure occur.

DevilWAH

m3zilla wrote: »

Do you any articles or studies to back this up? I'd be interested to see the performance between routing vs switching the access layer.



Unless you know your L2 topology inside out, troubleshooting routing is much easier than troubleshooting an STP issue. It's also much easier to predict behavior when a failure occur.

Cost of a layer 2 switch compared to a Layer 3 is a good start so see the difference, OK almost all devices now seem to be layer 3, and as soon as you need to step out side the basic feature sets of layer 3 switching the costs very quickly mount up. Remember layer 3 switching only supports very simple functions in hardware, any thing out side of this gets shunted in to the software layer. Once you have routing at the access layer, its then easy to start looking at being "cleaver" pushing the network out side the realms of your basic layer3 devices.

And as for point two, rouble shooting Layer 2 is just as straight forward as Layer 3. I think a Lot of people get brought up on Layer 3 and assume that Layer 2 is the "weaker brother" and complicated. However this is far from true, Layer 2 has some great features and is in many ways just like routing. The logic is the same and the trouble shooting steps the same. So I have to completely disagree with your statement that routing is easier and more predictable. It is simple not true, if you want to be a complete network engineer you should have a full understanding of both. Other wise you are missing a very important half of the picture.

networker050184

DevilWAH wrote: »

And as for point two, rouble shooting Layer 2 is just as straight forward as Layer 3. I think a Lot of people get brought up on Layer 3 and assume that Layer 2 is the "weaker brother" and complicated. However this is far from true, Layer 2 has some great features and is in many ways just like routing. The logic is the same and the trouble shooting steps the same. So I have to completely disagree with your statement that routing is easier and more predictable. It is simple not true, if you want to be a complete network engineer you should have a full understanding of both. Other wise you are missing a very important half of the picture.

I have to disagree with this, but I think a lot of it is just personal preference. I think routing is a lot more straight forward, predictable and tunable than a flat L2 network. When I set up a network its routing as far down as possible. Leave the broadcast and STP domains as small and confined as possible.

And to the point of LAGs by it_consultant you can do L3 LAGs as well and bring that same resiliency to the routed network.

DevilWAH

networker050184 wrote: »

I have to disagree with this, but I think a lot of it is just personal preference. I think routing is a lot more straight forward, predictable and tunable than a flat L2 network. When I set up a network its routing as far down as possible. Leave the broadcast and STP domains as small and confined as possible.

I am all for keeping broadcast and STP small And I think for management and monitoring of traffic routing is by far the winner.

I came in to networking on a flat network that it was not possible to move to Layer 3, so its where my knowledge of networking has grown from. I think Like you say is very much a personal preference, and what you are use to dealing with.

I like to keep VLANS logical relevant rather than physically. So while a vlan may only have 5 devices in it, they may be on 5 separate buildings. I sill have large numbers of VLANS but all routing is done with in the core network locations. Gives you small broadcast and reasonable size STP domains. I suppose its collapsing the core and distribution, which for medium size networks has benefits.. and issues

m3zilla

DevilWAH wrote: »

Cost of a layer 2 switch compared to a Layer 3 is a good start so see the difference

We weren't talking about cost, we were talking about performance. Unless when you said "routing proformance is more costly than switching", you were literally talking about cost as in $$?

And as for point two, rouble shooting Layer 2 is just as straight forward as Layer 3. I think a Lot of people get brought up on Layer 3 and assume that Layer 2 is the "weaker brother" and complicated. However this is far from true, Layer 2 has some great features and is in many ways just like routing. The logic is the same and the trouble shooting steps the same. So I have to completely disagree with your statement that routing is easier and more predictable. It is simple not true, if you want to be a complete network engineer you should have a full understanding of both. Other wise you are missing a very important half of the picture.

I disagree. I don't think L2 is the weaker brother, I think it's harder to predict what happens when failure occurs. In a routed topology, if a link fails on one of my router, I can tell exactly what's going to happen. In a L2 topology, I can only do so to a certain extent. It's simply harder to predict the behavior when a switch dies, or a link flaps in a L2.