Network Design Questions

DevilWAH

Why do people think its unpredictable? every thing in networking is totally logical.. how L2 fails or has issues is just as predictable as routing. if you know where to look and how its set up. But that's the same as routing.

m3zilla

I don't think it's unpredictable, it's just harder to predict what's going to happen. If you have a L2 topology, a few switches deep, and a switch fail/link flap? Without really getting into it, can you tell whether or not the users downstream will experience an outage when STP is converging?

Like I said, if you know your L2 like the back of your hand, you'll be able to tell what's going to happen, but that doesn't mean it's not harder. I know what's 20+20 is, and I know what 20x20 is, but that doesn't mean I don't think addition is easier!

DevilWAH

m3zilla wrote: »

Without really getting into it, can you tell whether or not the users downstream will experience an outage when STP is converging?

yes I can, i know the timers I know the configuration I know what links will go down and for how long. if you know the root and the prioritys (you have set them right) it is easy to know what will happen.

m3zilla

The root and the priority will only tell you what the final topology will look like, it doesn't tell you how the network will behave when stp re-converge. But if you are as comfortable troubleshooting STP as you are routing, and think it's just as easy, then I bow to you because I, along with many others, cannot say the same.

it_consultant

m3zilla wrote: »

I don't think it's unpredictable, it's just harder to predict what's going to happen. If you have a L2 topology, a few switches deep, and a switch fail/link flap? Without really getting into it, can you tell whether or not the users downstream will experience an outage when STP is converging?

Like I said, if you know your L2 like the back of your hand, you'll be able to tell what's going to happen, but that doesn't mean it's not harder. I know what's 20+20 is, and I know what 20x20 is, but that doesn't mean I don't think addition is easier!

Using a LAG would make this scenario unlikely. A flapping port would just mean the bandwidth would go down. I keep spanning tree on in case some bozo plugs port 1 into 8 in a desktop switch behind his desk. My ISLs are always a LAG.

There is a lot of fear surrounding STP and rSTP but STP is a last resort, it should not be a principle design element for redundant switching. I have had this argument SO many times with CCNPs. They bust out an uber complex routing plan, I ask them "why", they say redundancy, I say "really?" and it boils down to a fear of STP and broadcasts. Plus, when does Cisco go over "etherchannel"? After you are inundated with a bunch of lessons on routing. In Brocade; you do layer 2 redundancy in the first few chapters. Metro ring protocol, STP and its variants, dynamic and static trunking, etc. Then into routing and redundant routing. Brocade's philosophy is that if a link goes down and packets are lost, you have failed. They get that from the storage side of the house I guess.

I used to work with couple of network engineers that were CCIE level and their philosophy was, essentially, why climb OSI for no reason. There is a whole lot of networking science surrounding delivering layer 2 links where we used to route because in most instances, it is better and easier to be on net.

Besides us chestbumping about whether routing or switching is preferable, we have actual design requirements now for levels of performance which dictate simplicity and speed. We are ALL running virtual machines which will failover to another building or something, they need to be on the same network if they have to boot in the failover site. Many of us are starting to see FCOE/FCIP traffic which works best nice and flat. This is the trend in design that I have seen.

PurpleIT

it_consultant wrote: »

I used to work with couple of network engineers that were CCIE level and their philosophy was, essentially, why climb OSI for no reason. There is a whole lot of networking science surrounding delivering layer 2 links where we used to route because in most instances, it is better and easier to be on net.

This can be a hard habit to overcome - once you have the knowledge required to accomplish a more complex/advanced/fancy/whiz-bag solution you want to use it.

Then, six months later when part of your network acts up or you need to make a change you have to rediscover all the tricks you had to use to make you fancy solution work, you run into limitations or you just plain forget some of the nuances you start to think that maybe, just maybe, you should have kept it simple.

networker050184

There is a lot of fear around STP and a lot of it is valid. STP domains can be very unpredictable. Is it all tunable to do exactly what you want? Sure, but it takes more design and operational overhead to keep a complex switched network flowing how you want it than it does for a routed network. The industry as a whole recognizes these short comings and the need to stay away from them. Its not just something people on this thread are making up. This is where initiatives like TRILL come into play because it is known that large flat networks that rely on STP are not a great design, just something we have to live with at this point.

There are times and places for both and neither is a simple cookie cutter solution. Flat L2 networks are being pushed by virtualization right now, not because they are better networks, just because the L2 connectivity is needed. That's why a lot of larger organizations and DCs are going the way of internal VPLS. Keeps your routed infrastructure along with the ability to have L2 connectivity between any devices anywhere in your network. Until intelligent L2 networks are available the routed infrastructure is the most stable but not always feasible.

Eildor

it_consultant wrote: »

No, you don't need a routed access layer in order to have reliable links that, if they fail, users won't be able to detect. We do this regularly with link aggregate port groups, what I call a LAG. In fact, the failover will be more seemless with a LAG than an equal cost route link. I have run a continuous ping over a LAG while we failed out ports in the LAG and nary a packet dropped. With VRRP there will be an interruption, all be it a small one.

Link aggregation - Wikipedia, the free encyclopedia

It isn't necessarily a bad thing to have routing, but to have a separate network for each access switch is totally unnecessary. You could have an access layer which is routed to the core; you would use VLAN for that. That way you have the flexibility of adding switches without needing another network plus, if your requirements change throughout the life of the network, your addressing isn't married to the switch's physical location.

You can get really elaborate with LAGs. Traditionally switch A and switch B would be connected by one link. Lets say I deploy a traditional LAG. I plug another link between A and B and I get a bond. What happens if switch A or B fails. Those links don't amount to a hill of beans. So, I take switch A1 and A2, then B1 and B2, plug A1 and A2 together in a stacking configuration (40GB stacking cables) and do the same for B1 and B2. Then I take A1 and plug into B2 and A2 into B1. Since I have a stack those ports bond over a LAG, even though they are in different switches. This is the old picture Cisco uses to demonstrate spanning tree protocol, the one where we have to tell which ports will forward and which ones will block to prevent a loop. Except, in a LAG a loop will not occur (remember to set up the LAG BEFORE you plug the switches together). In a LAG, all ports will be forwarding.

Not only do you get good throughput using LAGs, you can also isolate a switch failure to the individual switch at the access, distribution, and core layers.

Are you saying that it's possible to aggregate two links connecting to two separate switches? i.e., connecting an access layer switch to two distribution layer switches over an aggregated link? How does that work when you have STP enabled? Is there a Cisco white paper on this? And by LAG you do mean an EtherChannel right?

Thanks.

networker050184

It's possible though not on all devices. Look into multi chassis LAG.

it_consultant

Yes, I am saying it is possible to bond two switch links together. You can even bond two switches together while they are plugged into a different switch if they are in a "stack" configuration. You keep STP enabled, but LAG's do not use STP so when you deploy the trunk (the word "trunk" is used in non-cisco switches to describe a bonded set of ports) STP packets don't go over that trunk since STP is not necessary.

Setting up a bunch of redundant switch links which are purposely blocked by STP in case a switch goes down is a very old school way of doing things. Like, pre 2002, or whenever 802.1ax and 802.1ad were codified. I use the world "LAG" as in "Link Aggregate Group" as a generic term for bonded ports. The actual IEEE protocol is LACP or 'link aggregation control protocol'.

EtherChannel is Cisco proprietary; but if you run it in LACP mode as opposed to PAgP mode, you should be able to run a trunk to any non-cisco 802.3ad compliant switch.

it_consultant

networker050184 wrote: »

It's possible though not on all devices. Look into multi chassis LAG.

You said two things I couldn't agree with more - multi-chassis LAGs and TRILL. Brocade's old MCT protocol (multi-chassis trunking) is similarish to TRILL because both chassis are aware of all the MAC addresses on both switches, something necessary for any type of ethernet fabric.

DevilWAH

In addition to networker' comment about LAG to separate devices.

Look up Virtual switch system (VSS) in the cisco world. HP and other venders have there own versions but this allows two physicly separate devices to be seen as a single device in the network. 6500 model is the prime example. And allows ou to device etherchannel/LAG links to them.

Of course LAG and PVST are all ways to avoid redundant links not being used and to simplify Layer 2, and as has been mentioned the new (ish) kid on the block TRILL, which not all devices yet support, but is a kind of half way hosue between layer 3 and layer 2 designs.

Eildor

I don't know how I'm going to make it into a networking role once I graduate, too much to learn.

I don't think I can implement mLACP in GNS3, so I might just stick to the routed access layer design. As long as it works and I can justify what I've done it should be OK.

Thanks.

it_consultant

You will make it fine, you just need some diversity and to and get some experience under your belt. Using GNS3 is pretty limiting. I looked it up and their simulated switches do not support even quasi advanced features. It does not support etherchannel, so far as I can tell. In fact, GNS3 is not really a switch simulator at all. It only supports routers with a switch WIC which kind of gives the appearance of a switch.

DevilWAH

it_consultant wrote: »

You will make it fine, you just need some diversity and to and get some experience under your belt. Using GNS3 is pretty limiting. I looked it up and their simulated switches do not support even quasi advanced features. It does not support etherchannel, so far as I can tell. In fact, GNS3 is not really a switch simulator at all. It only supports routers with a switch WIC which kind of gives the appearance of a switch.

GNS3 is very limited for switching, however for other things it is great and much cheaper than a real lab. having a GNS 3 server along with 3 or 4 hard ware switch's and you can set up some quite complex stuff. You can run VirtualBOX/VM guests within GNS 3 and set up things such as ASA firewalls and Juniper routers. And it is very easy to connect to the real world (set up a router port in GNS3 to be the physical port on your machine).

I have replicated entire networks with 20+ routers and firewalls using GNS3 on a single machine, and while it is limited with the newer feature sets not available its still a good tool to get to grips with the routing side of networks.

And yes there's loads to learn, and the top CCIE's don't know most of it, but thats the fun bit as IT_Consultent says its all about the experince.

Eildor

it_consultant wrote: »

You will make it fine, you just need some diversity and to and get some experience under your belt. Using GNS3 is pretty limiting. I looked it up and their simulated switches do not support even quasi advanced features. It does not support etherchannel, so far as I can tell. In fact, GNS3 is not really a switch simulator at all. It only supports routers with a switch WIC which kind of gives the appearance of a switch.

If by diversity you mean learning other areas of IT, then I'm going to be in trouble. I've concentrated purely on networking, I don't know anything about setting up or managing a server. And even the networking I have been learning over 2 years is Cisco CCNA and CCNP stuff... so I wouldn't know how to configure a firewall, or monitor network traffic, or configure a Juniper device. The networking side of things is what I'm interested in; I do want to learn security too but I figured routing and switching would be more important as I can't see someone hiring a newbie for a security role.

I think GNS3 does support L2 EtherChannel, just not L3 EtherChannel.

DevilWAH

Eildor wrote: »

If by diversity you mean learning other areas of IT, then I'm going to be in trouble. I've concentrated purely on networking, I don't know anything about setting up or managing a server. And even the networking I have been learning over 2 years is Cisco CCNA and CCNP stuff... so I wouldn't know how to configure a firewall, or monitor network traffic, or configure a Juniper device. The networking side of things is what I'm interested in; I do want to learn security too but I figured routing and switching would be more important as I can't see someone hiring a newbie for a security role.


I think GNS3 does support L2 EtherChannel, just not L3 EtherChannel.

Any network engineer should have knowledge of servers. Not massive in depth knowledge but a basic understanding, otherwise you will find it very hard designing networks to support them. The juniour/entry level jobs are all about learning the fundamentals and diversity of IT. By concentrating only on networking at such an early stage could limit your options later on. Unless you get a job in a large speclised network company, most small to medium size organisations will expect there network staff to have some knowledge of IT servers and applications. (DHCP and DNS for example).

phoeneous

Eildor wrote: »

I've concentrated purely on networking

You shouldn't limit yourself like that. Bad idea in my opinion.

it_consultant

Eildor wrote: »

If by diversity you mean learning other areas of IT, then I'm going to be in trouble. I've concentrated purely on networking, I don't know anything about setting up or managing a server. And even the networking I have been learning over 2 years is Cisco CCNA and CCNP stuff... so I wouldn't know how to configure a firewall, or monitor network traffic, or configure a Juniper device. The networking side of things is what I'm interested in; I do want to learn security too but I figured routing and switching would be more important as I can't see someone hiring a newbie for a security role.

I think GNS3 does support L2 EtherChannel, just not L3 EtherChannel.

If your going to specialize in networking then you really need to diversify beyond Cisco routers and into Juniper routers/firewalls/switches, Cisco firewalls, Foundry/Brocade switches, Extreme switches, HP switches, etc. Ultimately they are all very similar. Cisco mucks up the water a little bit because they call thing weird names, like VLAN trunking and Etherchannel as opposed to dual-mode ports and port trunking. Whatever, it is those little details that keep us well paid.

networker050184

phoeneous wrote: »

You shouldn't limit yourself like that. Bad idea in my opinion.

Why is that? I've never bothered learning anything besides networking because I know its what I want to do. I don't see a problem with having a goal and going for it without straying.

DevilWAH

networker050184 wrote: »

Why is that? I've never bothered learning anything besides networking because I know its what I want to do. I don't see a problem with having a goal and going for it without straying.

I would say from personal experience with consultants that the ones that have been brought up on purely network backgrounds give poorer solutions than those that have a wider knowledge base. Pure engineers go with textbook solutions that are not tuned to the customer needs, as they do not understand the constraints that the applications place on a design. The guys who have knowledge of the server side are much eaiser to sit down with an discuss the solution and work around specifice requirements.

Of course there are exceptions to the rule, but having worked in one of the worlds largest CISCO gold partners, they would often send there CCIE's on entry level server and sysadmin courses if they did not have any back ground in them.

I not suggesting you need to be a SQL guru, just understand what SQL is and why people might use it for example.

networker050184

The way it should be done is a design is made by a group of experts, not one single person. Each person brings their piece of the pie to the table and then an overall design is created. You have your application expert that works with the application and knows what it needs to work. Then you have your network experts that know how to make that possible. If you have one guy filling both roles that is not a scalable solution. Applications and networks are both full time jobs. Sure some very small shops can get away with a jack of all trades but its not optimal by any means.

And I completely disagree that engineers go with text book solutions that don't meet customer needs. That's ridiculous. The whole point of a network is to meet the needs. Why would an engineer, or anyone else for that matter, design something that doesn't fit those needs? Only reason I can think of is because they don't want a job any more or they just don't have the skill set to do the job. That has nothing to do with them being an engineer though.

it_consultant

What I have seen with consultants, which is what I think Devil was talking about, is that they take a cookie cutter approach without really analyzing the network and its needs. I dare say that Cisco only consulting shops are pretty guilty of this. The shops with engineers that know both diverse switching hardware manufacturers as well as understand things in the system admin realm tend to provide more complete and detailed solutions.

I hold sysadmins to the same standards. They have to know a little of the other side so they can be useful on the team.

networker050184

I'd say that's an issue with their business process and nothing to do with individual knowledge.

DevilWAH

the issue is that if the application guy does not under stand a thing about networks and the network guy does not understand applications. they are not efficent at passing information between each other and you get the old "Chinese wispers" senario. in a good team the application guy and the network guy will have some under standing about each others areas. This may simple be through years of working together and knowledge picked up while working on projects.

There is a great difference between a "jack of all trades" and a specialist who take an interest in understanding the applications his network will be supporting. But then again we had a whole thread around the generalist / specialist debate recently . There is deffently a trade of, and the deeper you go in one area the less time you can afford to spend on others, but that does not mean they should become a black hole in your knowledge.

Apollo80

I have to also agree that having a high-level understanding of an entire IT infrastructure is necessary in order for a consultant to make a solid design recommendation. This extends to having a basic understanding of environmentals (power, cooling, rack space, air flow, etc.)

networker050184

Of course a high level understanding of how devices communicate is needed, but knowing what you want to do and studying it is not a bad thing. Everyone doesn't have to spread their self so thin. Its one of the hugest mistakes I think people on TE make when trying to start out. You see the people that try to get their CCNA, their MSCE, their CEH and everything under the sun and still stuck on the help desk. Then you have the guy that has the laser focus to get himself somewhere. Studying servers and individual applications in depth is completely unnecessary to having a successful career in networking.

Eildor

networker050184 wrote: »

Of course a high level understanding of how devices communicate is needed, but knowing what you want to do and studying it is not a bad thing. Everyone doesn't have to spread their self so thin. Its one of the hugest mistakes I think people on TE make when trying to start out. You see the people that try to get their CCNA, their MSCE, their CEH and everything under the sun and still stuck on the help desk. Then you have the guy that has the laser focus to get himself somewhere. Studying servers and individual applications in depth is completely unnecessary to having a successful career in networking.

This is how I have been looking at it. I want a networking position, so that's what I've been concentrating on. Why spend a year or two learning servers when I could be spending that time furthering my knowledge of networking. I want to specialise in networking; I'd hate to be stuck in a job where I'm being paid an average wage, to have average knowledge of both networking and servers.

I don't know, maybe I'm wrong. I don't have experience so I don't know how it is in the real world. I'm just trying to concentrate on what I am interested in hoping that it will all work out in the end. However, I look at job listings weekly just to see what's out there and it's always overwhelming to see the amount of knowledge you need.

Anyway, I guess we are going way off topic!

it_consultant

The best network engineer I ever worked with was a quite capable server admin as well. Let me tell you how helpful it is to have a network engineer who can help me set up both the IP helper address and the DHCP server itself. It is also helpful to them when I (I roll on sysadmin and network engineering) can diagnose a networking problem - recently it was port security on VPLS links. Their hold down timer for MAC addresses was 2 hours instead of 5 minutes or something reasonable. Point being, I knew the symptoms....people leave one office for another and when they plug in they can't get to resources inside the office. If they connect to wifi they are fine. It cuts a lot of BS troubleshooting when I can call and say "look, I think your device is suppressing my ARPs because it still sees the MAC on the other device". That isn't necessarily low level network troubleshooting. I am a Windows and virtualization admin most of the time. Last Thursday I was troubleshooting BGP routes between my router and the ISP. BGP was fine, the port on ISP side was frozen, I could tell because the ARP status was such that I saw their IP but no MAC addy. Hard to get a BGP table when layer 2 isn't working. I call telecom and tell them to shut and no shut their port, etc etc.

Today, the problem is in my Hyper-V cluster, I have 5 nodes and only 4 can get reservations on the storage system. What is really maddening is that if I force one of the servers to release their reservation, a random server will lose its reservation and it wont get it back.

What I am illustrating is that unless you are going to be SUPER good at one thing. It is really not a bad idea to diversify your skillset. You might end up working for an ISP and only ever doing networking for the rest of your career. Lets be real for a second, that represents a fraction of the jobs out there. When you get into enterprise architecture, where you get paid a LOT for your opinion, you need to know a good depth on a wide range of stuff.

phoeneous

networker050184 wrote: »

Why is that? I've never bothered learning anything besides networking because I know its what I want to do. I don't see a problem with having a goal and going for it without straying.

Because IT goes beyond just the network. Why limit yourself?

In some places, there is only so much networking that you can do. I work for a company with 10 sites and less than 200 staff, if I only did networking I'd be bored out of my fricking mind. And by networking I'm referring to stricly routing and switching, not voice or security or wireless.