Server load balancing

Eildor

I started a similar thread to this in the past but don't think I explained myself as well as I could have, which caused some confusion.

I have a server connected to two layer 3 switches via two 4 gigabit EtherChannels (thats a 4 gigabit EtherChannel per switch). I want to send 50% of traffic down one EtherChannel link to one switch, and 50% of traffic down the other EtherChannel link to the other switch.

There must be an easy way to achieve this on a 3750-X, surely? Why does any configuration need to be done on the switches anyway? All I want to do is alternate between physical links.

Thanks.

srg

You're talking about traffic "down" to the switches from the server, so that all depends on what hashing mechanisms the server provides. What kind of redundancy are you doing? Active/Passive to two standalone 3750X or are they stacked?

Zartanasaurus

You could do this with vPC on a Nexus switch...

pert

There is no way to do this that I know of if you're uplinking to two 3750s. You could if youre uplinking to nexus switches, or a 6500 pair with VSS, but that's the only way I know how.

srg

pert wrote: »

There is no way to do this that I know of if you're uplinking to two 3750s. You could if youre uplinking to nexus switches, or a 6500 pair with VSS, but that's the only way I know how.

vPC, VSS or 3750 stack. MLAG either way.

Eildor

What is the easiest (least switch configuration) way to achieve this on a 3750? I haven't read much about stacking, but it seems quite straight forward. What happens if the master goes down? The slave just takes over as the master right? How quickly does this happen? Also, I guess if you stacked the switches then you'd no longer need a FHRP for gateway redundancy -- or would you?

I'm still not clear as to why I can't just configure the server in such a way that it shares traffic out of all links... is there a reason why this wouldn't work? As the access layer is routed, return traffic would be load balanced between the 2 switches anyway, so what's the big deal? I'm just curious as to why it wouldn't work.

pert

3750 stack will work if all links terminate on the same stack, but most places I see still use separate stacks for dual homed connections, in that case it won't work.

srg

Eildor wrote: »

What is the easiest (least switch configuration) way to achieve this on a 3750? I haven't read much about stacking, but it seems quite straight forward. What happens if the master goes down? The slave just takes over as the master right? How quickly does this happen? Also, I guess if you stacked the switches then you'd no longer need a FHRP for gateway redundancy -- or would you?

I'm still not clear as to why I can't just configure the server in such a way that it shares traffic out of all links... is there a reason why this wouldn't work? As the access layer is routed, return traffic would be load balanced between the 2 switches anyway, so what's the big deal? I'm just curious as to why it wouldn't work.

You're right about FHRP not beeing neede for redundancy, its all managed in the stack.

With routed access this should be no problem, but you'll need double IP addresses on the server so might not be what you like. Go with the stack if you're able.

Eildor

srg wrote: »

You're right about FHRP not beeing neede for redundancy, its all managed in the stack.

With routed access this should be no problem, but you'll need double IP addresses on the server so might not be what you like. Go with the stack if you're able.

This is for a University project, so I want to make use of a FHRP just to show I can. I'm still not entirely clear on how this is going to work... how will the 4 link EtherChannel to each switch work? Practically I guess what I need to do is configure the EtherChannels on the switches and servers, and then team the two NICs (two 4 port NICs) together?

Am I going to have a problem with the gateways? Because what I'd need to do is use one gateway on one EtherChannel, and another gateway on the other EtherChannel (the access layer switches act as the gateway).

srg

Eildor wrote: »

This is for a University project, so I want to make use of a FHRP just to show I can. I'm still not entirely clear on how this is going to work... how will the 4 link EtherChannel to each switch work? Practically I guess what I need to do is configure the EtherChannels on the switches and servers, and then team the two NICs (two 4 port NICs) together?

Am I going to have a problem with the gateways? Because what I'd need to do is use one gateway on one EtherChannel, and another gateway on the other EtherChannel (the access layer switches act as the gateway).

If you have the two switches stacked, you make one 8 port etherchannel in the switchstack, and one 8 port team on the server. The server will think its connected to one switch, not two.

You cannot do one team to two switches if they're not stacked (or vPC, or VSS). If the two switches are separate you can make a 4 port channel between each NIC and switch and use them as L2 Active/Passive.

If you make a separate 4p team between each NIC and switch and the switches are gateways you'll need to set two gateways and hope that your OS supports that in a nice way.

Eildor

srg wrote: »

If you have the two switches stacked, you make one 8 port etherchannel in the switchstack, and one 8 port team on the server. The server will think its connected to one switch, not two.

You cannot do one team to two switches if they're not stacked (or vPC, or VSS). If the two switches are separate you can make a 4 port channel between each NIC and switch and use them as L2 Active/Passive.

You might be able to route directly between the server and switch with a separate IP per team on the server and run it active/active but its really a hack.

Can you tell me why this is technically not possible though; do you know? I just want to understand why I can't load balance between two EtherChannels connected to two different switches. I'm sure there is some reasonable technical explanation, but I just can't think of it.

Thank you for your help.

srg

Eildor wrote: »

Can you tell me why this is technically not possible though; do you know? I just want to understand why I can't load balance between two EtherChannels connected to two different switches. I'm sure there is some reasonable technical explanation, but I just can't think of it.

Thank you for your help.

You can, but not on L2. Your MAC can't show up on ports in two different switches for example. Two routed/L3 links are no problem.

networker050184

The issue is the way MACs and IPs are bound and transmitted. So when you have a server with the IP address 10.0.0.1 and MAC address aaaa.aaaa.aaaa.aaaa the mac will be forwaded out a single interface on the switch. So what happens when that destination MAC starts being seen out of two destination in the network? Nothing good that's for sure. Forwarding tables contantly changing and traffic not getting to where it's supposed to. There are technologies that can make this function like pointed our with the Cisco Nexus.

Eildor

networker050184 wrote: »

The issue is the way MACs and IPs are bound and transmitted. So when you have a server with the IP address 10.0.0.1 and MAC address aaaa.aaaa.aaaa.aaaa the mac will be forwaded out a single interface on the switch. So what happens when that destination MAC starts being seen out of two destination in the network? Nothing good that's for sure. Forwarding tables contantly changing and traffic not getting to where it's supposed to. There are technologies that can make this function like pointed our with the Cisco Nexus.

The access layer in this design is routed. I'm sorry, I should have mentioned that in the very first post. The ports connecting to servers are of course layer 2 ports, but the ports connecting to other switches (to distribution layer switches) are layer 3. There are no layer 2 connections between the two access layer switches. That makes it ok, right?

srg

Eildor wrote: »

The access layer in this design is routed. I'm sorry, I should have mentioned that in the very first post. The ports connecting to servers are of course layer 2 ports, but the ports connecting to other switches (to distribution layer switches) are layer 3. There are no layer 2 connections between the two access layer switches. That makes it ok, right?

Ok well that might work, but the server will have 2 IP adresses from different subnets and two separate gateways. So depending on the OS you might or might not get a good load balancing going.

networker050184

Agreed with srg again, the OS might not want play nice with what you are wanting to set up. Would probably have to get some systems guys with more knowledge on the specific OS you want. Regardless I think it's an unnecessary setup and I've never seen it in the real world. What would be the benefit of this setup? What are you trying to accomplish? I know this is for a school project, but unnecessary complexity is never a good thing.

pert

Eildor wrote: »

Can you tell me why this is technically not possible though; do you know? I just want to understand why I can't load balance between two EtherChannels connected to two different switches. I'm sure there is some reasonable technical explanation, but I just can't think of it.

Thank you for your help.

I'm not an expert on all things theory but here's my explanation.
1 - You can't have Active/Active connections to two different switches due to spanning tree, to get around this you have to use a technology other than spanning tre
2 - Every technology that allows active/active that I know of requires the upstream switch to be one device logically, and have some sort of peer link/keepalive
3 - IIRC from NP studies, when a switch sees a MAC it already knows appear on a different port it updates the port the MAC is known on with the new port and the old entry/port no longer exists. I assume this is why the switches need to be one logical switch in order for it to be load balanced across two different pieces of hardware. This is why you need a switch stack, vPC, or VSS.

My 2c

Eildor

srg wrote: »

Ok well that might work, but the server will have 2 IP adresses from different subnets and two separate gateways. So depending on the OS you might or might not get a good load balancing going.

Why does the server need to have two IP addresses? I thought the whole point of NIC bonding was to make multiple NICs appear to be one NIC.

The way I imagine it working in my head is that the distribution layer switch has two equal cost routes to the server via the two access layer switches that the servers are connected to.

I'm sorry if I'm being a pain, just curious.

srg

The bottom line; what you want to do is possible in theory, but that doesnt mean its good practice to do so. As networker said, this is not a regular setup really.

Eildor

networker050184 wrote: »

What would be the benefit of this setup? What are you trying to accomplish? I know this is for a school project, but unnecessary complexity is never a good thing.

1. I want redundancy, which is why servers are dual-homed.
2. I also want to make use all links for extra bandwidth.
3. I want to use a FHRP protocol just to show I can.
4. I want something I can more or less implement on GNS3.

Also, I know little to nothing about VSS, MLAG or stacking. Stacking sounds like the easiest option, but that would mean I wouldn't need a FHRP... and I kind of already said in my proposal that I would be implementing a FHRP (VRRP).

srg

Eildor wrote: »

1. I want redundancy, which is why servers are dual-homed.
2. I also want to make use all links for extra bandwidth.
3. I want to use a FHRP protocol just to show I can.
4. I want something I can more or less implement on GNS3.

I cant see an easy way to have them all. Active/Passive redundancy with FHRP is no problem. You cant bond all the NICs in ONE channel going to TWO separate switches.

Eildor

pert wrote: »

I'm not an expert on all things theory but here's my explanation.
1 - You can't have Active/Active connections to two different switches due to spanning tree, to get around this you have to use a technology other than spanning tre
2 - Every technology that allows active/active that I know of requires the upstream switch to be one device logically, and have some sort of peer link/keepalive
3 - IIRC from NP studies, when a switch sees a MAC it already knows appear on a different port it updates the port the MAC is known on with the new port and the old entry/port no longer exists. I assume this is why the switches need to be one logical switch in order for it to be load balanced across two different pieces of hardware. This is why you need a switch stack, vPC, or VSS.

My 2c

No STP here, routed design therefore no STP between switches, or possible MAC address table instability issues.

Routing protocols can load balance between multiple physical devices, all I really want to do is implement that kind of load balancing on a host machine. Anyway, this is probably really technical and there's probably a good reason as to why it's not done.

networker050184

1. Makes sense.
2. This is a bad design strategy. If you have enough traffic that you need to utilize your redundant links you are going to have a bad day when one of them actually breaks. This is a situation a lot of people find themselves in due to poor prior planning or exponetial growth. It's not something people usually willing design themselves into.
3. If I were your professor I'd want to see a well designed network. Not one that uses stuff just because.
4. Probably going to limit you from a lot of the fancy things you are trying to accomplish like the MLAG technologies you'd need.

srg

Let me turn this around; how did you think this would work? You have two switches with routing between them, server is connected to both switches and you're asking why it needs to have two different IP-addresses. How do you suppose it could manage with one IP address, connected to two different L3 domains?

Eildor

networker050184 wrote: »

1. Makes sense.
2. This is a bad design strategy. If you have enough traffic that you need to utilize your redundant links you are going to have a bad day when one of them actually breaks. This is a situation a lot of people find themselves in due to poor prior planning or exponetial growth. It's not something people usually willing design themselves into.
3. If I were your professor I'd want to see a well designed network. Not one that uses stuff just because.
4. Probably going to limit you from a lot of the fancy things you are trying to accomplish like the MLAG technologies you'd need.

You're probably right, of course. Alright, thanks for all of your help guys, much appreciated. Sorry for the weird questions.

Eildor

srg wrote: »

Let me turn this around; how did you think this would work? You have two switches with routing between them, server is connected to both switches and you're asking why it needs to have two different IP-addresses. How do you suppose it could manage with one IP address, connected to two different L3 domains?

Why would it not work? I source traffic from a single IP address out of both interfaces, is that a big deal?

Let's say I configured both access layer switches to use the same IP address on their SVIs, would that theoretically not work? I could even configure ports connecting to servers with the same MAC address if that helps with the NIC bonding process (I don't know about NIC bonding).

Anyway, yes this is getting silly now ...I just found it to be an interesting problem to think about. But obviously there's already a solution which is much easier... just stack the bloody things!

Thank you!

pert

I'm going to admit it. I don't know why this doesn't work. I've never attempted it. I'm also curious as to what the exact error causes this topology to not work.

A single traffic flow from core to access will always use the same side, wouldn't it? As far as I can think it through this gets back to the server in one piece.

Is the problem that the server with a single bonded channel to both switches will break up a single flow across both access switches? Even if it does so, what does that break exactly? There may be high levels of jitter to the destination?

networker050184

It's going to come down to the OS and what type of traffic pattern you are running here. How is the upstream router going to route these packets? Most likely flow based so you end up worrying about hashing to try and get traffic to use more than one link. If you are using an FHRP and point the next hop as the VIP only one is going to respond anyway. Many things to consider here besides will packets technically flow.

pert

networker050184 wrote: »

It's going to come down to the OS and what type of traffic pattern you are running here. How is the upstream router going to route these packets? Most likely flow based so you end up worrying about hashing to try and get traffic to use more than one link. If you are using an FHRP and point the next hop as the VIP only one is going to respond anyway. Many things to consider here besides will packets technically flow.

I'm not arguing that, I'm just saying I don't know where this actually breaks down and fails.

srg

Eildor wrote: »

Why would it not work? I source traffic from a single IP address out of both interfaces, is that a big deal?

Let's say I configured both access layer switches to use the same IP address on their SVIs, would that theoretically not work? I could even configure ports connecting to servers with the same MAC address if that helps with the NIC bonding process (I don't know about NIC bonding).

Anyway, yes this is getting silly now ...I just found it to be an interesting problem to think about. But obviously there's already a solution which is much easier... just stack the bloody things!

Thank you!

This is gonna be all kinds of funky . As I said, that probably gonna work in theory but its far from best practice and is definitely not recommended. Just stack them. If you're running L3 between the switches you're not gonna be able to run a FHRP anyhow.

Eildor

srg wrote: »

This is gonna be all kinds of funky . Just stack them. If you're running L3 between the switches you're not gonna be able to run a FHRP anywho.

Well yes, in the scenario I described I guess you wouldn't need a FHRP because both switches would be configured with the same IP address on the SVI.

That's a good point though, I didn't consider the impact of having a L2 link between the switches. That wouldn't be a problem though, would it? Server ARPs for its gateway, the directly connected switches respond without the ARP crossing over between switches L2 link.

Edit: Anyway, yes, stack is the way to go. I'm just being too curious about something which doesn't even matter.