Etherchannel Load Balancing

cjthedj45cjthedj45 Member Posts: 331 ■■■□□□□□□□
in CCNP
Hi,

I'm trying to figure out if there is a way to load balance traffic across port channel members evenly. We have a situation where a backup runs and causes one of the port channels to exceed its bandwidth. We have an ether channel with two Gb ports in between the Core and Server switch. The server switch has all servers patched in that need to be backed up. When the backup runs we receive alerts stating that one port channel is exceeding its bandwidth but no alerts for the other. I want to evenly distribute this traffic across both channels. This is not causing an issue as it is out of hours but as out network grows this could cause an issue.

The current configuration is this:
9 Po9(RU) - Gi1/0/26(P) Gi2/0/26(P)
sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-ip

I don’t fully understand how the portchannel link is selected. You have a few options for configuration I.E
port-channel load-balance {dst-ip | dst-mac | src-dst-ip | src-dst-mac | src-ip | src-mac} global configuration command.

What I have read on Cisco leads me to believe that the address in the frame is hashed to a numerical value that then selects one of the links in the channel. My configuration is source and destination IP address for load balancing. Therefore does this mean that the IP address is hashed and given a value and then sent down the associated channel? If it does then I don’t understand why both channels don’t appear to be being utilized. As each hash should be different generating a different value each time. The only other explanation is that perhaps there is a certain server that has a really huge back up and for the whole duration of this back up the one port channel is used. If this is the case then does anyone know if this can be changed so that traffic does not consume the whole bandwidth of that channel but instead it used the other channel?
· Share on FacebookShare on Twitter

Comments

  • CoolhandlukeCoolhandluke Member Posts: 118
    Hi,

    Your right about the load balancing methods, when using Src & Dst addresses they are XOR'd first to produce a value for the last bit in the address (allowing a value of 0 or 1) which will choose the channel (port 0 in the channel or port 1 in the channel). If your destination backup server has a value of 1 for the last bit, then the only variable bit will be your source servers, so for each server that ends in a 1, channel 0 will be used (xor) and for others channel 1 will be used. If you workout your addresses on paper you can manually figure out which servers will use which paths. If you find that lots of the servers are using the same channel path, a better solution may be to select src-mac load-balancing as most data will be going from servers to a single destination and mac's should be a little more random.

    hope this helps

    CoolHandLuke
    [CCENT]->[CCNA]->[CCNP-ROUTE]->COLOR=#0000ff]CCNP SWITCH[/COLOR->[CCNP-TSHOOT]
    · Share on FacebookShare on Twitter
  • marhomarho Member Posts: 14 ■□□□□□□□□□
    Hello,

    your load balancing algorithm works this way:
    You have source and destination devices:
    - server1 have IP address for example 192.168.0.11 (server switch)
    - backup server have IP address for example 192.168.0.1 (core switch)

    Now algorithm will apply logical function XOR on least bit of meaning (32th bit):
    - server1 have 32th bit 1
    - backup server have 32th bit 1
    -> 1 XOR 1 = 0, (this mean that selected link in channel is with index 0)

    As you can see, efficiency of load balancing depends on IP addresses. Therefore if you have more IP addresses with 32th bit with value 1 (or with value 0, it is same), your algorithm will be unbalanced. Absoluty good load balancing is not possible with Etherchannel, but if your servers on server switch generate equal bandwidth at same time, you can try reassign IP addresses for this servers. Half will have odd IP addresses and the other half will have even IP addresses.

    Another method is, that you assign odd IP addresses to all servers. And then, you will gradually reassign IP addresses from odd to even until load on both links in EthnerChannel will be equal.

    Then please let me know how it works.


    Kind regards, Martin


    // Sorry for my bad English
    · Share on FacebookShare on Twitter
  • jason_lundejason_lunde Member Posts: 567
    better than writing it on paper, just log onto 1 of the switches and run:
    test etherchannel load-balance...
    Plug in the ip's on the src and dst and it will tell you which link its using. If you are going from 1 IP to the same IP all the time with src-dst-ip then it will always use the same port-channel link.
    · Share on FacebookShare on Twitter
  • ZartanasaurusZartanasaurus Member Posts: 2,008 ■■■■■■■■■□
    Etherchannel isn't really for load-balancing one session. So if you have a backup server making a connection to a server, all of that traffic will flow over one link in the bundle. If you had a backup server backing up multiple servers over that connection at once, you should get some semblance of load-balancing depending on the algorithm. It's possible that they all come out to the same link though. Do what jason_lunde advised above to test how the load would be spread over multiple servers.

    If you want to be able to utilize multiple links for one backup session, you'd have to do MPIO between source and destination.
    Currently reading:
    IPSec VPN Design 44%
    Mastering VMWare vSphere 5​ 42.8%
    · Share on FacebookShare on Twitter
  • cjthedj45cjthedj45 Member Posts: 331 ■■■□□□□□□□
    Okay I have run the test from the backup server 10.248.232.6 to servers being backed and vice versa. As you can see from the results it’s fairly equal as to which channel gets chosen. Therefore this must mean that perhaps one particular back up causes this issue as it utilizes the one channel. It’s still a bit strange how it’s always gi1/0/26 that gets utilized and never gi2/0/26 especially as we have lots of backs up that run that could be equally as big as the one that seems to be over utilizing gi1/0/26. As it would appear that the load balancing is doing its job we may need to look at other solutions. Zartanasaurus what is MPIO all about? I did a quick Google and information about multipath comes up. Is this a Cisco feature? We may have to look at this as an alternative. Not unless anyone else has any suggestions? Thanks everyone for your reply’s it’s been a great help.

    From servers being backed up to back up server
    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.1
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.2
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.3
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.4
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.5
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.7
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.8
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.9
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.10
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.11
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.12
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.13
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.14
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.15
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.16
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.17
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.18
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.19
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.20
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.21
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.22
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.23
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.24
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.25
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.26
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.27
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.28
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.29
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.30
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.31
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.32
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.33
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.34
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.35
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.232.6 10.248.234.36
    Would select Gi1/0/26 of Po9











    Servers being back up to Back Up server.

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.1 10.248.232.6
    Would select Gi2/0/26 of Po9

    TM#test etherchannel load-balance interface port-channel 9 ip 10.248.234.2 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.3 10.248.232.6
    Would select Gi1/0/26 of Po9

    #
    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.4 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.5 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.6 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.7 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.8 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.9 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.10 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.11 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.12 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.13 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.14 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.15 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.16 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.17 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.18 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.18 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.19 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.20 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.21 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.22 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.23 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.24 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.25 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.26 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.27 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.28 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.29 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.30 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.31 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.32 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.33 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.34 10.248.232.6
    Would select Gi1/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.35 10.248.232.6
    Would select Gi2/0/26 of Po9

    #test etherchannel load-balance interface port-channel 9 ip 10.248.234.36 10.248.232.6
    Would select Gi1/0/26 of Po9
    · Share on FacebookShare on Twitter
  • tokhsstokhss Member Posts: 473
    Curious, did you ever try to the different load balance algorithms and retest ? SRC-DST-IP vs SRC-DST-MAC .. etc ?
    · Share on FacebookShare on Twitter
  • cjthedj45cjthedj45 Member Posts: 331 ■■■□□□□□□□
    tokhss wrote: »
    Curious, did you ever try to the different load balance algorithms and retest ? SRC-DST-IP vs SRC-DST-MAC .. etc ?

    Hi Tokhss nope I never tried it in the end because from the test results it looks as if the channels are being chosen equally. Therefore I thought changing to SRC-DST-MAC would have the same results. I guess I need something that can identify that a certain channel is being saturated and then to pro-actively switch some of this load to the other channel that hardly has any load. Why did you have any reasoning for using SRC-DST-MAC?
    · Share on FacebookShare on Twitter
Sign In or Register to comment.