Switch Memory Utilization

burbankmarcburbankmarc Member Posts: 460
in CCIE
Hey all,

I have 2 3560's on the outside/DMZ of my ASAs. Now all my exiting traffic goes through these guys. I have OSPF, QoS, SPAN, and a couple minor features turned on.

My problem is that the memory is pretty much chewed up. It sits at about 70% constant. I had an older IOS and I think a memory leak or something occured because 100% memory was being used. I rebooted the switches and upgraded the IOS but still it sits at about 70%.

So I guess my question is, is it time to upgrade these to 3750's? I thought the 3560 would be able to handle this, we don't push a whole lot of traffic..less than 20mbps.

Any thoughts?
· Share on FacebookShare on Twitter

Comments

  • jason_lundejason_lunde Member Posts: 567
    Use a command like
    show mem proc allocating-process totals

    to see what processes are chewing up all of your memory. Before I replace anything I would always open up a cisco tac case and let them take a look as well...I take advantage of our support contracts :)
    · Share on FacebookShare on Twitter
  • mikej412mikej412 Member Posts: 10,086 ■■■■■■■■■■
    burbankmarc wrote: »
    My problem is that the memory is pretty much chewed up.
    Which SDM template is configured on the switches?
    :mike: Cisco Certifications -- Collect the Entire Set!
    · Share on FacebookShare on Twitter
  • burbankmarcburbankmarc Member Posts: 460
    I don't use SDM. I removed all the SDM files and disabled the http server.

    So far here's the services:
    OSPF
    Classification (1 port)
    WRR-Queueing
    HSRP
    CEF
    STP
    SNMP
    NTP
    SSH
    SPAN

    The OSPF process isn't too complex either. The routing table is relatively small and only has 2 areas.

    These seem to be taking up the majority of the memory:
        PC          Total   Count  Name
0x01B8565C    2469640     310  Process Stack
0x020D2A5C    1910436      85  CEF: Brkr Update Rec
0x013AB988    1575420    1705  *Packet Header*
0x01FAA824    1004652       9  Init
0x0151D354     665132       1  Init
0x0239ABD8     592268       1  LC message chunk
0x029BEE9C     443556      27  TW Buckets
0x016B4A74     401760      40  Flashfs Sector
0x028AA198     393480       6  STP Port Control Block Chunk
0x0151D320     388012       1  Init
0x013B0E8C     284840       6  pak subblock chunk
0x020A924C     262232       2  CEF: hash table
0x029EABD8     215600     308  Process
0x021D7934     204064    3000  IPC Fragment
0x021D6D28     196740       3  IPC Message He
0x015E35B4     196652       1  HRM: destination array
0x0151D388     166316       1  Init
0x0199A5F8     163884       1  Init
0x0260DE4C     153316       1  Init

    Maybe I'll open a TAC, but lately they haven't really been all that helpful.
    · Share on FacebookShare on Twitter
  • jason_lundejason_lunde Member Posts: 567
    burbankmarc wrote: »
    I don't use SDM. I removed all the SDM files and disabled the http server.

    So far here's the services:
    OSPF
    Classification (1 port)
    WRR-Queueing
    HSRP
    CEF
    STP
    SNMP
    NTP
    SSH
    SPAN

    The OSPF process isn't too complex either. The routing table is relatively small and only has 2 areas.

    These seem to be taking up the majority of the memory:
        PC          Total   Count  Name
0x01B8565C    2469640     310  Process Stack
0x020D2A5C    1910436      85  CEF: Brkr Update Rec
0x013AB988    1575420    1705  *Packet Header*
0x01FAA824    1004652       9  Init
0x0151D354     665132       1  Init
0x0239ABD8     592268       1  LC message chunk
0x029BEE9C     443556      27  TW Buckets
0x016B4A74     401760      40  Flashfs Sector
0x028AA198     393480       6  STP Port Control Block Chunk
0x0151D320     388012       1  Init
0x013B0E8C     284840       6  pak subblock chunk
0x020A924C     262232       2  CEF: hash table
0x029EABD8     215600     308  Process
0x021D7934     204064    3000  IPC Fragment
0x021D6D28     196740       3  IPC Message He
0x015E35B4     196652       1  HRM: destination array
0x0151D388     166316       1  Init
0x0199A5F8     163884       1  Init
0x0260DE4C     153316       1  Init

    Maybe I'll open a TAC, but lately they haven't really been all that helpful.

    I think Mike was talking about the SDM, Switching Database Manager, on the 3750's, not the SDM you are thinking of. What IOS are you running, it isnt a 15.x version is it?
    · Share on FacebookShare on Twitter
  • burbankmarcburbankmarc Member Posts: 460
    jason_lunde wrote: »
    I think Mike was talking about the SDM, Switching Database Manager, on the 3750's, not the SDM you are thinking of. What IOS are you running, it isnt a 15.x version is it?

    Oh duh, I forgot all about that stuff. These are 3560's though, not 3750s.
    C3560-1-ches-va#sh sdm prefer
 The current template is "desktop routing" template.
 The selected template optimizes the resources in
 the switch to support this level of features for
 8 routed interfaces and 1024 VLANs. 

  number of unicast mac addresses:                  3K
  number of IPv4 IGMP groups + multicast routes:    1K
  number of IPv4 unicast routes:                    11K
    number of directly-connected IPv4 hosts:        3K
    number of indirect IPv4 routes:                 8K
  number of IPv4 policy based routing aces:         0.5K
  number of IPv4/MAC qos aces:                      0.5K
  number of IPv4/MAC security aces:                 1K
    · Share on FacebookShare on Twitter
  • APAAPA Member Posts: 959
    Like Mike said... it's more than likely the TCAM allocation due to the SDM template applied...

    Looks like you are running the default SDM template for those switches....

    run 'show platform tcam utilization' to give you a general view of current TCAM utilization based on the template and features configured...

    Then read this document at CCO to see whether a better allocation of resources via a the appropriate SDM template change will help bring down the CPU utilization..

    Catalyst 3560 Switch Software Configuration Guide, Rel. 12.2(25)SEE - Configuring SDM Templates [Cisco Catalyst 3560 Series Switches] - Cisco Systems

    I dare say that you are software punting packets due to QoS policies applied that are exceeding the QoS memory allocation... (but that is a wild stab in the dark, without seeing your full configuration) - and if this is the case I don't think any SDM template change allocates extra memory to qos aces\policies.. so you may be out of luck there... icon_sad.gif
    CCNA | CCNA:Security | CCNP | CCIP
    JNCIA:JUNOS | JNCIA:EX | JNCIS:ENT | JNCIS:SEC
    JNCIS:SP | JNCIP:SP
    · Share on FacebookShare on Twitter
  • burbankmarcburbankmarc Member Posts: 460
    Well I'm stupid. It ended up being a problem with my SNMP server. All the stats were all wrong and I just restarted the service and things seem to be right again.

    I was worried because I did experience a memory leak on one of these things last week, which resulted in me being locked out and needing to reboot and upgrade the IOS on the thing.

    Thanks for the help, I learned a few handy commands.
    · Share on FacebookShare on Twitter
Sign In or Register to comment.