Options
Memory Error?
notgoing2fail
Member Posts: 1,138
in CCNP
Just wondering if anyone has seen this error themselves? I received it when trying to apply IP NAT INSIDE or OUTSIDE to an interface. It seems like I am low on RAM in some way but that's kinda crazy since the router isn't doing anything.
Here's my show version:
RTR-1811W(config-if)#ip nat inside % NBAR Error : Activation failed due to insufficient dynamic memory % NBAR Error: Stile could not add protocol node %NAT: Error activating CNBAR on the interface FastEthernet1 RTR-1811W(config-if)# *May 10 17:20:46.763: %SYS-2-MALLOCFAIL: Memory allocation of 10420 bytes failed from 0x82146094, alignment 0 Pool: Processor Free: 106796 Cause: Memory fragmentation Alternate Pool: I/O Free: 80 Cause: Not enough free memory -Process= "Virtual Exec", ipl= 0, pid= 115 *May 10 17:20:46.763: %NBAR-2-NOMEMORY: No memory available for StILE lmalloc RTR-1811W(config-if)#
Here's my show version:
Cisco 1811W (MPC8500) processor (revision 0x300) with [B]118784K/12288K[/B] bytes of memory. Processor board ID FTX1001Y077, with hardware revision 0000
Comments
-
Optionsnotgoing2fail
Member Posts: 1,138
jason_lunde wrote: »hm, what does a show proc mem display?
Darn, I wish I knew about that command before I rebooted. I don't get the error anymore. (I had a feeling it would go away after the reboot).
But I have seen this error before when I'm playing with my router a lot and putting in commands here and there, so I'll have to issue this command when I run into the error again.
For FYI purposes, is there anything specific I should be looking for in this command? Here's what it currently says....
The list is pretty long so I'll just show a snippet...Processor Pool Total: 45539524 Used: 36287436 Free: 9252088 I/O Pool Total: 12582912 Used: 6655376 Free: 5927536 PID TTY Allocated Freed Holding Getbufs Retbufs Process 0 0 80984720 42886580 34297172 2054 551 *Init* 0 0 12316 359712 12316 0 0 *Sched* 0 0 8100548 5738620 1988568 11 9 *Dead* 0 0 0 0 1713148 0 0 *MallocLite* 1 0 323388 1552 329064 0 0 Chunk Manager 2 0 236 236 4228 0 0 Load Meter 3 0 13972 712 39064 0 0 LICENSE AGENT 4 0 0 0 7228 0 0 RO Notify Timers 5 0 3344 236 10336 0 0 Check heaps 6 0 0 0 7228 45 0 Pool Manager 7 0 0 0 7228 0 0 DiscardQ Backgro 8 0 236 236 7228 0 0 Timers 9 0 0 0 4228 0 0 WATCH_AFS 10 0 0 0 7228 0 0 License Client N 11 0 236 236 7228 0 0 LED Timers 12 0 0 0 25228 0 0 Crash writer 13 0 352 0 7580 0 0 Exception contro 14 0 0 0 7228 0 0 Over Heat Deboun 15 0 0 0 7228 0 0 Fan Fault Deboun 16 0 0 0 7228 0 0 IPC Event Notifi 17 0 0 0 7228 0 0 IPC Dynamic Cach 18 0 0 0 7228 0 0 IPC Session Serv 19 0 0 0 7228 0 0 IPC Zone Manager 20 0 0 0 7228 0 0 IPC Periodic Tim 21 0 0 0 7228 0 0 IPC Deferred Por 22 0 0 0 7228 0 0 IPC Process leve 23 0 1704 0 8932 0 0 IPC Seat Manager 24 0 0 0 7228 0 0 IPC Seat RX Cont 25 0 0 0 7228 0 0 IPC Seat TX Cont 26 0 0 0 7228 0 0 IPC Keep Alive M -
Options
jason_lunde
Member Posts: 567
Well in your case the "free" memory at the top of the output. If that is looking pretty low, or if you have not configured anything and see it slowly depleting over a matter of time you may have problems (such as a slow memory leak); or you have just configured too much stuff and your router does not have the memory to allocate to that particular process. Another tip on that command (if you see it giving you that message again) is to issue a:
show proc mem sorted
This will display the processes consuming the most memory at the top. you might be able to find your offender and tie it to your config, or an ios bug. HTH's man, some others may have some more useful insight. -
Optionsnotgoing2fail
Member Posts: 1,138
jason_lunde wrote: »Well in your case the "free" memory at the top of the output. If that is looking pretty low, or if you have not configured anything and see it slowly depleting over a matter of time you may have problems (such as a slow memory leak); or you have just configured too much stuff and your router does not have the memory to allocate to that particular process. Another tip on that command (if you see it giving you that message again) is to issue a:
show proc mem sorted
This will display the processes consuming the most memory at the top. you might be able to find your offender and tie it to your config, or an ios bug. HTH's man, some others may have some more useful insight.
Thanks, I'm jotting this down. Sounds a lot like unix's TOP command....
The router was only up for a couple minutes. It could be a bug because I've ran into this issue before without much configuration on it.
What I was doing prior to the error was outputting debug info for ICMP and ARP. I went ahead and turned off ALL debugging.
I then played around with IP NAT in which I then saw the error...
I'll see if I can replicate it again.... -
Optionstiersten
Member Posts: 4,505
Buy more memory. You're running what appears to be 15.0 or 15.1 with only 128MB installed when CCO lists a minimum of 256MB. The memory in a 1841 is a regular PC133 SODIMM but I'm unsure whether it is the same for a 1811.
On a clean boot, you've got less than 9MB free in the process pool which is where all the processes allocate their memory. The IO pool has less than 6MB as well so you'll get odd things happening if you start pushing a large amount of data through the router.
The error from your original post shows that it has only 104KB free but it failed the allocation of 10KB because it couldn't find a contiguous section of memory large enough.
My 1841 which is currently running 15.1 uses around 192MB just idling. -
Optionsnotgoing2fail
Member Posts: 1,138
Buy more memory. You're running what appears to be 15.0 or 15.1 with only 128MB installed when CCO lists a minimum of 256MB. The memory in a 1841 is a regular PC133 SODIMM but I'm unsure whether it is the same for a 1811.
On a clean boot, you've got less than 9MB free in the process pool which is where all the processes allocate their memory. The IO pool has less than 6MB as well so you'll get odd things happening if you start pushing a large amount of data through the router.
The error from your original post shows that it has only 104KB free but it failed the allocation of 10KB because it couldn't find a contiguous section of memory large enough.
My 1841 which is currently running 15.1 uses around 192MB just idling.
You're right! I am running 15.1(1) XB, how did you know that? I don't see anywhere up on my post that shows the IOS version....
I just bought 128 compact flash for this thing....the flash should be ok right?
I guess I need to open this thing up eh? -
Optionstiersten
Member Posts: 4,505
There is a license agent in your process list which appeared in 15.0 along with the new licensing system.notgoing2fail wrote: »You're right! I am running 15.1(1) XB, how did you know that? I don't see anywhere up on my post that shows the IOS version....
Any particular reason why you're running a special branch of IOS? The release notes for 15.1(1)XB say it is 15.1.1T but for people who specifically want LISP. I assume that you don't need LISP so it would be better to just run the T train or the M train. The T releases are already bleeding edge enough with possible bugs and broken features. You don't want to run the special releases which may not get the same test coverage unless you have a specific reason to do so.
The CF card will be fine but you'll have to investigate to see what type of memory it takes. The 1811 has a maximum of 384MB which is 128MB from the built in memory and 256MB from a SODIMM. The part number for the 256MB upgrade is MEM181X-256D which is different than the 1841 one but I'm unsure whether the actual SODIMM is different or whether its just a SKU difference.notgoing2fail wrote: »I just bought 128 compact flash for this thing....the flash should be ok right? -
Optionsjason_lunde
Member Posts: 567
There is a license agent in your process list which appeared in 15.0 along with the new licensing system.
Dear god you are observant! Good catch, and when I initially saw this thread I thought to myself "tiersten will be all over this!" cheers man, good reply -
Optionsnotgoing2fail
Member Posts: 1,138
jason_lunde wrote: »Dear god you are observant! Good catch, and when I initially saw this thread I thought to myself "tiersten will be all over this!" cheers man, good reply
I know, as I've said before, he's an IOS walking dictionary!!! -
Optionsnotgoing2fail
Member Posts: 1,138
There is a license agent in your process list which appeared in 15.0 along with the new licensing system.
Any particular reason why you're running a special branch of IOS? The release notes for 15.1(1)XB say it is 15.1.1T but for people who specifically want LISP. I assume that you don't need LISP so it would be better to just run the T train or the M train. The T releases are already bleeding edge enough with possible bugs and broken features. You don't want to run the special releases which may not get the same test coverage unless you have a specific reason to do so.
The CF card will be fine but you'll have to investigate to see what type of memory it takes. The 1811 has a maximum of 384MB which is 128MB from the built in memory and 256MB from a SODIMM. The part number for the 256MB upgrade is MEM181X-256D which is different than the 1841 one but I'm unsure whether the actual SODIMM is different or whether its just a SKU difference.
No reason at all, this was early on when I downloaded the IOS and wasn't sure what I was downloading. LOL...
I probably should revert back to 12.4 or the 15.1.1T that you mentioned...
I'll look into the SODIMM and see what is out there in good old eBay!
