Slowness issue scenario

Cat5 · October 2011

I have a branch office complaining of slow speeds. So, I SSHed to their router and pinged to its default gateway and from router to one of its hosts. No packet loss. I logged in to one of the hosts there, then FTPed a big file to an outside server, and I saw slow times that should have been much faster, so I see the problem. The default gateway is a router in the ISPs territory somewhere, and it eventually comes back to us at the main office. I also saw no errors on the interfaces.

Suggestions are welcome.

Bl8ckr0uter · October 2011

Check syslog for any error messages and check the utilization on the links. I would also bust out ttcp or nttcp and start testing througput. Are you collecting any netflow information? You may want to gather some info before you go to the isp

nel · October 2011

i take it all was ok and this as only just started happening recently or has this always been an issue?

1) do an mtr to identify any packet loss between you > isp > Branch. so run an mtr from the head office to a server or something on the branch LAN.
2) check bandwidth utilisation at the time if you can. even screw down into netflow data if possible to give you an indicator of the route cause in the traffic flow. That failing you could mirror the port and **** the data into ntop to take a look.
3) what kind of link are you using as it could be a constraint of the connection. i.e. adsl has faster download than upload speeds, so users uploading may notice this if they pull/push alot of data.
4) confirm the hardware utilisation (cpu etc) on necessary devices. i.e. branch router
5) confirm speed/duplex settings on key links if you see any interface errors.
6) create a baseline performance indicator and try and capture the performance when you recreate the issue. you wont know whats "normal" unless you have a baseline to compare against.
7) check syslog for any problems around that time.

those are just a few ideas anyway

Cat5 · October 2011

I use Netflow and don't see any maxing out. The speed/duplex settings match, also. This is a recent event - the past several days, anyway. Gonna sift through the logs now.

Brakarific · October 2011

Is QoS being implemented at all? If so, a couple of video conference calls would slow http traffic down a bit depending on your bandwidth. Also, look for open tcp sessions. You might not be using all your bandwidth, but a nasty virus or a compromised email relay could be bombing the crap out of your router by opening several thousand tcp sessions at once.

nerdydad · October 2011

Just be sure speed and duplex actually match, not just both sides set to auto. I have just seen it too many times.

creamy_stew · October 2011

nerdydad wrote: »

Just be sure speed and duplex actually match, not just both sides set to auto. I have just seen it too many times.

I've maybe spent triple digit hours on troubleshooting where both ends were set to auto. (the cause is usually crappy hardware at the customer/stub end.

I've probably spent 1000 hrs troubleshooting where someone decided to hard-set speed/duplex.

Bonus fun:

Someone (ISP/MPLS/VLAN-provider) configures a switch/router port, telling you: "make sure sure you set it to 10/full". You get get there and set it to 10/full. And you get approx. 1.5 Mbit. Ok duplex problem. You set the port to auto/auto. Everything works great. In.fact, they now get 100/full.

Fast forward a month/year:

One day, your customer gets reaally crappy performance. You check the interface and see errors. Someone at the other end has found out that the interface was set to auto and "fixed the problem".

Extra bonus happy fun:

If you try to inform the SP that it seems thev'e set the port to auto/auto, you might want to substitute "Fast forward a month/year:" for "Fast forward a day/week:"

Slowness issue scenario

Comments