Network Re-Design

cjthedj45

Hello Tech Exam Buddys,

Just wanted to get people thoughts on a network re-design I’m planning.

I’m a new employee at my company and in the 3 months I have been there. I have seen several switch and firewall failures and most recently we have been hacked twice at the same branch office.

The network re-design I have to do currently hosts external customer websites, has a small internal lan with not that many users and connection into the MPLS network to connect to all the other branch offices.

The brief I have been given is it needs to be done quickly but due to the security incidents the board will allow a budget for this outside of the normal IT budget. As we have been given the opportunity to have money outside the budget I have said we need to enhance security and create a resilient network. My opinion is we should really use this site as flagship site for how the other networks should be designed as the other networks have flaws as well.

Currently the network has two MPLS router Primary and Back up. Unfortunately that is where the redundancy stops. After that we have one firewall with one IPS, DMZ, MPLS, Internet and Internal LAN is all connected of this firewall.

My suggestion would be to add an active/standby Outer firewall pair with IPS. That’s the first new layer of security.
These Perimeter firewalls would then connect into two separate Outer intermediate switches with a Trunk between the two with the same vlans in case one failed. The DMZ would be hosted of one these switches.

The next layer of security would be active/standby Inner firewall pair with IPS. Again these are both connecting into two separate intermediate switches with a trunk between the two in case one fails. The intermediate switch would host the MPLS connection. The intermediate switches would then connect into two Core switches and of the Core switches an even number of access and server switches would hang of the cores.

This design has created more resilience as we are doubled up on switches and firewalls and effectively have a Primary and Standby side. Security is enhanced as we now have some defence in depth with the two firewalls and IPS scanning at the Outer and Inner layer.

The challenge I have is the mind-set of the company is to always save money. This mind-set seems to have dissipated down to the line managers as well. Even though we have been told we have extra money. My line manager and another manager running with this project straight away said this is overkill. The board won’t spend that much and it does not make it that much more secure. It will make it more secure in my opinion and will make it resilient.

What are peoples thoughts on this should I push forward and try to persuade them by showing the benefits? Or do I just sit back and take second best. I think they may go for a two tier firewall approach which should make it more secure but they will probably say adding the standbys, intermediate switches and new core is overkill. Therefore we will have a network with one outer and inner firewall so no resilience.
As a network engineer I strive for my network to be available, secure, reliable, and resilient. Any thoughts are much appreciated.

wes allen

You need to do some risk analysis - get your SLE, ARO, and ALE learn on. If the numbers say that your design will cost less then the other in the long run, then it is much easier to sell to your manager.

SteveO86

I've done a good amount of network design for various companies. It's easy and fun to draw of the perfect design in visio, and type up a well written document for management to read at the same time. Maybe even go in there with an animated powerpoint simulating a failure on generic devices, but in the end it always comes done to the money (I've had one exception to that rule to date, where money no option dual MPLS carriers and dual hardware it was!)

In the end only you can venture to guess what your company may or may not go for (and not anyone else on this forum), but I'd also be prepared to scale it down just in case. Always ask for more so when they cut it down you end up with what you need/want

networker050184

It sounds like you are trying to make the perfect network. Nothing wrong with that except the reality is we all have to work within restrained budgets. You could save money by using a single pair of firewalls broken into different contexts/secrutiy domains how ever you call them and keep the same level of operational security. Have it all hanging off a pair of core siwtches, possibly collapse the dist layer up and save some money depending on the size of the network.

it_consultant

Right off the bat I would analyze the quality of how you were hacked. If you were hacked because of SQL injection scripts on your webpage, then your firewall pair may be essentially useless.

My experience has been that when you add complexity you add the risk of downtime - I have seen it happen over and over where an HA pair of switches fails where one switch would have stayed up because of an HA flaw or and admin doing something incorrectly during a routine change.

A switch, right out of the box, will achieve 99% uptime with no special configuration. This is generally true of all the manufacturers. What was the quality of the hardware failures? Are the switches old and reaching MTF, are they getting hit with power spikes, are they not in temperature controlled environments?

lordy

it_consultant wrote: »

Right off the bat I would analyze the quality of how you were hacked. If you were hacked because of SQL injection scripts on your webpage, then your firewall pair may be essentially useless.

^^ THIS!

You keep adding Firewalls and IPS but you haven't said a word about what the security incidents were. There are a lot of threads that neither a Firewall nor an IPS will protect you from (SQL Injection, Buffer Overflows, Software-Bugs, etc.).

Bonus question: Who will be analyzing the IPS log files?

cjthedj45

Hi all thanks for the quick responses.

I think what I'm proposing will be viewed as overkill but I do believe this is a standard we should all strive for as network engineers. It does come down to money that's for sure. Even if I point out that if a switch dies then there will be downtime. At my new company this seems to be acceptable. Whereas where I last worked we had resilience built right through the network with a layered security approach as well.

The servers that were hacked were due to a vulnerability in the apache version. Servers were cross contaminated as well. I did advise that the servers should be managed so they are patched regularly and we should run vulnerability assessments so we are aware what vulnerabiltes are on the servers and we can then patch or mitigate any risks so that the vulnerabilities are not exploited. They said this probably would not end up getting done therefore servers would end up not being patched or vulnerabilty assessments not acted upon. Instead they want to try and configure the access so cross contamination can not occur something to do with using ip tables. To me this is a reactive approach.

Perhaps I may have to scale down we have a 5505 that could be used for the perimeter and dmz could hang of this. We then have two tiers but the 5505 has no IPS. Also this means I have scaled down on all the other resilience . It's a different way of working for me as where I last worked we were able to build in the resilience and security. This was a site that had to be pic compliant whereas this site does not.

it_consultant

Using IPTABLES or some sort of host based intrusion detection/prevention is actually a great idea. It sounds too late if the malicious traffic is already at the server but it isn't. We run a bit of intrusion detection that can detect patterns which would indicate attacks on the public service (Palo Alto), ultimately those are signature based and not infallible. Running HIPS and securing your public facing websites is a must. Firewalls won't block traffic they are told to allow.

*ON SOAPBOX* Windows guys know they need to patch their sh!t and run a firewall/IPS because we know that MS products have vulnerabilities. Linux guy sometimes live in a world where Linux is "secure" and therefore special attention doesn't need to be paid to securing their systems.

*OFF SOAPBOX*

Seriously, you need to update your servers. We are scanned quarterly and do monthly patches. Right now we are mitigating the SSL/BEAST vulnerability in our Cisco gear.

lordy

If you do not patch your servers you might as well throw out Firewalls and IPS. It's like wearing a bullet-proof vest when everybody is aiming at your head.

wes allen

cjthedj45 wrote: »

Even if I point out that if a switch dies then there will be downtime. At my new company this seems to be acceptable.

The servers that were hacked were due to a vulnerability in the apache version. Servers were cross contaminated as well. I did advise that the servers should be managed so they are patched regularly and we should run vulnerability assessments so we are aware what vulnerabiltes are on the servers and we can then patch or mitigate any risks so that the vulnerabilities are not exploited. They said this probably would not end up getting done therefore servers would end up not being patched or vulnerabilty assessments not acted upon.

Maybe switch downtime doesn't cost them as much as having fully redundent switches would. Also, like it_consultant I have seen unnecessarily redundant network gear cause more downtime and trouble then non redundant gear would have. Your goal should be to find the best solution for your business case.

Same with the server patching - depending on what the incident cost you all, it might not be the best use of funds to spend x on solutions to protect against y damages, if x is greater then y.

PurpleIT

If I were presenting this, I would preface this with, "Ideally we would do the following", but as others have said, have a backup plan and be ready to present it immediately after the ideal design. I would compare on contrast the two so management knows exactly what they are giving up with the lesser design, but point out the positive parts of the alternate; if all they hear are negatives they might decide to do nothing.

Also, if you are planning on two layers of IPS look to different vendors. If the outside IPS doesn't catch an attack there is no reason to think the inside one using the same engine will detect it. Finally, as lordy pointed out, you have to analyze those IPS logs.

cjthedj45

Hi all

Thanks again for your replies. This particular branch office host customers website. Personally if I was running a business especially one such as ours with a global presence then ensuring the network was secure and resilient would be quite important. Especially as this site host customers website. If there is no resilience or good security policy then there is a risk that your customers will experience downtime if something fails and if a security incident occurs. I disagree that a resilient network can cause more problems than its worth. I have managed a network that was resilient all the way through for over 5 years. The network availability over those 5 years was probably 99.9 percent and this was with devices failing.

Anyway today I submitted 3 network designs. I did not hear anything straight away. Then one of the managers pipes up saying the board wants a network with no single point of failures. I'm thinking hello what did I just send you. Anyway I said to him that's exactly what I have just designed. Then all the sudden he is like well we need to sell it to them that its going to be to expensive and then they will have to move it to the data center. It looks like that's what they might do. So I'm thinking a bit of a waste of time really. It does not sound like anything is decided but it was a bit frustrating because he the starts asking my manager what do we need and let's make it expensive so the move its to the data center. I got up at 05.30 this morning as well to get it right

pert

Think you're a little too gung ho about the ideal. Its not your job to decide if redundancy is "worth it", your job is to show what the options are, and the accurate cost/benefit analysis of each. Many companies absolutely have no justification to have a fully redundant network. There are others where anything short of full redundancy requires someone's head rolling. You present the options, they pick them. If they pick a horrible one, just make clear what future problems they are being exposed to then move on.

networker050184

I agree 100% with pert. Business is about money. We as technology nerds would like to have the best possible solution at all times, but that is rarely the reality. Work to provide the best possible solutions within your budget restraints and you will be good to go.