IPSec Tunneling, NAT, and weird behavior(Cisco Lab Manual - CCNP ROUTE Lab 7-1)

pogue

For those that have the lab manual, I noticed some wierd behavior in this lab... Was seeing if there is just something I do not understand, or if this behavior is by design.

In this lab, an access list is created that stops traffic from the two private networks from being NAT'ed when communicating with each other. This is so that a second access list can designate "interesting traffic", that will be sent into the IPSec tunnel.

Okay, straightforward enough at this point. Until you realize that the server you configured a static NAT mapping for earlier in the lab cannot communicate over the tunnel to the other private network. When I trace from the server IP address to the other private network, it looks like it is being sent to the next hop router, which I should not see if it is going into the tunnel.

Am I missing something here?

In short, here is my question:

If you have two private networks that need to communicate over an IPSec tunnel, how do you handle the individual computers/servers that have static NATing configured? By the behavior I am seeing in GNS3, it looks like these systems get NAT'ed regardless due to the static NAT mapping, and will ignore any NAT access list that attempts to prevent the address from being NATed.

Help.. I hammered away at this for a few hours tonight trying to figure out what I was doing wrong..

Thanks,

Russ

wave

pogue wrote: »

If you have two private networks that need to communicate over an IPSec tunnel, how do you handle the individual computers/servers that have static NATing configured? By the behavior I am seeing in GNS3, it looks like these systems get NAT'ed regardless due to the static NAT mapping, and will ignore any NAT access list that attempts to prevent the address from being NATed.

Hey Russ, Can you clarify a little? Remember, even that you can still communicate between the private addresses on each side with no NAT. Traffic will only get NAT'd when it goes outside the tunnel, even though the tunnel itself is NAT'd.

When you're at the Branch communicating with the HQ servers then you'll use their private IPs.

wave

NAT is happening on the outside interfaces but not on the tunnel interface itself....so traffic going through the tunnel isn't NAT'd.

ip access-list extended HQ-NAT-ACL
remark Do not translate HQ LAN to Branch LAN addresses
deny ip 10.10.0.0 0.0.255.255 192.168.1.0 0.0.0.255
remark Translate Local LAN to all other Internet destinations
permit ip 10.10.0.0 0.0.255.255 any
exit

pogue

HQ#

crypto map BRANCH-MAP 10 ipsec-isakmp
set peer 209.165.200.242
set transform-set Branch-VPN
match address 101


ip nat inside source list 100 pool HQ-NAT-POOL
ip nat inside source static 10.10.20.238 209.165.200.238

(With this command in, Email Server cannot communicate with Branch private LAN. When I remove it, pings work. With this command removed, I do a sho ip nat trans on the HQ router, and get nothing. This is as expected, and tells me access-list 100 is acting to prevent the NAT action based on source and destination networks. When I put the static NAT entry back in, I get U.U U on a ping from the email server loopback to 192.168.1.1.

I thought that what was supposed to happen is that outbound traffic from 10.10.20.238 is supposed to match the deny statement in the ip nat inside source list 100 access-list, and be denied from the static NAT mapping as it enters interface l0? THEN, the crypto map on s0/1 sees the proper souce/destination networks in the packets, and sends it into the tunnel? This does not seem to be happening.)

access-list 100 deny ip 10.10.0.0 0.0.255.255 192.168.1.0 0.0.0.255
access-list 100 permit ip 10.10.0.0 0.0.255.255 any

access-list 101 permit ip 10.10.0.0 0.0.255.255 192.168.1.0 0.0.0.255

pogue

Again, what I think is happening here is that when the static NAT mapping is configured for the email server, it doesn't seem like access-list-list 100 is properly identifying the traffic to prevent the NAT'ing.

No static NAT entry configured-

HQ#traceroute 192.168.1.1 source 10.10.10.1

Tracing the route to 192.168.1.1


1 * * *
2 * * *
3 * * *
4 * * * (Ctrl + Shift + 6)



HQ#traceroute 192.168.1.1 source 10.10.20.238
Tracing the route to 192.168.1.1

1 * * *
2 * * *
3 * * *
4 * * * (Ctrl + Shift + 6)



WITH static NAT configured..

HQ(config)#ip nat inside source static 10.10.20.238 209.165.200.238


HQ#traceroute 192.168.1.1 source 10.10.10.1

Tracing the route to 192.168.1.1

1 * * *
2 * * *
3 * * *
4 * * * (Ctrl + Shift + 6)


HQ#traceroute 192.168.1.1 source 10.10.20.238

Tracing the route to 192.168.1.1

1 209.165.200.225 48msec 36msec 56msec
2 209.165.200.225 !H !H *

HQ#



It looks to me with the static NAT mapping configured, the packets are routed directly out S0/1 (and NAT applied), rather than through the IPSec tunnel. I may be wrong about what is really going on here, but either way, it looks like packets are getting handled differently when a static NAT mapping is configured, when the access lists have not changed.

Thoughts?

Russ

pogue

One last thought... In the case of the NAT access-list, when the incoming packets are allowed to just grab an IP address from the NAT pool, it seems that the access-list logic is acting first to decide whether to NAT or no.

It looks to me like the static NAT'ing may be taking place BEFORE the packet is run through the NAT access list. Like the following "decision tree"..



Loopback0 recieves an inbound packet - source: 10.10.20.238 destination:192.168.1.1

Is static NAT configured? If so, perform NAT translation and forward out S0/1.

int S0/1 recieves one of two potential packets-

source: 209.165.200.238 destination: 192.168.1.1 (in the hypothetical case that static NAT mapping is "forced".) S0/1 decision- This does not match NAT ACL, as it does not originate in the 10.10.0.0 network.. Pass to routing decision. Routing decision says this goes out S0/1, default route. This does not match crypto map, as it does not originate from the 10.10.0.0 network. ... Forward to next hop 209.165.200.225 (This EXACTLY mimics what is happening when I have the static NAT applied. See above traceroutes.)

source: 10.10.20.238 destination: 192.168.1.1 This packet is run throught the NAT access-list, is denied NAT translation, and is passed off to crypto map ACL for evaluation...

As a further test..With the static mapping on, I went to the ISP router and did a debug ip packet while doing a traceroute from R2 server IP:

HQ#traceroute 192.168.1.1 source 10.10.20.238

Tracing the route to 192.168.1.1

1 209.165.200.238
2 209.165.200.238 !H !H *


ISP# debug ip packet

Mar 1 IP:s=209.165.200.238 (Serial0/1), d=192.168.1.1 len 28 unroutable


Now.. my access-lists are not changing on the HQ router... the only thing that is changing is a static NAT mapping. Am I wrong in my troubleshooting methodology here? It really seems like the static NAT is performed BEFORE the NAT ACL evaluation...


If this is actually correct, then how would one configure communications between two private corporate networks when one server needed to be accessed both over the internet via it's public IP and across the corporate IPSec conenction via it's private address.

I am so confused right now.

Russ

wave

Can you paste your Config from your branch and hq routers? I will be back at my desk soon and will look at my lab in gns3. I didn't experience the same problem

networker050184

This may clear it up a bit.

NAT Order of Operation - Cisco Systems

wave

networker050184 wrote: »

This may clear it up a bit.

NAT Order of Operation - Cisco Systems

Exactly...Russ you must have something messed up in your config somewhere. Because the IPsec/ACLs should be checked first i.e:

ip access-list extended Branch-VPN-ACL
permit gre host 209.165.200.226 host 209.165.200.242
ip access-list extended NAT-TRANS
deny ip 10.10.10.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 10.10.10.0 0.0.0.255 any

When I did this lab I had to add the overload keyword due to a bug in GNS3 I found after googling.

pogue

Branch#
*Mar 1 14:25:05.240: %SYS-5-CONFIG_I: Configured from console by consolesho run
Building configuration...

Current configuration : 1924 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Branch
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
memory-size iomem 5
ip cef
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
archive
log config
hidekeys
!
!
crypto isakmp policy 1
encr aes
authentication pre-share
group 2
crypto isakmp key cisco123 address 209.165.200.226
!
!
crypto ipsec transform-set HQ-VPN esp-3des esp-sha-hmac
!
crypto map HQ-MAP 10 ipsec-isakmp
set peer 209.165.200.226
set transform-set HQ-VPN
match address 101
!
!
!
!
!
!
!
interface Loopback1
description Branch LAN
ip address 192.168.1.1 255.255.255.0
ip virtual-reassembly
!
interface FastEthernet0/0
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/0
no ip address
shutdown
clock rate 2000000
!
interface FastEthernet0/1
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/1
description Connection to ISP
bandwidth 64
ip address 209.165.200.242 255.255.255.248
ip nat outside
ip virtual-reassembly
clock rate 2000000
crypto map HQ-MAP
!
interface Serial0/2
no ip address
shutdown
clock rate 2000000
!
interface Serial0/3
no ip address
shutdown
clock rate 2000000
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 209.165.200.241
!
!
no ip http server
no ip http secure-server
ip nat pool BRANCH-NAT-POOL 209.165.200.249 209.165.200.254 prefix-length 29
ip nat inside source list 100 pool BRANCH-NAT-POOL
!
access-list 100 deny ip 192.168.1.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 100 permit ip 192.168.1.0 0.0.0.255 any
access-list 101 permit ip 192.168.1.0 0.0.0.255 10.10.0.0 0.0.255.255
access-list 125 permit ip 209.165.200.0 0.0.0.255 192.168.1.0 0.0.0.255
!
!
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
exec-timeout 0 0
line aux 0
line vty 0 4
login
!
!
end

Branch#








HQ#sho run
Building configuration...

Current configuration : 2067 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname HQ
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
memory-size iomem 5
ip cef
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
archive
log config
hidekeys
!
!
crypto isakmp policy 1
encr aes
authentication pre-share
group 2
crypto isakmp key cisco123 address 209.165.200.242
!
!
crypto ipsec transform-set Branch-VPN esp-3des esp-sha-hmac
!
crypto map Branch-MAP 10 ipsec-isakmp
set peer 209.165.200.242
set transform-set Branch-VPN
match address 101
!
!
!
!
!
!
!
interface Loopback0
description HQ email server address
ip address 10.10.20.238 255.255.255.0
ip nat inside
ip virtual-reassembly
!
interface Loopback1
description Headquarters LAN
ip address 10.10.10.1 255.255.255.0
ip nat inside
ip virtual-reassembly
!
interface FastEthernet0/0
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/0
no ip address
shutdown
clock rate 2000000
!
interface FastEthernet0/1
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/1
description Connection to ISP
bandwidth 64
ip address 209.165.200.226 255.255.255.248
ip nat outside
ip virtual-reassembly
clock rate 64000
crypto map Branch-MAP
!
interface Serial0/2
no ip address
shutdown
clock rate 2000000
!
interface Serial0/3
no ip address
shutdown
clock rate 2000000
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 209.165.200.225
!
!
no ip http server
no ip http secure-server
ip nat pool HQ-NAT-POOL 209.165.200.233 209.165.200.237 prefix-length 29
ip nat inside source list 100 pool HQ-NAT-POOL
ip nat inside source static 10.10.20.238 209.165.200.238
!
access-list 100 deny ip 10.10.0.0 0.0.255.255 192.168.1.0 0.0.0.255
access-list 100 permit ip 10.10.0.0 0.0.255.255 any
access-list 101 permit ip 10.10.0.0 0.0.255.255 192.168.1.0 0.0.0.255
!
!
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
exec-timeout 0 0
line aux 0
line vty 0 4
login
!
!
end

HQ#






ISP#sho run
Building configuration...

Current configuration : 1373 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname ISP
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
memory-size iomem 5
ip cef
!
!
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
archive
log config
hidekeys
!
!
!
!
!
!
!
!
interface Loopback1
description Simulating the Internet
ip address 209.165.202.129 255.255.255.240
!
interface FastEthernet0/0
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/0
description Connection to Branch
bandwidth 64
ip address 209.165.200.241 255.255.255.248
clock rate 64000
!
interface FastEthernet0/1
no ip address
shutdown
duplex auto
speed auto
!
interface Serial0/1
description Connection to HQ
bandwidth 64
ip address 209.165.200.225 255.255.255.248
clock rate 2000000
!
interface Serial0/2
no ip address
shutdown
clock rate 2000000
!
interface Serial0/3
no ip address
shutdown
clock rate 2000000
!
ip forward-protocol nd
ip route 209.165.200.232 255.255.255.248 Serial0/1
ip route 209.165.200.248 255.255.255.248 Serial0/0
!
!
no ip http server
no ip http secure-server
!
access-list 125 permit ip 209.165.200.0 0.0.0.255 192.168.1.0 0.0.0.255
!
!
!
!
!
!
control-plane
!
!
!
!
!
!
!
!
!
!
line con 0
exec-timeout 0 0
line aux 0
line vty 0 4
login
!
!
end

ISP#

pogue

Heh heh.. Caught it before I could reply, wave...

Please remember...Without the static NAT entry, everything works perfect, and the packets from the server are denied in the NAT ACL, matched by the Crypto ACL, and forwarded through the tunnel.

With the static NAT, it seems as if the packet ignores the NAT ACL, therefore arriving at the crypto ACL with a 209.165.200.238 address, so does not match that ACL, and uses the default route out.

I tried to read the order of operations link, but it was a little hard to understand which of many possible ACLs they were referring to as being evaluated first, second, third, etc.

It seemed to say in that link that NATing is done before crypto ACL evaluation? If so, that means in this particular lab, it seems that static NAT mappings ignore any NAT ACL.

Russ

wave

heh, yeah....still thinking this through.

One thing - you're missing ip nat inside here on Branch

interface Loopback1
description Branch LAN
ip address 192.168.1.1 255.255.255.0
ip virtual-reassembly

pogue

wave,

Yeah, I had it there before and was changing things around because I was having a difficult time getting the IPSec portion of the lab to work.. It did not seem to affect the NATing for some reason.

The order of operations page does mention that NAT is performed prior to crypto map ACL evaluation, and I really do get that. That is the reason why the NAT ACL must deny the 10.10.0.0 - 192.168.1.0 traffic, so that it arrives at the crypto map ACL with the private source addresses.

The real question is the order of operations of the NAT internals themselves. Does static NAT get performed "first", (prior to NAT ACL) or does it "override" the NAT ACL? The order of operations page makes no reference to any difference in NAT operation performance between static and dynamic NATing.

Russ

wave

Yeah, I have the GRE tunnel in mine and everything works perfectly even with the static NAT entries. I'm backing the GRE tunnel out of the config now to test yours.

wave

Okay now I'm in the exact same position as you! It's been a while since I did this lab and I can't remember if I skipped past this and went onto the gre tunnel.

My pings aren't making it from 192.168.1.1 to 10.10.20.238 or 10.10.10.1 either.

I'll keep working on it

wave

dammit, I am completely glued to this right now!

wave

Lots of other threads on this on CLN:

https://learningnetwork.cisco.com/thread/29799
https://learningnetwork.cisco.com/thread/30979
https://learningnetwork.cisco.com/message/115148#115148

I'm pretty sure adding the overload keyword solved my problem when I first did this lab, but it doesn't seem to work now

wave

Okay, so I figured out a work-around. There certainly is plenty of comment on this subject in various forums. I'm surprised this made it into the lab manual using the printed configuration.

I can see why it's happening but don't really have the time to dig into it further as my ROUTE exam is two weeks away and I have other areas to focus on.

I simply created a route-map that matches the existing ACL. Then I changed the ip nat inside source statement to use the route-map + the overload keyword on the end. Here's the config:

ip nat pool BRANCH-NAT-POOL 209.165.200.249 209.165.200.254 prefix-length 29
ip nat inside source route-map NATPERMIT pool BRANCH-NAT-POOL overload

route-map NATPERMIT permit 10
match ip address BRANCH-NAT-ACL

ip access-list extended BRANCH-NAT-ACL
remark Do not translate Local LAN to HQ LAN addresses
deny ip 192.168.1.0 0.0.0.255 10.10.0.0 0.0.255.255
remark Translate Local LAN to all other Internet destinations
permit ip 192.168.1.0 0.0.0.255 any
ip access-list extended HQ-VPN-ACL
remark Branch to HQ traffic to trigger VPN
permit ip 192.168.1.0 0.0.0.255 10.10.0.0 0.0.255.255

Branch#ping 10.10.10.1 source 192.168.1.1

BOOM

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.1
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 52/89/104 ms

Interestingly the NAT table shows the EIGRP multicast address as the Outside local/global but EIGRP nor any routing protocol is running on the router:

Branch#sh ip nat trans
Pro Inside global Inside local Outside local Outside global
eigrp 209.165.200.250:0 192.168.1.1:0 224.0.0.10:0 224.0.0.10:0


HQ#debug ip packet
*Mar 1 00:22:16.427: IP: tableid=0, s=192.168.1.1 (Serial0/0), d=10.10.10.1 (Loopback1), routed via RIB
*Mar 1 00:22:16.431: IP: s=192.168.1.1 (Serial0/0), d=10.10.10.1, len 100, rcvd 4
*Mar 1 00:22:16.435: IP: tableid=0, s=10.10.10.1 (local), d=192.168.1.1 (Serial0/0), routed via FIB
*Mar 1 00:22:16.435: IP: s=10.10.10.1 (local), d=192.168.1.1 (Serial0/0), len 100, sending

pogue

wave,

I have been able to ping 10.10.10.1 the entire time.. It is only pinging 10.10.20.238 that does not work, as it appears that the static NAT entry forces the NATing before the NAT ACL can deny it. Does your configuration allow you to ping 10.10.20.238?

The goal is to be able to ping between 192.168.1.1 and the email server, while still being able to reach the email server's public IP from the internet.

Either way, don't worry about it man.. It seems it might be an eccentricity of GNS3, or just an oddity about how static NATs work. I implemented the GRE tunnel through the IPSec tunnel, and it seems to be a much easier way to do the same task..

Thanks for the help..

Russ

wave

pogue wrote: »

wave,

I have been able to ping 10.10.10.1 the entire time.. It is only pinging 10.10.20.238 that does not work, as it appears that the static NAT entry forces the NATing before the NAT ACL can deny it. Does your configuration allow you to ping 10.10.20.238?

Yes indeed. I should have mentioned that you must also tack the route-map onto your static NAT statement.

ip nat inside source static 10.10.20.238 209.165.200.238 route-map NATPERMIT

Branch#ping 10.10.20.238 source 192.168.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.20.238, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 84/90/100 ms

No worries, I enjoyed troubleshooting this. But you're totally right, waaaay easier with a GRE tunnel!