Site-to-Site VPN issues have me befuddled!!!!!!!!

I always seem to do things the most difficult way and have been banging my head for several days with this.

Scenario:

I have set up a site-to-site VPN from my office router (c2821 w/ IOS 15.1(3)T ) to a remote router (c2621XM w/ 12.4(25b) ) both advEnt-k9 images.

I am able to ping the interesting remote private networks from my desktop using the private IP addresses so I know the tunnel is working properly.

The problem is that I am unable to access a particular device (Wireless LAN Controller) using a web browser to enable remote management of the device. I think I know the problem source but before I get to my suspicions here are the associated configs.

c2821 Configs

!
security authentication failure rate 3 log
security passwords min-length 6
logging buffered 4096
enable secret 5 somesecretpassword
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication login ciscocp_vpn_xauth_ml_1 local
aaa authorization exec default local 
aaa authorization network ciscocp_vpn_group_ml_1 local 
!
!
!
!
!
aaa session-id common
!
!
crypto isakmp policy 9
 hash md5
 authentication pre-share
crypto isakmp key somekeyknowtomealone address XXX.XXX.XXX.XXX ! Public IP address at other end of tunnel
!
crypto ipsec transform-set 3deshmac esp-3des esp-sha-hmac 
!
crypto map toQIDenEast 1 ipsec-isakmp 
 set peer XXX.XXX.XXX.XXX
 set transform-set 3deshmac 
 match address HDTVPN1
!
!
interface GigabitEthernet0/0
 description $FW_OUTSIDE$
 ip address YYY.YYY.YYY.YYY SSS.SSS.SSS.SSS
 ip access-group hdtWANsecure in
 no ip redirects
 no ip unreachables
 ip flow ingress
 ip nat outside
 ip virtual-reassembly in
 duplex auto
 speed auto
 crypto map torRemoteSite1
 crypto ipsec df-bit clear
!
!
interface GigabitEthernet0/1.10
 description $FW_INSIDE$ Device Management sub-interface
 encapsulation dot1Q 10 native
 ip address 192.168.254.1 255.255.255.224
 no ip redirects
 no ip unreachables
 ip nat inside
 ip virtual-reassembly in
!
interface GigabitEthernet0/1.11
 description $FW_INSIDE Data Subnet$
 encapsulation dot1Q 11
 ip address 10.10.11.1 255.255.255.224
 ip access-group hdtLANsecure in
 no ip redirects
 no ip unreachables
 ip nat inside
 ip virtual-reassembly in
!
!
!
!
ip nat inside source static udp 10.10.11.11 69 interface GigabitEthernet0/0 69
ip nat inside source static udp 192.168.254.5 162 interface GigabitEthernet0/0 162
ip nat inside source static udp 192.168.254.5 514 interface GigabitEthernet0/0 514
ip nat inside source static tcp 192.168.254.5 1645 interface GigabitEthernet0/0 1645
ip nat inside source static tcp 192.168.254.5 1646 interface GigabitEthernet0/0 1646
ip nat inside source route-map SDM_RMAP_1 interface GigabitEthernet0/0 overload
ip route 0.0.0.0 0.0.0.0 nex.tho.p.rtr
!
ip access-list extended HDT_NAT
 deny   ip 10.10.11.0 0.0.0.31 192.168.255.0 0.0.0.7
 deny   ip 10.10.11.0 0.0.0.31 10.0.100.0 0.0.0.255
 permit ip 192.168.255.0 0.0.0.31 any
 permit ip 172.16.0.4 0.0.0.1 any
 permit ip 10.10.11.64 0.0.0.31 any
 permit ip 10.10.11.32 0.0.0.31 any
 permit ip 10.10.11.0 0.0.0.31 any
 permit ip 10.10.102.0 0.0.1.255 any
 permit ip 10.10.100.0 0.0.0.63 any
ip access-list extended hdtLANsecure
 remark Apply to Gi0/1.11 as ip access-group in
 remark CCP_ACL Category=17
 permit udp host 10.10.11.3 eq domain any
 permit ip 10.10.11.0 0.0.0.31 any
 deny   ip any any
ip access-list extended hdtVTYsecure
 remark ACL applied to line vty 0 4 as access-class hdtVTYsecure in
 remark CCP_ACL Category=17
 permit tcp 10.10.11.0 0.0.0.255 any eq 22
 permit tcp 10.10.11.0 0.0.0.255 any eq telnet
 permit tcp 172.16.0.0 0.0.0.15 any eq 22
 permit tcp 172.16.0.0 0.0.0.15 any eq telnet
 permit tcp 192.168.255.0 0.0.0.255 any eq 22
 permit tcp 192.168.255.0 0.0.0.255 any eq telnet
 deny   ip any any
ip access-list extended hdtWANsecure
 remark Apply to Gi0/0 as ip access-group in
 remark CCP_ACL Category=17
 permit ip 192.168.255.0 0.0.0.7 10.10.11.0 0.0.0.31
 permit ip 10.0.100.0 0.0.0.255 10.10.11.0 0.0.0.31
 permit udp host XXX.XXX.XXX.XXX host YYY.YYY.YYY.YYY eq non500-isakmp
 permit udp host XXX.XXX.XXX.XXX host YYY.YYY.YYY.YYY eq isakmp
 permit esp host XXX.XXX.XXX.XXX host YYY.YYY.YYY.YYY
 permit ahp host XXX.XXX.XXX.XXX host YYY.YYY.YYY.YYY
 permit udp host 8.8.8.4 eq domain any
 permit udp host 8.8.8.8 eq domain any
 deny   ip 10.0.0.0 0.255.255.255 any log
 deny   ip 127.0.0.0 0.255.255.255 any log
 deny   ip 172.16.0.0 0.15.255.255 any log
 deny   ip 192.168.0.0 0.0.255.255 any log
 permit udp any 10.0.0.0 0.63.255.255 eq ntp
 permit tcp any 10.10.11.0 0.0.0.31 eq 5070
 permit udp any 10.10.11.0 0.0.0.31 eq 5060
 permit ip any any
ip access-list extended hdtWAPmgmtSecure
 remark Apply to F0/1.101 as ip access-group hdtWAPmgmtSecure in
 permit ip 10.10.100.0 0.0.1.63 any
 deny   ip any any
ip access-list extended HDTVPN1
 permit ip 10.10.11.0 0.0.0.31 10.0.100.0 0.0.0.255
 permit ip 10.10.11.0 0.0.0.31 192.168.255.0 0.0.0.7
!
!
no route-map SDM_RMAP_1 permit 1
route-map SDM_RMAP_1 permit 1
 match ip address HDT_NAT
!

c2621XM Router at remote location

!
aaa new-model
!
!
aaa authentication login default local
aaa authorization exec default local 
!
aaa session-id common
!
!
crypto isakmp policy 9
 hash md5
 authentication pre-share
crypto isakmp key somekeyknowtomealone address ZZZ.ZZZ.ZZZ.ZZZ
!
crypto ipsec security-association lifetime seconds 86400
!
crypto ipsec transform-set 3deshmac esp-3des esp-sha-hmac 
!
crypto map toHDT 1 ipsec-isakmp 
 set peer ZZZ.ZZZ.ZZZ.ZZZ
 set transform-set 3deshmac 
 match address DenEastVPN
!
!
!
!
interface FastEthernet0/0
 description Connection to ISP through Workgroup Switch $FW_OUTSIDE$
 ip address uuu.uuu.uuu.uuu SSS.SSS.SSS.SSS
 ip access-group secureISP in
 ip verify unicast reverse-path
 ip nat outside
 ip virtual-reassembly
 duplex auto
 speed auto
 crypto map toHDT
 crypto ipsec df-bit clear
!
interface FastEthernet0/1
 no ip address
 duplex auto
 speed auto
!
!
interface FastEthernet0/1.11
 description $FW_DMZ$ Device Management sub-interface
 encapsulation dot1Q 11 native
 ip address 192.168.255.1 255.255.255.248
 ip nat inside
 ip virtual-reassembly
!
interface FastEthernet0/1.100
 description $FW_INSIDE$ WLAN Management sub-interface
 encapsulation dot1Q 100
 ip address 10.0.100.1 255.255.255.0
 ip access-group WLANMgnt in
 ip nat inside
 ip inspect SDM_LOW in
 ip virtual-reassembly
!
!ip nat inside source route-map SDM_RMAP_1 interface FastEthernet0/0 overload
ip nat inside source static tcp 10.0.100.21 80 interface FastEthernet0/0 10000
!
!
ip access-list extended Nat
 deny   ip 192.168.255.0 0.0.0.7 10.10.11.0 0.0.0.31
 deny   ip 10.0.100.0 0.0.0.255 10.10.11.0 0.0.0.31
 permit ip 10.0.102.0 0.0.1.255 any
 permit ip 10.0.100.0 0.0.0.255 any
 permit ip 192.168.255.0 0.0.0.7 any
 permit ip 172.16.0.0 0.0.0.255 any
!
ip access-list extended WLANMgnt
 remark Apply to F0/1.100 as ip access-group WLANMgnt
 permit udp host 192.43.244.18 eq ntp host 10.0.100.1 eq ntp
 deny   ip 10.0.0.0 0.0.1.255 any
 deny   ip 192.168.255.0 0.0.0.7 any
 deny   ip 172.16.0.0 0.0.0.255 any
 deny   ip 65.112.204.112 0.0.0.7 any
 deny   ip host 255.255.255.255 any
 deny   ip 127.0.0.0 0.255.255.255 any
 permit ip any any
ip access-list extended secureISP
 remark Apply to F0/0 as ip access-group secureISP in
 remark SDM_ACL Category=17
 permit ip 10.10.11.0 0.0.0.31 192.168.255.0 0.0.0.7
 permit ip 10.10.11.0 0.0.0.31 10.0.100.0 0.0.0.255
 permit udp host YYY.YYY.YYY.YYY host XXX.XXX.XXX.XXX eq non500-isakmp
 permit udp host YYY.YYY.YYY.YYY host XXX.XXX.XXX.XXX eq isakmp
 permit esp host YYY.YYY.YYY.YYY host XXX.XXX.XXX.XXX
 permit ahp host YYY.YYY.YYY.YYY host XXX.XXX.XXX.XXX
 permit udp host 192.43.244.18 eq ntp any eq ntp
 permit udp host YYY.YYY.YYY.YYY any log
 permit icmp any XXX.XXX.XXX.XXX echo-reply
 permit icmp any host XXX.XXX.XXX.XXX time-exceeded
 permit icmp any host XXX.XXX.XXX.XXX unreachable
 permit tcp any host XXX.XXX.XXX.XXX eq 443
 permit tcp any host XXX.XXX.XXX.XXX eq 10000 log
 permit tcp any host XXX.XXX.XXX.XXX eq 22
 permit tcp any host XXX.XXX.XXX.XXX eq cmd
 permit tcp any host 192.168.255.4 eq 443 log
 deny   ip 10.0.0.0 0.255.255.255 any
 deny   ip 127.0.0.0 0.255.255.255 any
 deny   ip 172.16.0.0 0.0.255.255 any
 deny   ip 192.168.0.0 0.0.255.255 any
 deny   ip host 255.255.255.255 any
 deny   ip host 0.0.0.0 any
 deny   ip any any log
ip access-list extended DenEastVPN
 permit ip 10.0.100.0 0.0.0.255 10.10.11.0 0.0.0.31
 permit ip 192.168.255.0 0.0.0.7 10.10.11.0 0.0.0.31
!
route-map SDM_RMAP_1 permit 1
 match ip address Nat
no match ip address 102 Nat

!
route-map SDM_RMAP_1 permit 1
 match ip address Nat

I have searched for an answer on Cisco and many other sources and have not been able to find anything to define the behavior.

I suspect the problem is in the c2621XM's config concerning this line:

ip nat inside source static tcp 10.0.100.21 80 interface FastEthernet0/0 10000

Remote access to the device (Wireless LAN Controller) currently uses port forwarding.
So, the questions:

Is my suspicion correct? If so could someone provide a link to documentation?
Is there a way to be able to use port forwarding and VPN access together, or even including access via VPN Client Software?
- Route Maps?
- Other methods?

I’m not looking for a handout on this, just direction. Normally, I would LAB this to achieve a proof of concept, but I am in the process of re-working/re-cabling my LAB (several days work).
I will be adding additional functionality in the future (RADIUS/TACACS+, Port Monitoring, etc) that will depend on this VPN connection(s) so I am trying to lay the groundwork for that functionality.

Thanks,

Find more posts tagged with

Comments

SteveO86

Does the VPN stay up the entire time or does it tear itself down and rebuild itself? (Sh cry isa sa and watch the connection ID, I've seen it were a few pings will succeed while the tunnel is up but then just tears itself down)

Is the WLC on the same subnet your VPN'ed into on the remote side? If not ping default gateways, then neighboring switch just to see how far you get.

Is their any management restrictions on the WLC? (As far as what subnets can access it)

hermeszdata

I am able to access the Router interfaces (10.0.100.1, 192.168.255.1) using HTTPS via the VPN and I am also able to configure the router using SMD over the VPN (same IP addresses).

I am able to console into the WLC using its service port interface (192.168.255.4 when consoled into the router @ 192.168.255.1)) and the Management interface (10.0.100.21) with a direct VPN connection. This is the expected behavior. (Note: The WLC Service Port is not a routable interface and access to the port MUST be from a device on the same subnet, 192.168.255.0/29.)

HTTPS is intentionally disabled on the WLC due to another issue (WLC does not support Chained SSL Certificates and Un-chained Certificates are no longer available) which is one of the primary reasons for implementing a VPN solution for primary remote management and monitoring.

Management via GUI is on ip address 10.0.100.21 (the WLC management interface) and attempts at HTTP access via IE time out with "Page cannot be displayed"

3-Way Monitoring (console connection to the 2821 in my office, SSH connection to the remote 2621XM, both seperate windows using SecureCrt with debug crypto ipsec enabled, and SDM connection to 10.0.100.1, the default gateway for the WLC's Management interface) show the tunnel is up, stable and without errors.

The above are the reasons I suspect the problem is associated with port forwarding to the WLC's Management IP address (10.0.100.21). I cannot disable port forwarding to the WLC because Quest support currently depends on this for monitoring and support (they currently have Lobby privileges only, no admin privileges.) I do have a duplicate WLC in my office, but a s mentioned in my OP, I am in the process of re-cabling my LAB Rack and this will take a few days to complete.

Jollycork

Have you tried it without any ACLs?

Just setup the Site to Site VPN between the 2 routers, establish the tunnel and then use the tunnel. No ACLs.

once you have that, you've narrowed down the problem to, probably, your ACLs.

Also Note: site to site VPN is a tunnel that both routers establish and maintain. Hosts on both sides access the tunnel to exchange data. Like any network, if hosts on either side are in 2 different subnets, hosts on one subnet, can't talk to hosts on another without some help.

[helper address]

hermeszdata

The issue is definitely associated with this line in the config

ip nat inside source static tcp 10.0.100.21 80 interface FastEthernet0/0 10000

Once I remove it the VPN issue goes away.

For other reasons I need to keep port forwarding to the device available as a backup.

Have to find another way.

Thanks

hermeszdata

Gotta love head banging.

As I stated earlier, the issue was with the port forwarding!

ip nat inside source static tcp 10.0.100.21 80 interface f0/0 10000

The resolution follows.

! Delete the original port forwarding statement to the WLC
no ip nat inside source static tcp 10.0.100.21 80 interface f0/0 10000
!
ip access-list extended DENY_NAT
 remark access list to deny VPN traffic 
 deny   ip 10.0.100.0 0.0.0.255 10.10.11.0 0.0.0.31
 deny   ip 192.168.255.0 0.0.0.7 10.10.11.0 0.0.0.31
exit
!
route-map NAT_Deny_VPN permit 10
 remark route-map to apply to the new /replacement port forwarding statement
 match ip address DENY_NAT
exit
!
! Port Forwarding statement using public IP address vs. Interface and the newly created route-map
ip nat inside source static tcp 10.0.100.21 80 XXX.XXX.XXX.XXX 10000 NAT_Deny_VPN extendable

Giving credit where due, I hound the solution here: Problutions.com » Route Map No NAT – VPN » Problutions.com

I also added the following to ingress and egress ACLs (Extended Named) on both routers.

 permit tcp any gt 0 any gt 0 log
 permit udp any gt 0 any gt 0 log
 permit icmp any any

This allowed me to see the ACL traps and further isolate the source of the porblem.