Implementing Site-to-Site IPsec VPN with crypto map (Part 1)

In this lab we're going to establish secure communication between two private subnets located at two different sites over an unsecure network (the internet for example) using IPsec. Here is the topology we're going to configure:

Routing and prerequisites

We're going to create a secure tunnel between Site 1 and Site 2 and tunnel the traffic between the subnet 192.168.1.0/24 and 172.16.2.0/24. The routers between Site 1 and Site 2 don't have to know these subnets, these can be RFC 1918 private addresses, they are not advertised towards the internet. The tunnel will be built between the public WAN address of R1 (100.12.0.1) and the WAN address of R3 (100.23.0.3). The only requirement is that R1 and R3 must have reachability to each other over the unsecure network, which is AS 65002 in this example. For the sake of simplicity I use BGP, and just advertise a default route from R2 to both R1 and R2:

R2#show run | sec bgp
router bgp 65002
 bgp log-neighbor-changes
 neighbor 100.12.0.1 remote-as 65001
 neighbor 100.12.0.1 default-originate
 neighbor 100.23.0.3 remote-as 65003
 neighbor 100.23.0.3 default-originate

R2 only has connected routes in his routing table, it doesn't know the 192.168.1.0/24 and 172.16.2.0/24 subnets. R1 and R3 only have a default route pointing towards R2:

R1#show bgp ipv4 uni | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>   0.0.0.0          100.12.0.2                             0 65002 i

R3#show bgp ipv4 uni | begin Network
     Network          Next Hop            Metric LocPrf Weight Path
 *>   0.0.0.0          100.23.0.2                             0 65002 i

Defining the Phase 1 policy

IPsec VPNs are negotiated in two phases. In Phase 1 the peers must agree on four main attributes: authentication, encryption, hash, and Diffie-Hellman (DH) Group. These attributes are used to build the Phase 2 tunnel, not to protect the actual data packets.

R1(config)#crypto isakmp policy 10
R1(config-isakmp)#encryption aes 256
R1(config-isakmp)#authentication ?
  pre-share  Pre-Shared Key
  rsa-encr   Rivest-Shamir-Adleman Encryption
  rsa-sig    Rivest-Shamir-Adleman Signature

R1(config-isakmp)#authentication pre-share 
R1(config-isakmp)#hash sha256
R1(config-isakmp)#group 19
R1(config)#crypto isakmp key cisco123 address 100.23.0.3

We can have multiple Phase 1 policies, here I defined the attributes under priority 10. The priorities and the policies are checked incrementally when the remote peer initiates the session, and starts to establish the Phase 1 tunnel. The priorities don't have to match, but we must have a matching entry with the same four attributes (encryption, authentication, hash, DH). Optionally we can also define a lifetime under the policy (the default value is 86400 seconds), the lifetimes don't have to match, the lower value will be negotiated. In this lab we use AES with a 256 bit key for encryption which meets today's standards, alternatively we could also use DES or 3DES (these are obsolete, legacy algorithms, don't use these). For hashing we use SHA-256 which creates a 32 byte message digest, and for key exchange we use DH Group 19. For authentication we use a pre-shared key here, alternatively we could also use certificates (we need to set up a PKI). Finally with the last command we define the pre-shared key (cisco123) with the public WAN address of our remote peer (100.23.0.3).

Defining the Phase 2 policy

We do this by creating a transform-set:

R1(config)#crypto ipsec transform-set TRANS-SET ?
  ah-md5-hmac      AH-HMAC-MD5 transform
  ah-sha-hmac      AH-HMAC-SHA transform
  ah-sha256-hmac   AH-HMAC-SHA256 transform
  ah-sha384-hmac   AH-HMAC-SHA384 transform
  ah-sha512-hmac   AH-HMAC-SHA512 transform
  comp-lzs         IP Compression using the LZS compression algorithm
  esp-3des         ESP transform using 3DES(EDE) cipher (168 bits)
  esp-aes          ESP transform using AES cipher
  esp-des          ESP transform using DES cipher (56 bits)
  esp-gcm          ESP transform using GCM cipher
  esp-gmac         ESP transform using GMAC cipher
  esp-md5-hmac     ESP transform using HMAC-MD5 auth
  esp-null         ESP transform w/o cipher
  esp-seal         ESP transform using SEAL cipher (160 bits)
  esp-sha-hmac     ESP transform using HMAC-SHA auth
  esp-sha256-hmac  ESP transform using HMAC-SHA256 auth
  esp-sha384-hmac  ESP transform using HMAC-SHA384 auth
  esp-sha512-hmac  ESP transform using HMAC-SHA512 auth

R1(config)#crypto ipsec transform-set TRANS-SET esp-aes 256 esp-sha-hmac 

Look at how many options we have here. IPsec is just a framework, we can actually choose the encryption algorithm, the hashing etc. We have basically two implementation to choose from: Encapsulating Security Payload (ESP) or Authentication Header (EH), or actually we can use both together if we want. Both provide integrity, authentication and anti-replay services, ESP also provides encryption. In this example we use ESP for encryption (AES-256) and for integrity check (HMAC-SHA).

Now we move on to R3, and copy all of these:

R1(config)#do show run | sec crypto isakmp pol|crypto ipsec
crypto isakmp policy 10
 encr aes 256
 hash sha256
 authentication pre-share
 group 19
crypto ipsec transform-set TRANS-SET esp-aes 256 esp-sha-hmac 
 mode tunnel

These commands are going to be the same on both R1 and R3. Notice the mode tunnel command under the transform-set: I didn't issue this command, but this is the default. IPsec can transport the packets in two modes: in ether tunnel or in transport mode. The difference is that tunnel mode encapsulates the entire IP packet (with the IP header included) and creates a new IP header. If we use encryption this also encrypts the original source and destination IP addresses as well. With transport mode we retain the original IP header and the AH/ESP headers a placed after the original IP header. We define the PSK also on R3 with the WAN address (100.12.0.1) of R1:

R3(config)#crypto isakmp key cisco123 address 100.12.0.1

Defining the interesting traffic (proxy ACL)

Next we create an ACL and define the traffic we want to protect and send though the IPsec tunnel: on R1 this is basically any IP traffic sourced from the subnet 192.168.1.0/24 and destined to subnet 172.16.2.0/24:

R1(config)#ip access-list extended S2S_TRAFFIC 
R1(config-ext-nacl)#permit ip 192.168.1.0 0.0.0.255 172.16.2.0 0.0.0.255

On R3 the ACL is going to be the mirror image of this: any IP traffic sourced from 172.16.2.0/24 and destined 192.168.1.0/24:

R3(config)#ip access-list extended S2S_TRAFFIC
R3(config-ext-nacl)#permit ip 172.16.2.0 0.0.0.255 192.168.1.0 0.0.0.255

Create the crypto map

We basically put all of these into a container which is called the crypto map:

R1(config)#crypto map S2S_VPN 1 ipsec-isakmp 
% NOTE: This new crypto map will remain disabled until a peer
        and a valid access list have been configured.
R1(config-crypto-map)#set peer 100.23.0.3 
R1(config-crypto-map)#set transform-set TRANS-SET
R1(config-crypto-map)#match address S2S_TRAFFIC

R3(config)#crypto map S2S_VPN 1 ipsec-isakmp 
% NOTE: This new crypto map will remain disabled until a peer
        and a valid access list have been configured.
R3(config-crypto-map)#set peer 100.12.0.1
R3(config-crypto-map)#set transform-set TRANS-SET
R3(config-crypto-map)#match address S2S_TRAFFIC

Under the crypto map we have to set the WAN address of the remote peer, the transform-set and the proxy ACL. The downside of this crypto map solution is that it is very static: dynamic routing is not supported through the tunnel, the crypto map works as a data plane filter.

Apply the crypto map to the interface

And finally we apply the crypto map to the actual physical interface which is used to send the traffic to the remote site (crypto maps are always applied outbound):

R1(config)#interface gigabitEthernet 0/0
R1(config-if)#crypto map S2S_VPN
%CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
%LINEPROTO-5-UPDOWN: Line protocol on Interface NVI0, changed state to up

R3(config)#int gigabitEthernet 0/0
R3(config-if)#crypto map S2S_VPN
%CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON
%LINEPROTO-5-UPDOWN: Line protocol on Interface NVI0, changed state to up

Verification and troubleshooting

The IPsec VPN tunnel initiation is only triggered when the matching traffic (proxy ACL) is sent between between Site 1 and Site 2. Note that both R1 and R3 must have a route to the remote subnet (192.168.1.0/24 and 172.16.2.0/24 respectively) in their RIB out of their G0/0 interface which the crypto map was applied on. This can be just a default route as in this example. The routers between R1 and R3 don't need to have matching entries for these private subnets, they only have to route the packets to the WAN interface of R1 and R3. To test the communication between the two sites I'm going to ping R5 (172.16.2.5) from R4 (192.16.1.4), both have a default route to R1 and R3 respectively:

R4(config)#ip route 0.0.0.0 0.0.0.0 192.168.1.1
R5(config)#ip route 0.0.0.0 0.0.0.0 172.16.2.3

And I issue debugs on both R1 and R3 to see how the tunnels are going to be built.

R1#debug crypto isakmp 
Crypto ISAKMP debugging is on
R3#debug crypto isakmp 
Crypto ISAKMP debugging is on

First the routers establish the Phase 1 (ISAKMP) tunnel. This can be done in two modes: we either use Main Mode (MM, 6 message exchange, default) or Aggressive Mode (AM, 3 message exchange). AM is less secure and not recommended in general because the authentication hash is sent in clear text in the second message. At the end of Phase 1 negotiation we'll have an IKE SA. For the Phase 2 negotiation we use Quick Mode (QM) with 3 message exchange. Notice that at this point the ISAKMP tunnel is already established, so everything is encrypted at this point, we use this tunnel to protect the Phase 2 negotiation. In Phase 2 the peers negotiate how the actual data packets are going to be protected on the data plane (ESP/AH). Phase 2 negotiation results two unidirectional IPsec SA.

The ESP packets with SPI (Security Parameter Index) values are actual data packets (ICMP Echoes in this case) encrypted. All we can see in the packet capture is the SPI values, these are unique 32-bit values in the ESP header which the routers create to identify the Security Associations (SAs). The routers can have multiple peers and tunnels, they need the SPI to determine which encryption and hashing algorithm they should use to decode the packet. 

ISAKMP uses UDP port 500 both as a source and destination, if there is a NAT device between the two peers the port number changes to 4500 (NAT traversal). The first message is in clear text, here we can actually see the four attributes we've defined for the Phase 1 policy: encryption, hashing, DH group, authentication and lifetime (we use the default value of 84600 seconds). The initiator (R1 in this example) proposes his own policy, the responder (R3) replies (second message) with the policy he agrees to use.

These are the debug messages on R1 (the output of the debug crypto isakmp is extremely detailed, so I only show the most relevant parts):

ISAKMP: (0):Created a peer struct for 100.23.0.3, peer port 500
ISAKMP: (0):New peer created peer = 0xD1258D0 peer_handle = 0x80000002
ISAKMP: (0):Locking peer struct 0xD1258D0, refcount 1 for isakmp_initiator
ISAKMP: (0):local port 500, remote port 500
ISAKMP: (0):set new node 0 to QM_IDLE      
ISAKMP: (0):insert sa successfully sa = DB59ED0
ISAKMP: (0):Can not start Aggressive mode, trying Main mode.
ISAKMP: (0):found peer pre-shared key matching 100.23.0.3
<output omitted>
ISAKMP: (0):beginning Main Mode exchange
ISAKMP-PAK: (0):sending packet to 100.23.0.3 my_port 500 peer_port 500 (I) MM_NO_STATE
ISAKMP: (0):Sending an IKE IPv4 Packet.
ISAKMP-PAK: (0):received packet from 100.23.0.3 dport 500 sport 500 Global (I) MM_NO_STATE
<output omitted>
ISAKMP: (0):processing SA payload. message ID = 0
ISAKMP: (0):processing vendor id payload
ISAKMP: (0):vendor ID seems Unity/DPD but major 69 mismatch
ISAKMP: (0):vendor ID is NAT-T RFC 3947
ISAKMP: (0):found peer pre-shared key matching 100.23.0.3
ISAKMP: (0):local preshared key found
ISAKMP: (0):Scanning profiles for xauth ...
ISAKMP: (0):Checking ISAKMP transform 1 against priority 10 policy
ISAKMP: (0):      encryption AES-CBC
ISAKMP: (0):      keylength of 256
ISAKMP: (0):      hash SHA256
ISAKMP: (0):      default group 19
ISAKMP: (0):      auth pre-share
ISAKMP: (0):      life type in seconds
ISAKMP:      life duration (VPI) of  0x0 0x1 0x51 0x80 
ISAKMP: (0):atts are acceptable. Next payload is 0

The second message is the same ISAKMP Proposal message from R3 (100.23.0.3), R3 sends his attributes to R1, and if they match they can proceed and negotiate the Phase 1 tunnel. In message three and four they do the DH key exchange (KE), this is a very complex mathematical operation, they send nonces to each other and generate large prime numbers. With the help of the DH key exchange the peers create symmetric keys over the unsecure medium (internet for example) which they can use to encrypt any subsequent packets.

ISAKMP: (0):processing KE payload. message ID = 0
ISAKMP: (0):processing NONCE payload. message ID = 0
ISAKMP: (0):found peer pre-shared key matching 100.23.0.3
<output omitted>
ISAKMP-PAK: (1001):sending packet to 100.23.0.3 my_port 500 peer_port 500 (I) MM_KEY_EXCH
<output omitted>
ISAKMP-PAK: (1001):received packet from 100.23.0.3 dport 500 sport 500 Global (I) MM_KEY_EXCH

At end of this (after message four) they come up with the same shared secret key which they use to encrypt the subsequent packets.

Message five and six are already encrypted, this is where the peers authenticate each other using certificates or PSK:

ISAKMP: (1001):SA authentication status:
        authenticated
ISAKMP: (1001):SA has been authenticated with 100.23.0.3
ISAKMP: (0):Trying to insert a peer 100.12.0.1/100.23.0.3/500/, 
ISAKMP: (0): and inserted successfully D1258D0.

After that the peers build the Phase 2 (IPsec) tunnel using Quick Mode (these messages are already encrypted in Wireshark):

ISAKMP-PAK: (1001):received packet from 100.23.0.3 dport 500 sport 500 Global (I) QM_IDLE      
ISAKMP: (1001):processing HASH payload. message ID = 2959809146
ISAKMP: (1001):processing SA payload. message ID = 2959809146
ISAKMP: (1001):Checking IPSec proposal 1
ISAKMP: (1001):transform 1, ESP_AES 
ISAKMP: (1001):   attributes in transform:
ISAKMP: (1001):      encaps is 1 (Tunnel)
ISAKMP: (1001):      SA life type in seconds
ISAKMP: (1001):      SA life duration (basic) of 3600
ISAKMP: (1001):      SA life type in kilobytes
ISAKMP:      SA life duration (VPI) of  0x0 0x46 0x50 0x0 
ISAKMP: (1001):      authenticator is HMAC-SHA
ISAKMP: (1001):      key length is 256
ISAKMP: (1001):atts are acceptable.

Here is where they negotiate how they protect the actual data packets (transform-set, which has to match): we use ESP with AES-256 for encryption and HMAC-SHA for hashing.

Since we use AES-256 for encryption this is how the actual data packets (ICMP Echo) look like: 

In the new IP header the Protocol field is set to 50 indicating that this is an ESP packet. All we can see is the unique SPI value which R3 uses to identify the SA and decrypt the data packets.

And just to prove that IPsec doesn't necessarily mean encryption let's modify the transform-set on both sides and use only AH without encryption:

R1(config)#crypto ipsec transform-set TRANS_SET ah-sha256-hmac
R3(config)#crypto ipsec transform-set TRANS-SET ah-sha256-hmac

Now we just use SHA-256 for hashing and integrity check. This is how our data packets look like in Wireshark:

As it can be seen above, Wireshark now could identify and display our actual data packets. Because we use tunnel mode (that's the default) our original IP packet (SRC: 192.168.1.4, DST: 172.16.2.5) is encapsulated in a new IP packet. In the new IP header (SRC: 100.12.0.1, DST: 100.23.0.3) the Protocol field is now set to 51 indicating that we use AH to protect the data packet. The ICV (Integrity Check Value) field is used to send hash value of the data packet which the remote peer also calculates separately and compares to this value. So we can use AH if don't want to encrypt our data packets, but we want to ensure that our packets are not modified in transit.

Look at the SPI value, this is the SPI value of the outbound SA on R1, we can verify this with the following command:

R1#show crypto ipsec sa

interface: GigabitEthernet0/0
    Crypto map tag: S2S_VPN, local addr 100.12.0.1

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (192.168.1.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (172.16.2.0/255.255.255.0/0/0)
   current_peer 100.23.0.3 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 104, #pkts encrypt: 104, #pkts digest: 104
    #pkts decaps: 104, #pkts decrypt: 104, #pkts verify: 104
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0

     local crypto endpt.: 100.12.0.1, remote crypto endpt.: 100.23.0.3
     plaintext mtu 1452, path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0
     current outbound spi: 0x970D0032(2534211634)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
          
     inbound ah sas:
      spi: 0x790108EE(2030110958)
        transform: ah-sha256-hmac ,
        in use settings ={Tunnel, }
        conn id: 1, flow_id: SW:1, sibling_flags 80004050, crypto map: S2S_VPN
        sa timing: remaining key lifetime (k/sec): (4335687/3377)
        replay detection support: Y
        Status: ACTIVE(ACTIVE)

     inbound pcp sas:

     outbound esp sas:

     outbound ah sas:
     spi: 0x970D0032(2534211634)
        transform: ah-sha256-hmac ,
        in use settings ={Tunnel, }
        conn id: 2, flow_id: SW:2, sibling_flags 80004050, crypto map: S2S_VPN
        sa timing: remaining key lifetime (k/sec): (4335687/3377)
        replay detection support: Y
        Status: ACTIVE(ACTIVE)

     outbound pcp sas:

Notice that the SAs are unidirectional, we also have an inbound AH SA with the SPI of 0x790108EE, R3 uses this value when it sends an AH packet to R1.

This post has already become too long, so let's continue further IPsec troubleshooting in the next post.