Fix IPsec Tunnel Not Establishing

What's Actually Happening

IPsec VPN tunnel fails to establish between peers. Security association (SA) negotiation fails or tunnel doesn't pass traffic.

The Error You'll See

```bash $ ipsec status

Security Associations (0 up, 0 connecting): NONE ```

IKE negotiation failure:

IKE negotiation failed: AUTHENTICATION_FAILED

Phase 1 error:

IKE SA negotiation failed: no proposal chosen

Phase 2 error:

IPsec SA negotiation failed: policy mismatch

Connection timeout:

max number of retransmissions reached

Why This Happens

1. 1.Key mismatch - Pre-shared keys don't match
2. 2.Proposal mismatch - Encryption/hash algorithms differ
3. 3.Policy mismatch - Traffic selectors don't align
4. 4.Network blocking - UDP 500/4500 blocked
5. 5.NAT traversal issues - NAT-T not configured
6. 6.Certificate problems - Invalid or expired certificates

Step 1: Check IPsec Status

```bash # Check StrongSwan status: ipsec status ipsec statusall

# Check IKEv2 status: swanctl --list-sas

# Check connections: ipsec status | grep CONN

# Check all SAs: ip xfrm state ip xfrm policy

# Check IPsec process: ps aux | grep ipsec

# Check service status: systemctl status strongswan systemctl status ipsec

# Check logs: journalctl -u strongswan -f tail -f /var/log/syslog | grep ipsec

# Enable debug logging: # In /etc/strongswan/strongswan.conf: charon { debug { ike = 2 cfg = 2 net = 2 enc = 2 } }

# Restart service: systemctl restart strongswan ```

Step 2: Check Network Connectivity

```bash # Test peer reachable: ping peer-ip

# Check IKE ports: nc -zuv peer-ip 500 nc -zuv peer-ip 4500

# Using nmap: nmap -sU -p 500,4500 peer-ip

# Check ESP traffic (protocol 50): # IPsec uses ESP (Encapsulating Security Payload) # Protocol number 50

# Check NAT-T port: # UDP 4500 for NAT traversal

# Check firewall: iptables -L -n | grep -E "500|4500"

# Allow IKE ports: iptables -I INPUT -p udp --dport 500 -j ACCEPT iptables -I INPUT -p udp --dport 4500 -j ACCEPT

# Allow ESP: iptables -I INPUT -p esp -j ACCEPT

# Allow AH (if used): iptables -I INPUT -p ah -j ACCEPT

# Using ufw: ufw allow 500/udp ufw allow 4500/udp

# Using firewalld: firewall-cmd --add-port=500/udp --permanent firewall-cmd --add-port=4500/udp --permanent firewall-cmd --reload

# Check NAT traversal needed: # If either side has NAT, need NAT-T

# Force NAT-T: # In config: force_nat_t = yes ```

Step 3: Verify Pre-Shared Keys

```bash # Check PSK configuration: # In /etc/ipsec.secrets or /etc/strongswan/ipsec.secrets

# PSK format: : PSK "secret_key_here"

# Or specific peer: peer-ip : PSK "secret_key"

# Check secrets file: cat /etc/ipsec.secrets

# Verify key format: # Both peers must have identical PSK # Case-sensitive, no extra spaces

# Common errors: # 1. Different keys on each peer # 2. Wrong IP in secrets # 3. Extra whitespace/quotes

# Fix secrets: echo "peer-ip : PSK \"my_secret_key\"" >> /etc/ipsec.secrets

# Reload secrets: ipsec rereadsecrets

# Or restart: systemctl restart strongswan

# Test key: # Compare keys on both sides # Must be exactly identical

# For IKEv2 with EAP: : EAP "password" ```

Step 4: Check Proposal Configuration

```bash # IKE (Phase 1) proposals: # Both peers must agree on algorithms

# Check IKE proposals: ipsec statusall | grep IKE

# Common proposals: # Encryption: AES-256, AES-128, 3DES # Hash: SHA256, SHA1, MD5 # DH Group: 14 (2048), 15 (3072), 16 (4096), 2 (1024)

# StrongSwan config: # In /etc/ipsec.conf or swanctl.conf:

# IKE proposal: ike=aes256-sha256-modp2048

# ESP proposal: esp=aes256-sha256

# Example configuration: conn myvpn keyexchange=ikev2 ike=aes256-sha256-modp2048! esp=aes256-sha256! left=192.168.1.1 leftsubnet=10.0.0.0/24 right=peer-ip rightsubnet=192.168.2.0/24

# Common mismatches: # 1. Different encryption algorithms # 2. Different hash algorithms # 3. Different DH groups

# Use weaker proposal for compatibility: ike=aes128-sha1-modp1024

# Or multiple proposals: ike=aes256-sha256-modp2048,aes128-sha1-modp1024

# For IKEv1: keyexchange=ikev1

# For IKEv2: keyexchange=ikev2 ```

Step 5: Check Traffic Policies

```bash # Phase 2 (IPsec SA) requires matching policies

# Check traffic selectors: ipsec status | grep subnet

# Left subnet (local): leftsubnet=10.0.0.0/24

# Right subnet (remote): rightsubnet=192.168.2.0/24

# Must mirror on peer: # Peer's leftsubnet = your rightsubnet # Peer's rightsubnet = your leftsubnet

# Common errors: # 1. Subnet mismatch # 2. Wrong direction # 3. Missing subnet

# For host-to-host: leftsubnet=192.168.1.1/32 rightsubnet=peer-ip/32

# For site-to-site: leftsubnet=10.0.0.0/24,172.16.0.0/24

# Check policy in kernel: ip xfrm policy list

# Expected output: # src 10.0.0.0/24 dst 192.168.2.0/24 # dir out tmpl src local dst remote

# Manual policy (if needed): ip xfrm policy add src 10.0.0.0/24 dst 192.168.2.0/24 dir out tmpl src local dst remote proto esp reqid 1

# Check xfrm state: ip xfrm state list

# Add state manually (rare): ip xfrm state add src local dst remote proto esp spi 123 reqid 1 ```

Step 6: Fix Certificate Issues

```bash # For certificate-based auth:

# Check certificates: ls -la /etc/ipsec.d/certs/ ls -la /etc/ipsec.d/private/

# Required files: # certs/ - Certificate files # private/ - Private keys # cacerts/ - CA certificates

# Verify certificate: openssl x509 -in cert.pem -text -noout

# Check expiration: openssl x509 -in cert.pem -noout -dates

# Check certificate chain: openssl verify -CAfile cacert.pem cert.pem

# Certificate secrets: # In ipsec.secrets: : RSA key.pem

# Or specific: peer-ip : RSA "key.pem"

# Check certificate in config: leftcert=cert.pem rightcert=peer_cert.pem

# Verify certificate matches key: openssl x509 -noout -modulus -in cert.pem | openssl md5 openssl rsa -noout -modulus -in key.pem | openssl md5 # Should match

# Generate new certificate: openssl req -new -key key.pem -out cert.csr openssl x509 -req -in cert.csr -CA cacert.pem -CAkey cakey.pem -out cert.pem

# Load certificates: ipsec rereadcerts

# For IKEv2 with EAP-TLS: eap_identity = cert ```

Step 7: Handle NAT Traversal

```bash # NAT traversal (NAT-T) issues:

# Check NAT-T enabled: # In config: left=%any leftid=@local-id right=peer-ip

# Force NAT-T: force_nat_t = yes

# For peer behind NAT: right=%any rightid=@remote-id

# Check UDP encapsulation: # NAT-T uses UDP port 4500

# Check NAT detected: ipsec statusall | grep NAT

# Fix NAT-T issues: # 1. Enable NAT-T on both sides # 2. Use correct ports (500 + 4500) # 3. Use %any for dynamic peers # 4. Use ID instead of IP for peer ID

# Configuration with NAT: conn natvpn left=local-ip leftsubnet=10.0.0.0/24 leftid=@local right=%any rightsubnet=192.168.2.0/24 rightid=@remote force_nat_t=yes

# Check NAT mapping: # Keepalive packets maintain NAT mapping # Enable keepalive: dpdaction=clear dpddelay=30s

# Dead peer detection: dpdtimeout=120s ```

Step 8: Check IKEv1 vs IKEv2

```bash # IKE version matters:

# IKEv1: keyexchange=ikev1

# IKEv2: keyexchange=ikev2

# Both peers must use same IKE version

# IKEv2 advantages: # - Simpler, fewer messages # - Better NAT traversal # - EAP authentication # - Mobility and multihoming (MOBIKE)

# IKEv1 for legacy: # May need for older devices

# IKEv1 Phase 1/2: # Phase 1: IKE SA (key exchange) # Phase 2: IPsec SA (data tunnel)

# IKEv2: # Single exchange for both

# Check IKE version in logs: journalctl -u strongswan | grep IKEv

# Force IKEv2: keyexchange=ikev2 ikev2=insist

# Allow IKEv1 fallback: ikev2=propose

# Check connection type: ipsec status | grep IKE ```

Step 9: Debug IKE Negotiation

```bash # Enable verbose debug: # In /etc/strongswan/strongswan.conf: charon { debug { default = 2 ike = 3 cfg = 3 net = 2 enc = 1 knl = 2 } }

# Or runtime: ipsec start --debug-all

# Check specific phases: # Phase 1 debug: debug.ike = 3

# Phase 2 debug: debug.cfg = 3

# Monitor negotiation: journalctl -u strongswan -f | grep -E "IKE|IPsec|proposal"

# Look for: # 1. Proposal selection # 2. Key agreement # 3. Policy installation

# Check packet capture: tcpdump -i eth0 udp port 500 or udp port 4500

# Analyze IKE packets: # ISAKMP for IKEv1 # IKEv2 header for IKEv2

# Common negotiation errors: # "no proposal chosen" - Algorithm mismatch # "authentication failed" - Wrong PSK/cert # "policy mismatch" - Subnet mismatch

# Re-initiate connection: ipsec up myvpn ipsec restart

# Terminate and restart: ipsec down myvpn ipsec up myvpn ```

Step 10: IPsec Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-ipsec.sh #!/bin/bash

echo "=== StrongSwan Status ===" systemctl status strongswan 2>/dev/null || systemctl status ipsec 2>/dev/null || echo "Service not running"

echo "" echo "=== IPsec Process ===" ps aux | grep -E "charon|ipsec" | grep -v grep || echo "No IPsec process"

echo "" echo "=== Security Associations ===" ipsec status 2>/dev/null || swanctl --list-sas 2>/dev/null || echo "No SAs established"

echo "" echo "=== IKE Proposals ===" ipsec statusall 2>/dev/null | grep -E "IKE|ESP" | head -10 || echo "Cannot get proposals"

echo "" echo "=== IPsec Secrets ===" cat /etc/ipsec.secrets 2>/dev/null | head -5 || echo "No secrets file"

echo "" echo "=== IPsec Configuration ===" cat /etc/ipsec.conf 2>/dev/null | grep -E "conn|left|right|ike|esp" | head -20 || echo "No config file"

echo "" echo "=== XFRM Policies ===" ip xfrm policy list 2>/dev/null | head -10 || echo "No XFRM policies"

echo "" echo "=== XFRM States ===" ip xfrm state list 2>/dev/null | head -10 || echo "No XFRM states"

echo "" echo "=== Network Connectivity ===" for peer in $(grep "right=" /etc/ipsec.conf 2>/dev/null | grep -v "#" | awk -F= '{print $2}' | head -5); do if [ "$peer" != "%any" ]; then echo "Peer: $peer" ping -c 2 -W 2 $peer 2>&1 | tail -2 nc -zuv $peer 500 2>&1 || true nc -zuv $peer 4500 2>&1 || true fi done

echo "" echo "=== Firewall ===" iptables -L -n 2>/dev/null | grep -E "500|4500|ESP" || echo "No IPsec firewall rules"

echo "" echo "=== Recent Logs ===" journalctl -u strongswan --no-pager -n 10 2>/dev/null || tail /var/log/syslog | grep ipsec | tail -10

echo "" echo "=== Recommendations ===" echo "1. Verify PSK/certificates match on both peers" echo "2. Check IKE proposals (encryption, hash, DH)" echo "3. Ensure traffic selectors (subnets) match" echo "4. Allow UDP 500, 4500, and ESP in firewall" echo "5. Enable NAT-T if either peer behind NAT" echo "6. Use matching IKE version (v1 or v2)" echo "7. Check peer IP reachable" EOF

chmod +x /usr/local/bin/check-ipsec.sh

# Usage: /usr/local/bin/check-ipsec.sh ```

IPsec Tunnel Checklist

Verify the Fix

```bash # After fixing IPsec tunnel

# 1. Check SAs established ipsec status // Security Associations (1 up, 0 connecting)

# 2. Check IKE SA ipsec status | grep IKE // IKE SA established

# 3. Check IPsec SA ip xfrm state // Shows encryption state

# 4. Test traffic ping 192.168.2.1 // Goes through tunnel

# 5. Check policy ip xfrm policy list // Policy installed

# 6. Monitor logs journalctl -u strongswan -f // No negotiation errors ```

Related Issues

• [Fix WireGuard Handshake Failed](/articles/fix-wireguard-handshake-failed)
• [Fix OpenVPN Connection Failed](/articles/fix-openvpn-connection-failed)
• [Fix SSH Connection Refused](/articles/fix-ssh-connection-refused)