IPsec

Internet security protocol

Synopsis:

#include <sys/types.h>
#include <netinet/in.h>
#include <netinet6/ipsec.h>

int socket( PF_KEY, 
            SOCK_RAW, 
            PF_KEY_V2 );

Description:

IPsec is a security protocol for the Internet Protocol layer. It consists of these sub-protocols:

AH (Authentication Header): Guarantees the integrity of the IP packet and protects it from intermediate alteration or impersonation by attaching a cryptographic checksum computed by one-way hash functions.
ESP (Encapsulated Security Payload): Protects the IP payload from wire-tapping by encrypting it using secret-key cryptography algorithms.

IPsec has these modes of operation:

Transport — protects peer-to-peer communication between end nodes.
Tunnel — supports IP-in-IP encapsulation operation and is designed for security gateways, like VPN configurations.

Kernel interface

The IPsec protocol behavior is controlled by these engines:

Key management engine — accessed from an application using PF_KEY sockets. The RFC 2367 specification defines the PF_KEY socket API.
Policy engine — accessed with the PF_KEY API, the setsockopt() operations, and the sysctl() interface. (The sysctl utility is a cover for the sysctl() function.) The setsockopt() function defines per-socket behavior and the sysctl() interface defines host-wide default behavior.

These engines are located in the socket manager. The socket manager implements the PF_KEY interface and allows you to define IPsec policy similar to per-packet filters. Note that the socket manager code doesn't implement the dynamic encryption key exchange protocol IKE (Internet Key Exchange) — that implementation should be done at the application level (usually as daemons), using the previously described APIs.

Policy management

The socket manager implements experimental policy management. You can manage the IPsec policy in these ways:

Configure a per-socket policy using setsockopt().
Configure the socket manager packet filter-based policy using the PF_KEY interface or via the setkey utility.
In this case, the default policy is allowed with the setkey. By configuring the policy to default, you can use the system-wide sysctl utility variables. (The sysctl utility displays various runtime options.)

If the socket manager finds no matching policy, the system-wide default value is applied.

For a list of net.inet6.ipsec6.* variables, see the sysctl utility in the Utilities Reference.

Miscellaneous `sysctl` variables

The following variables are accessible via the sysctl utility for tweaking socket manager IPsec behavior:

Name	Type	Changeable?
net.inet.ipsec.ah_cleartos	Integer	Yes
net.inet.ipsec.ah_offsetmask	Integer	Yes
net.inet.ipsec.dfbit	Integer	Yes
net.inet.ipsec.ecn	Integer	Yes
net.inet.ipsec.debug	Integer	Yes
net.inet6.ipsec6.ecn	Integer	Yes
net.inet6.ipsec6.debug	Integer	Yes

The variables are interpreted as follows:

ipsec.ah_cleartos

When computing AH authentication data, the socket manager clears the type-of-service field in the IPv4 header if the value is set to a nonzero value. The variable tweaks AH behavior to interoperate with devices that implement RFC 1826 AH. Set this to a nonzero value (clear the type-of-service field) if you want to conform to RFC 2402.

ipsec.ah_offsetmask

When computing AH authentication data, the socket manager includes the 16-bit fragment offset field (including flag bits) in the IPv4 header, after computing a logical “AND” with the variable. This variable tweaks the AH behavior to interoperate with devices that implement RFC 1826 AH. Set this value to zero (clear the fragment offset field during computation) if you want to conform to RFC 2402.

ipsec.dfbit

Configures the socket manager behavior for IPv4 IPsec tunnel encapsulation. The variable is supplied to conform to RFC 2403 Chapter 6.1.

If the value is set to:	Then:
0	The DF bit on the outer IPv4 header is cleared.
1	The outer DF bit on the header is set from the inner DF bit.
2	The DF bit is copied from the inner header to the outer.

ipsec.ecn

If set to nonzero, the IPv4 IPsec tunnel encapsulation/decapsulation behavior supports ECN (Explicit Congestion Notification), as documented in the IETF draft draft-ietf-ipsec-ecn-02.txt.

ipsec.debug

If set to nonzero, debug messages are generated to the syslog.

Variables under the net.inet6.ipsec6 tree have meaning similar to their net.inet.ipsec counterparts.

Protocols

Because the IPsec protocol works like a plugin to the INET and INET6 protocols, IPsec supports most of the protocols defined upon those IP-layer protocols. Some of the protocols, like ICMP or ICMP6, may behave differently with IPsec. This is because IPsec can prevent ICMP or ICMP6 routines from looking into the IP payload.

Setting the policy

You can set the policy manually by calling setkey, or set it permanently in /etc/inetd.conf. Valid policy settings include:

for setkey:

-P direction discard
-P direction ipsec request ...
-P direction none

for /etc/inetd.conf:

direction bypass
direction entrust
direction ipsec request ...

where:

direction

The direction in which the policy is applied. It's either in or out.

bypass

(/etc/inetd.conf only) Bypass the IPsec processing and transmit the packet in clear text. This option is for privileged sockets.

discard

(setkey only) Discard the packet matching indexes.

entrust

(/etc/inetd.conf only) Consult the Security Policy Database (SPD) in the stack. The SPD is set by setkey (see the Utilities Reference).

ipsec request ...

Put the IPsec operation into the packet. You can specify one or more request strings using the following format:

protocol/mode/src-dst[/level]

For detailed descriptions of the arguments in the request string, see below.

none

(setkey only) Don't put the IPsec operation into the packet.

Arguments for request

protocol

One of:

ah — Authentication Header. Guarantees the integrity of the IP packets and protects them from intermediate alteration or impersonation, by attaching cryptographic checksums computed by one-way hash functions.
esp — Encapsulated Security Payload. Protects the IP payload from wire-tapping by encrypting it with secret key cryptography algorithms.
ipcomp — IP Payload Compression Protocol.

mode

Security protocol to be used, which is one of:

transport — Protects peer-to-peer communication between end nodes.
tunnel — Includes IP-in-IP encapsulation operations and is designed for security gateways, like VPN configurations.

dst, src

The “receiving node” (dst) and “sending node” (src) endpoint addresses of the Security Association (SA). When the direction specified is in, dst would represent this node and src the other node (peer).

If transport is specified as the mode, you can omit these values.

level

One of:

default — The stack should consult the system default policy that's set by the sysctl utility.
require — An SA is required whenever the kernel deals with the packet.
use — Use an SA if it's available; otherwise, keep the normal operation.
unique — (setkey only) Similar to require, but adds the restriction that the SA for outbound traffic is used only for this policy.
You may need the identifier in order to relate the policy and the SA when you define the SA by manual keying. You can put the decimal number as the identifier after unique, such as:
unique: number
The value of number must be between 1 and 32767. If the request string is kept unambiguous, the level and slash prior to level can be omitted. However, you should specify them explicitly to avoid unintended behaviors.

If the level isn't specified in the setkey command, unique is used by default.

Caveats:

The IPsec support is subject to change as the IPsec protocols develop.

There's no single standard for policy engine API, so the policy engine API described herein is just for KAME implementation.

The AH tunnel may not work as you might expect. If you configure the require policy against AH tunnel for inbound, tunneled packets will be rejected. This is because AH authenticates the encapsulating (outer) packet, not the encapsulated (inner) packet.

Under certain conditions, a truncated result may be returned from the socket manager from SADB_DUMP and SADB_SPDDUMP operations on a PF_KEY socket. This occurs if there are too many database entries in the socket manager and the socket buffer for the PF_KEY socket is too small. If you manipulate many IPsec key/policy database entries, increase the size of socket buffer.