SYN cookie is a technique used to resist SYN flood attacks. The technique's primary inventor Daniel J. Bernstein defines SYN cookies as "particular choices of initial TCP sequence numbers by TCP servers." In particular, the use of SYN cookies allows a server to avoid dropping connections when the SYN queue fills up. Instead of storing additional connections, a SYN queue entry is encoded into the sequence number sent in the SYN+ACK response. If the server then receives a subsequent ACK response from the client with the incremented sequence number, the server is able to reconstruct the SYN queue entry using information encoded in the TCP sequence number and proceed as usual with the connection.
Implementation
In order to initiate a TCP connection, the client sends a TCP SYN packet to the server. In response, the server sends a TCP SYN+ACK packet back to the client. One of the values in this packet is a sequence number, which is used by the TCP to reassemble the data stream. According to the TCP specification, that first sequence number sent by an endpoint can be any value as decided by that endpoint. As the sequence number is chosen by the sender, returned by the recipient, and has no otherwise-defined internal structure, it can be overloaded to carry additional data. The following describes one possible implementation, however as there is no public standard to follow, the order, length, and semantics of the fields may differ between SYN cookie implementations.
SYN cookies are initial sequence numbers that are carefully constructed according to the following rules:
let t be a slowly incrementing timestamp (typically logically right-shifted 6 positions, which gives a resolution of 64 seconds) let m be the maximum segment size (MSS) value that the server would have stored in the SYN queue entry let s be the result of a cryptographic hash function computed over the server IP address and port number, the client IP address and port number, and the value t. The returned value s must be a 24-bit value.
The initial TCP sequence number, i.e. the SYN cookie, is computed as follows: Top 5 bits: t mod 32 Middle 3 bits: an encoded value representing m Bottom 24 bits: s (Note: since m must be encoded using 3 bits, the server is restricted to sending up to 8 unique values for m when SYN cookies are in use.)
When a client sends back a TCP ACK packet to the server in response to the server's SYN+ACK packet, the client must (according to the TCP spec) use n+1 in the packet's Acknowledgement number, where n is the initial sequence number sent by the server. The server then subtracts 1 from the acknowledgement number to reveal the SYN cookie sent to the client.
The server then performs the following operations. Checks the value t against the current time to see if the connection has expired. Recomputes s to determine whether this is, indeed, a valid SYN cookie. Decodes the value m from the 3-bit encoding in the SYN cookie, which it then can use to reconstruct the SYN queue entry. From this point forward, the connection proceeds as normal.
Drawbacks
The use of SYN cookies does not break any protocol specifications, and therefore should be compatible with all TCP implementations. There are, however, two caveats that take effect when SYN cookies are in use. Firstly, the server is limited to only 8 unique MSS values, as that is all that can be encoded in 3 bits. Secondly, early implementations rejected all TCP options (such as large windows or timestamps), because the server discarded the SYN queue entry where that information would otherwise be stored.; however v2.6.26 of the Linux kernel added partial support of TCP options by encoding them into the timestamp option. Finally, SYN cookies place increased load on server resources. Encrypting responses is computationally expensive. The SYN cookie does not reduce traffic, which makes it ineffective against SYN flooding attacks that target bandwidth as the attack vector.
While these restrictions necessarily lead to a sub-optimal experience, their effect is rarely noticed by clients because they are only applied when under attack. In such a situation, the loss of the TCP options in order to save the connection is usually considered to be a reasonable compromise.