HTTP compression is a capability that can be built into web servers and web clients to improve transfer speed and bandwidth utilization.
HTTP data is compressed before it is sent from the server: compliant browsers will announce what methods are supported to the server before downloading the correct format; browsers that do not support compliant compression method will download uncompressed data. The most common compression schemes include gzip and Brotli; a full list of available schemes is maintained by the IANA.
There are two different ways compression can be done in HTTP. At a lower level, a Transfer-Encoding header field may indicate the payload of an HTTP message is compressed. At a higher level, a Content-Encoding header field may indicate that a resource being transferred, cached, or otherwise referenced is compressed. Compression using Content-Encoding is more widely supported than Transfer-Encoding, and some browsers do not advertise support for Transfer-Encoding compression to avoid triggering bugs in servers.
Compression scheme negotiation
The negotiation is done in two steps, described in RFC 2616 and RFC 9110:
1. The web client advertises which compression schemes it supports by including a list of tokens in the HTTP request. For Content-Encoding, the list is in a field called Accept-Encoding; for Transfer-Encoding, the field is called TE. GET /encrypted-area HTTP/1.1 Host: www.example.com Accept-Encoding: gzip, deflate 2. If the server supports one or more compression schemes, the outgoing data may be compressed by one or more methods supported by both parties. If this is the case, the server will add a Content-Encoding or Transfer-Encoding field in the HTTP response with the used schemes, separated by commas.
HTTP/1.1 200 OK
Date: mon, 26 June 2016 22:38:34 GMT
Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux) Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT Accept-Ranges: bytes Content-Length: 438 Connection: close Content-Type: text/html; charset=UTF-8 Content-Encoding: gzip
The web server is by no means obligated to use any compression method – this depends on the internal settings of the web server and also may depend on the internal architecture of the website in question.
Content-Encoding tokens
The official list of tokens available to servers and client is maintained by IANA, and it includes:
br – Brotli, a compression algorithm specifically designed for HTTP content encoding, defined in and implemented in all modern major browsers. compress – UNIX "compress" program method (historic; deprecated in most applications and replaced by gzip or deflate) deflate – compression based on the deflate algorithm (described in ), a combination of the LZ77 algorithm and Huffman coding, wrapped inside the zlib data format (); exi – W3C Efficient XML Interchange gzip – GNU zip format (described in ). Uses the deflate algorithm for compression, but the data format and the checksum algorithm differ from the "deflate" content-encoding. This method is the most broadly supported as of March 2011. identity – No transformation is used. This is the default value for content coding. pack200-gzip – Network Transfer Format for Java Archives zstd – Zstandard compression, defined in
In addition to these, a number of unofficial or non-standardized tokens are used in the wild by either servers or clients:
bzip2 – compression based on the free bzip2 format, supported by lighttpd
lzma – compression based on (raw) LZMA is available in Opera 20, and in elinks via a compile-time option peerdist – Microsoft Peer Content Caching and Retrieval rsync – delta encoding in HTTP, implemented by a pair of rproxy proxies. xpress – Microsoft compression protocol used by Windows 8 and later for Windows Store application updates. LZ77-based compression optionally using a Huffman encoding. xz – LZMA2-based content compression, supported by a non-official Firefox patch; and fully implemented in mget since 2013-12-31.