Zipf's law (, ) is an empirical law that often holds, approximately, when a list of measured values is sorted in decreasing order. It states that the value of the nth entry is inversely proportional to n.
The best known instance of Zipf's law applies to the frequency table of words in a text or corpus of natural language: It is usually found that the most common word occurs approximately twice as often as the next common one, three times as often as the third most common, and so on. For example, in the Brown Corpus of American English text, the word "the" is the most frequently occurring word, and by itself accounts for nearly 7% of all word occurrences (69,971 out of slightly over 1 million). True to Zipf's Law, the second-place word "of" accounts for slightly over 3.5% of words (36,411 occurrences), followed by "and" (28,852). It is often used in the following form, called Zipf-Mandelbrot law:where are fitted parameters, with , and .
This "law" is named after the American linguist George Kingsley Zipf, and is still an important concept in quantitative linguistics. It has been found to apply to many other types of data studied in the physical and social sciences.
In mathematical statistics, the concept has been formalized as the Zipfian distribution: a family of related discrete probability distributions whose rank-frequency distribution is an inverse power law relation. They are related to Benford's law and the Pareto distribution.
Some sets of time-dependent empirical data deviate somewhat from Zipf's law. Such empirical distributions are said to be quasi-Zipfian.
History
In 1913, the German physicist Felix Auerbach observed an inverse proportionality between the population sizes of cities, and their ranks when sorted by decreasing order of that variable.
Zipf's law has been discovered before Zipf, by the French stenographer Jean-Baptiste Estoup' Gammes Stenographiques (4th ed) in 1916, with G. Dewey in 1923, and with E. Condon in 1928.
The same relation for frequencies of words in natural language texts was observed by George Zipf in 1932, but he never claimed to have originated it. In fact, Zipf did not like mathematics. In his 1932 publication, the author speaks with disdain about mathematical involvement in linguistics, a. o. ibidem, p. 21: (…) let me say here for the sake of any mathematician who may plan to formulate the ensuing data more exactly, the ability of the highly intense positive to become the highly intense negative, in my opinion, introduces the devil into the formula in the form of √(-i). The only mathematical expression Zipf used looks like a.b2 = constant, which he "borrowed" from Alfred J. Lotka's 1926 publication.
The same relationship was found to occur in many other contexts, and for other variables besides frequency. For example, when corporations are ranked by decreasing size, their sizes are found to be inversely proportional to the rank. The same relation is found for personal incomes (where it is called Pareto principle), number of people watching the same TV channel, notes in music, cells transcriptomes and more.
Formal definition
Formally, the Zipf distribution on elements assigns to the element of rank (counting from 1) the probability
where is a normalization constant, the th harmonic number:
The distribution is sometimes generalized to an inverse power law with exponent instead of 1. Namely,
where , is a generalized harmonic number
The generalized Zipf distribution can be extended to infinitely many items ( = ∞) only if the exponent exceeds 1. In that case, the normalization constant , becomes Riemann's zeta function,
If the exponent is 1 or less, the normalization constant , diverges as tends to infinity.
Empirical testing
Empirically, a data set can be tested to see whether Zipf's law applies by checking the goodness of fit of an empirical distribution to the hypothesized power law distribution with a Kolmogorov–Smirnov test, and then comparing the (log) likelihood ratio of the power law distribution to alternative distributions like an exponential distribution or lognormal distribution.
Zipf's law can be visuallized by plotting the item frequency data on a log-log graph, with the axes being the logarithm of rank order, and logarithm of frequency. The data conform to Zipf's law with exponent to the extent that the plot approximates a linear (more precisely, affine) function with slope −. For exponent = 1, one can also plot the reciprocal of the frequency (mean interword interval) against rank, or the reciprocal of rank against frequency, and compare the result with the line through the origin with slope 1.