Beliefs depend on the available information. This idea is formalized in probability theory by conditioning. Conditional probabilities, conditional expectations, and conditional probability distributions are treated on three levels: discrete probabilities, probability density functions, and measure theory. Conditioning leads to a non-random result if the condition is completely specified; otherwise, if the condition is left random, the result of conditioning is also random.
Conditioning on the discrete level
Example: A fair coin is tossed 10 times; the random variable X is the number of heads in these 10 tosses, and Y is the number of heads in the first 3 tosses. In spite of the fact that Y emerges before X it may happen that someone knows X but not Y.
Conditional probability
Given that X = 1, the conditional probability of the event Y = 0 is
More generally,
One may also treat the conditional probability as a random variable, — a function of the random variable X, namely,
The expectation of this random variable is equal to the (unconditional) probability,
namely,
which is an instance of the law of total probability
Thus, may be treated as the value of the random variable corresponding to X = 1. On the other hand, is well-defined irrespective of other possible values of X.
Conditional expectation
Given that X = 1, the conditional expectation of the random variable Y is More generally,
(In this example it appears to be a linear function, but in general it is nonlinear.) One may also treat the conditional expectation as a random variable, — a function of the random variable X, namely,
The expectation of this random variable is equal to the (unconditional) expectation of Y,
namely,
or simply
which is an instance of the law of total expectation
The random variable is the best predictor of Y given X. That is, it minimizes the mean square error on the class of all random variables of the form f(X). This class of random variables remains intact if X is replaced, say, with 2X. Thus, It does not mean that rather, In particular, More generally, for every function g that is one-to-one on the set of all possible values of X. The values of X are irrelevant; what matters is the partition (denote it αX)
of the sample space Ω into disjoint sets { X = xn}. (Here are all possible values of X.) Given an arbitrary partition α of Ω, one may define the random variable E ( Y | α ). Still, E ( E ( Y | α)) = E ( Y ).
Conditional probability may be treated as a special case of conditional expectation. Namely, P ( A | X ) = E ( Y | X ) if Y is the indicator of A. Therefore the conditional probability also depends on the partition αX generated by X rather than on X itself; P ( A | g(X) ) = P (A | X) = P (A | α), α = αX = αg(X).
On the other hand, conditioning on an event B is well-defined, provided that irrespective of any partition that may contain B as one of several parts.
Conditional distribution
Given X = x, the conditional distribution of Y is
for 0 ≤ y ≤ min ( 3, x ). It is the hypergeometric distribution H ( x; 3, 7 ), or equivalently, H ( 3; x, 10-x ). The corresponding expectation 0.3 x, obtained from the general formula
for H ( n; R, W ), is nothing but the conditional expectation E (Y | X = x) = 0.3 x.
Treating H ( X; 3, 7 ) as a random distribution (a random vector in the four-dimensional space of all measures on { 0,1,2,3}), one may take its expectation, getting the unconditional distribution of Y, — the binomial distribution Bin ( 3, 0.5 ). This fact amounts to the equality
for y = 0,1,2,3; which is an instance of the law of total probability.
Conditioning on the level of densities
Example. A point of the sphere x2 + y2 + z2 = 1 is chosen at random according to the uniform distribution on the sphere. The random variables X, Y, Z are the coordinates of the random point. The joint density of X, Y, Z does not exist (since the sphere is of zero volume), but the joint density fX,Y of X, Y exists,
(The density is non-constant because of a non-constant angle between the sphere and the plane.) The density of X may be calculated by integration,
surprisingly, the result does not depend on x in (−1,1),
which means that X is distributed uniformly on (−1,1). The same holds for Y and Z (and in fact, for aX + bY + cZ whenever a2 + b2 + c2 = 1).
Example. A different measure of calculating the marginal distribution function is provided below
Conditional probability
Calculation
Given that X = 0.5, the conditional probability of the event Y ≤ 0.75 is the integral of the conditional density,