notas.itmens
Law of Large Numbers
A sequence of random variables \(X_1,X_2,\ldots\) is said to
- satisfy the weak law of large numbers iff for some constant \(c\), \(S_n/n\) converges to \(c\) in probability,
- satisfy the strong law of large numbers iff for some constant \(c\), \(S_n/n\) converges to \(c\) almost surely,
Where \(S_n = \sum X_i\).
Borel-Cantelli Lemma#
Lemma (Borel-Cantelli). If \(\sum_{i=1}^\infty P(A_i) <\infty\), then \(P(A_i\text{ i.o})=0\).
Here i.o. stands for “infinitely often” and means the set
\[\lim \sup A_i = \lim_{m\to\infty}\cup_{i=m}^{\infty} A_i = \{\omega \text{ that are in infinitely many }A_i\}.\]Dual to this we have that
\[\lim \inf A_i = \lim_{m\to\infty}\cap_{i=m}^{\infty} A_i = \{\omega \text{ that are in all but finitely many }A_i\}.\]A consequence of Borel-Cantelli lemma can shed some light on the nature of almost sure convergence, and allows one to upgrade convergence in probability to almost sure convergence:
Theorem. \(X_n \to X\) in probability iff for every subsequence \(X_{n(m)}\) there is a further subsequence \(X_{n(m_k)}\) that converges almost surely to \(X\).
Observation. This is similar to (but very different in nature from) the result: let \(y_n\) be a sequence in a topological space, if every subsequence \(y_{n(m)}\) has a further subsequence \(y_{n(m_k)}\) that converges to \(y\) then \(y_n\to y\). From this we also see that almost sure convergence does not come from a metric or even from a topology since there is a sequence of random variables that converges in probability but not a. s.
An example of the implication of this is:
Theorem. If \(f\) is continuous and \(X_i\to X\) in probability then \(f(X_i)\to f(X)\) in probability. If in addition \(f\) is bounded then \(Ef(X_i) \to Ef(X)\).
Weak Law of Large Numbers#
Theorem. Let \(X_1,X_2,\ldots\) be i.i.d. (independent and identically distributed) with finite means, and let \(c = EX_1\), then \(X_i\) satisfies the weak law of large numbers.
Strong Law of Large Numbers#
Theorem. Let \(X_1,X_2,\ldots\) be pairwise independent and identically distributed with finite means, and let \(c = EX_1\), then \(X_i\) satisfies the strong law of large numbers.