Statistics, Conditional Probability and Random Variables

Definitions

• The sample space S is the set of all possible outcomes; an event A is a subset of S.
• P(A) = number of favourable outcomes / number of equally likely outcomes.
• 0 ≤ P(A) ≤ 1; P(S) = 1 and P(∅) = 0.
• Complement rule: P(A′) = 1 − P(A).

Addition rule

• P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
• If A and B are mutually exclusive, P(A ∩ B) = 0 and P(A ∪ B) = P(A) + P(B).
• Example: one die — P(even or greater than 4) = 3/6 + 2/6 − 1/6 = 4/6 = 2/3.

Multiplication rule

• For independent events, P(A ∩ B) = P(A)·P(B).
• Example: two fair coins — P(two heads) = ½ × ½ = ¼.
• Without replacement the second probability changes: two cards drawn from a pack, P(both aces) = 4/52 × 3/51 = 1/221.

The formula

• P(A | B) = P(A ∩ B) / P(B), provided P(B) > 0 — the probability of A given that B has happened.
• Equivalent product form: P(A ∩ B) = P(B)·P(A | B).

Worked examples

1. One die: P(odd | greater than 3). Greater than 3 = {4, 5, 6}, odd among them = {5} → 1/3.
2. An urn has 5 red and 3 blue balls. Two are drawn without replacement. P(second is red | first is red) = 4/7; P(both red) = 5/8 × 4/7 = 5/14.
3. In a class, 60% study maths, 40% study physics, 25% study both. P(physics | maths) = 0.25/0.60 = 5/12 ≈ 0.417.

Independence

• A and B are independent when P(A ∩ B) = P(A)·P(B), equivalently P(A | B) = P(A).
• In the example above, P(P) = 0.4 but P(P | M) ≈ 0.417, so the two subjects are not independent.
• Mutually exclusive and independent are different ideas: mutually exclusive events with non-zero probabilities are never independent.

Tree diagrams

• Multiply along a branch, add across branches.
• For two draws without replacement, the second-stage denominators are one less than the first.

Probability distribution

• A discrete random variable X takes countable values x₁, x₂, …, with probabilities p₁, p₂, … summing to 1.
• Example: X = number of heads in two tosses → P(0) = ¼, P(1) = ½, P(2) = ¼, and ¼ + ½ + ¼ = 1.

Expectation and variance

• E(X) = Σ x·P(x) — the mean of the distribution.
• Var(X) = Σ x²·P(x) − [E(X)]², and the standard deviation is √Var(X).
• For the two-toss example: E(X) = 0(¼) + 1(½) + 2(¼) = 1; E(X²) = 0 + ½ + 1 = 1.5; Var(X) = 1.5 − 1 = 0.5; σ ≈ 0.707.
• Properties: E(aX + b) = aE(X) + b and Var(aX + b) = a²Var(X).

1. A die is rolled; find P(prime) → {2, 3, 5} → 1/2
2. Two dice are rolled; find P(sum = 9) → 4/36 = 1/9
3. P(A) = 0.5, P(B) = 0.4, P(A ∩ B) = 0.2; find P(A ∪ B) and test independence → 0.7; and 0.5×0.4 = 0.2, so independent
4. From a pack, P(king | red card) → 2/26 = 1/13
5. An urn has 4 white and 6 black balls; two drawn without replacement — P(both white) → 4/10 × 3/9 = 2/15
6. X has P(1) = 0.2, P(2) = 0.5, P(3) = 0.3; find E(X) → 0.2 + 1.0 + 0.9 = 2.1
7. For the same X, find Var(X) → E(X²) = 0.2 + 2.0 + 2.7 = 4.9; Var = 4.9 − 4.41 = 0.49
8. P(A) = 0.6, P(B | A) = 0.5; find P(A ∩ B) → 0.3