Unit II · Lecture 15 of 40 · B.Sc. Data Science, 1st Year, Semester II
P(A) = n(A) / n(S) | n(A) = favourable outcomes | n(S) = total outcomes in sample space
P(A) = frequency of A / total observations | Based on actual data/experiments
Axiom 1: P(A) ≥ 0 | Axiom 2: P(S) = 1 | Axiom 3: P(A∪B) = P(A) + P(B) for mutually exclusive A, B
P(∅) = 0 | P(A') = 1 − P(A) | 0 ≤ P(A) ≤ 1 | P(A) + P(A') = 1
P(A∪B) = P(A) + P(B) − P(A∩B) | For any two events A and B
P(A∪B) = P(A) + P(B) | Since P(A∩B) = 0 when A and B cannot occur together
P(A|B) = P(A∩B) / P(B) | P(B|A) = P(A∩B) / P(A) | where P(B) > 0, P(A) > 0
P(A∩B) = P(A) × P(B|A) = P(B) × P(A|B) | General rule for any two events
P(A∩B) = P(A) × P(B) | P(A|B) = P(A) | P(B|A) = P(B) | Occurrence of one does not affect the other
P(B) = P(A₁)P(B|A₁) + P(A₂)P(B|A₂) + ... + P(Aₙ)P(B|Aₙ) | where A₁, A₂, ..., Aₙ partition S
P(Aᵢ|B) = P(Aᵢ) × P(B|Aᵢ) / Σⱼ P(Aⱼ) × P(B|Aⱼ) | "Inverse probability" — updating beliefs with new evidence
P(A∪B∪C) = P(A)+P(B)+P(C) − P(A∩B)−P(A∩C)−P(B∩C) + P(A∩B∩C)
Total balls: n(S) = 5 + 3 + 2 = 10
(a) P(Red):
n(Red) = 5
P(Red) = 5/10 = 0.5
(b) P(not Green):
P(Green) = 2/10 = 0.2
P(Green') = 1 − P(Green) = 1 − 0.2 = 0.8
(c) P(Red or Blue):
Red and Blue are mutually exclusive (a ball cannot be both).
P(Red∪Blue) = P(Red) + P(BLue) = 5/10 + 3/10 = 0.8
Verification: P(Red∪Blue) = 0.8 = P(Green') ✓ (Red or Blue = not Green)
Let C = plays cricket, F = plays football
P(C) = 30/60 = 0.5, P(F) = 25/60 = 5/12
P(C∩F) = 10/60 = 1/6
(a) P(C or F):
P(C∪F) = P(C) + P(F) − P(C∩F)
= 30/60 + 25/60 − 10/60
= 45/60 = 0.75
(b) P(neither):
P(neither) = 1 − P(C∪F) = 1 − 0.75 = 0.25
Verification: Students playing at least one = 30 + 25 − 10 = 45. Students playing neither = 60 − 45 = 15. P = 15/60 = 0.25 ✓
Total = 10 items. Defective = 4, Non-defective = 6.
(a) P(both defective):
P(D₁) = 4/10
P(D₂|D₁) = 3/9 (one defective removed, 3 left out of 9)
P(D₁∩D₂) = P(D₁) × P(D₂|D₁) = (4/10) × (3/9) = 12/90 = 2/15 ≈ 0.133
(b) P(second defective | first defective):
After removing one defective item: 3 defective out of 9 total
P(D₂|D₁) = 3/9 = 1/3 ≈ 0.333
Key insight: Without replacement changes the sample space. The second draw depends on the first — these events are NOT independent.
Independence test:
If independent: P(A∩B) = P(A) × P(B) = 0.4 × 0.5 = 0.2
Given P(A∩B) = 0.2
Since 0.2 = 0.2, A and B ARE independent ✓
P(A|B):
P(A|B) = P(A∩B) / P(B) = 0.2 / 0.5 = 0.4
Note: P(A|B) = 0.4 = P(A) ✓ (confirms independence — knowing B doesn't change A's probability)
P(B|A):
P(B|A) = P(A∩B) / P(A) = 0.2 / 0.4 = 0.5
Note: P(B|A) = 0.5 = P(B) ✓ (confirms independence — knowing A doesn't change B's probability)
(a) Mutually exclusive:
P(A∩B) = 0 (cannot occur together)
P(A∪B) = P(A) + P(B) = 0.3 + 0.4 = 0.7
(b) Independent:
P(A∩B) = P(A) × P(B) = 0.3 × 0.4 = 0.12
P(A∪B) = P(A) + P(B) − P(A∩B) = 0.3 + 0.4 − 0.12 = 0.58
(c) Both mutually exclusive AND independent?
If mutually exclusive: P(A∩B) = 0
If independent: P(A∩B) = P(A)×P(B) = 0.12
0 ≠ 0.12 → Contradiction!
Conclusion: A and B CANNOT be both mutually exclusive and independent (unless one has probability 0). These are fundamentally different concepts.
P(A∪B∪C) = P(A) + P(B) + P(C) − P(A∩B) − P(A∩C) − P(B∩C) + P(A∩B∩C)
= 0.4 + 0.5 + 0.3 − 0.2 − 0.1 − 0.15 + 0.05
= 1.2 − 0.45 + 0.05
= 0.8
Verification: P(A∪B∪C) = 0.8 ≤ 1 ✓. Also P(A∪B∪C) ≥ max(0.4, 0.5, 0.3) = 0.5 ✓
Interpretation: The probability that at least one of A, B, or C occurs is 0.8 (80%).
Given:
P(A₁) = 0.40, P(A₂) = 0.35, P(A₃) = 0.25
P(D|A₁) = 0.02, P(D|A₂) = 0.03, P(D|A₃) = 0.05
Total Probability:
P(D) = P(A₁)P(D|A₁) + P(A₂)P(D|A₂) + P(A₃)P(D|A₃)
= (0.40)(0.02) + (0.35)(0.03) + (0.25)(0.05)
= 0.008 + 0.0105 + 0.0125
= 0.031 (3.1%)
Verification: P(A₁)+P(A₂)+P(A₃) = 0.40+0.35+0.25 = 1.00 ✓ (valid partition). P(D) = 0.031 is between min(0.02,0.03,0.05) and max(0.02,0.03,0.05) ✓
From Problem 7: P(D) = 0.031
Bayes' Theorem:
P(A₃|D) = P(A₃) × P(D|A₃) / P(D)
= (0.25 × 0.05) / 0.031
= 0.0125 / 0.031
= 0.4032 (40.32%)
Full Bayes' table for all machines:
| Machine | P(Aᵢ) | P(D|Aᵢ) | P(Aᵢ)×P(D|Aᵢ) | P(Aᵢ|D) |
|---|---|---|---|---|
| A₁ | 0.40 | 0.02 | 0.0080 | 0.008/0.031 = 0.2581 |
| A₂ | 0.35 | 0.03 | 0.0105 | 0.0105/0.031 = 0.3387 |
| A₃ | 0.25 | 0.05 | 0.0125 | 0.0125/0.031 = 0.4032 |
| Total | 1.00 | 0.0310 | 1.0000 |
Interpretation: Even though Machine A₃ produces only 25% of items, it accounts for 40.32% of defective items — the highest proportion. This is because A₃ has the highest defect rate (5%). Bayes' Theorem lets us "reverse" the reasoning: from effect (defective) back to cause (which machine).
1. Confusing mutually exclusive with independent — they are DIFFERENT. Mutually exclusive: P(A∩B)=0. Independent: P(A∩B)=P(A)P(B). Both can only hold if P(A)=0 or P(B)=0.
2. Thinking P(A|B) = P(B|A) — they are generally NOT equal!
3. Applying the addition theorem P(A)+P(B) when events are NOT mutually exclusive — you must subtract P(A∩B).
4. Confusing "at least one" with "exactly one" — "at least one" = 1 − P(none).
1. In conditional probability, dividing by the wrong event: P(A|B) = P(A∩B)/P(B), NOT P(A∩B)/P(A).
2. In Bayes' Theorem, forgetting to include ALL terms in the denominator — it's a sum over ALL partitions.
3. Using P(A∩B) = P(A)×P(B) when events are NOT independent — this only works for independent events.
4. In "without replacement" problems, not updating the total after each draw.
1. Probabilities exceeding 1 or being negative — always check 0 ≤ P ≤ 1.
2. Partition probabilities not summing to 1 — verify P(A₁)+P(A₂)+...+P(Aₙ) = 1 before applying Bayes'.
3. Not defining the sample space clearly — count outcomes carefully for classical probability.
4. In counting problems, confusing permutations with combinations — order matters in permutations.
Always start by clearly defining events A, B, etc. and the sample space S. This prevents confusion later.
Before applying formulas, determine: Are events mutually exclusive? Independent? This determines which formula to use.
For Bayes' and Total Probability problems, a tree diagram makes the solution almost automatic.
"At least one" problems are easiest via P = 1 − P(none). Don't try to count all cases individually.
1. Compute P(A∪B) using the addition theorem. (Very common)
2. Find conditional probability P(A|B) from given data. (Very common)
3. Test whether two events are independent. (Common)
4. Distinguish between mutually exclusive and independent events. (Almost every exam)
5. Apply the multiplication theorem for "and" problems. (Common)
6. Solve a Bayes' Theorem problem (factory/medical test/defective items). (Almost every exam)
7. Use Total Probability to find P(B) from a partition. (Common)
8. "At least one" problems using the complement rule. (Occasional)
Q1: The probability of an impossible event is:
Q2: P(A∪B) = P(A) + P(B) is valid when:
Q3: If A and B are independent with P(A)=0.3 and P(B)=0.5, then P(A∩B) =
Q4: P(A|B) is defined as:
Q5: If A and B are mutually exclusive with P(A)=0.3 and P(B)=0.4, then P(A|B) =
Q6: Two events A and B are independent if:
Q7: P(A') equals:
Q8: Can two events be both mutually exclusive and independent (with non-zero probabilities)?
Q9: Bayes' Theorem is used to:
Q10: The sum of probabilities of all outcomes in a sample space is:
Q11: In "without replacement" sampling, successive draws are:
Q12: P(A∪B∪C) for three events includes the term P(A∩B∩C) with a:
Classical & Empirical Probability
Kolmogorov Axioms
Basic Results & Complements
Addition Theorem
General & Mutually Exclusive
Three Events
Conditional Probability
P(A|B) = P(A∩B)/P(B)
Intuition & Applications
Multiplication Theorem
Independent Events
With & Without Replacement
Total Probability
Bayes' Theorem
Partition & Inverse Problems
You have completed Unit II — Probability.
Probability is the language of uncertainty. The axioms, addition and multiplication rules, conditional probability, and Bayes' Theorem you've learned here are the foundation for everything in Unit III (distributions), and indeed for all of modern data science and machine learning. Keep practising — every problem strengthens your intuition.
Next up: Unit III — Random Variables & Distributions!
BDS113 Introduction to Statistics · B.Sc. Data Science, Semester II · Unit II Review Complete