Unit IV · Lecture 33 of 40 · B.Sc. Data Science, 1st Year, Semester II
b = [nΣxy − ΣxΣy] / [nΣx² − (Σx)²] | a = ȳ − bx̄
Σy = na + bΣx + cΣx² | Σxy = aΣx + bΣx² + cΣx³ | Σx²y = aΣx² + bΣx³ + cΣx⁴
Log transform: log y = log a + x·log b → Y = A + Bx | a = 10ᴬ, b = 10ᴮ
Log transform: ln y = ln a + bx → Y = A + bx | a = eᴬ
Log transform: log y = log a + b·log x → Y = A + bX | a = 10ᴬ
Minimise S = Σ(yᵢ − ŷᵢ)² — the Principle of Least Squares
r = [nΣxy − ΣxΣy] / √{[nΣx² − (Σx)²] · [nΣy² − (Σy)²]}
r = √(bᵧₓ · bₓᵧ) | r² = bᵧₓ · bₓᵧ
r² = (Explained variation) / (Total variation) = 1 − (Unexplained / Total)
rₛ = 1 − [6Σdᵢ²] / [n(n²−1)] where dᵢ = R(xᵢ) − R(yᵢ)
r₁₂·₃ = (r₁₂ − r₁₃·r₂₃) / √[(1−r₁₃²)(1−r₂₃²)]
R²₁·₂₃ = (r₁₂² + r₁₃² − 2r₁₂r₁₃r₂₃) / (1 − r₂₃²)
bᵧₓ = [nΣxy − ΣxΣy] / [nΣx² − (Σx)²] = r·Sᵧ/Sₓ | a = ȳ − bᵧₓx̄
bₓᵧ = [nΣxy − ΣxΣy] / [nΣy² − (Σy)²] = r·Sₓ/Sᵧ | a' = x̄ − bₓᵧȳ
bᵧₓ · bₓᵧ = r² | Both lines pass through (x̄, ȳ)
Sᵧₓ = Sᵧ · √(1 − r²) | Always Sᵧₓ ≤ Sᵧ
tan θ = |bᵧₓ − bₓᵧ| / |1 + bᵧₓ·bₓᵧ| = |bᵧₓ − bₓᵧ| / (1 + r²)
Correlation ≠ Causation | r = 0 ≠ Independence | Always plot first! | bᵧₓ ≠ 1/bₓᵧ
L24: Principle of Least Squares
→ ŷ = a + bx (straight line)
→ Normal equations: Σy=na+bΣx
L25: Fitting Polynomials
→ ŷ = a + bx + cx² (parabola)
→ 3 normal equations
L26: Exponential Curves
→ y = abˣ, y = aeᵇˣ, y = axᵇ
→ Linearise by logarithms
L27: Bivariate Data & Scatter Plots
→ Direction, form, strength, outliers
L28: Pearson's r
→ Measures linear association
→ −1 ≤ r ≤ +1, unit-free
L29: Partial & Multiple Correlation
→ r₁₂·₃ controls for X₃
→ R₁·₂₃ = combined predictive power
L30: Spearman's rₛ
→ Rank-based, monotonic
→ rₛ = 1 − 6Σd²/[n(n²−1)]
L31: Simple Linear Regression
→ y on x: ŷ = a + bᵧₓx
→ x on y: x̂ = a' + bₓᵧy
→ Both pass through (x̄, ȳ)
L32: Coefficient of Regression
→ bᵧₓ · bₓᵧ = r²
→ Sᵧₓ = Sᵧ√(1−r²)
→ R² = r² (simple regression)
L33: This Review Session!
Central thread: All three parts use the Principle of Least Squares. Curve fitting finds the best equation. Correlation measures how strong the relationship is. Regression uses the equation for prediction.
Step 1: Compute the table
| x | y | xy | x² | y² |
|---|---|---|---|---|
| 2 | 10 | 20 | 4 | 100 |
| 4 | 12 | 48 | 16 | 144 |
| 5 | 15 | 75 | 25 | 225 |
| 6 | 16 | 96 | 36 | 256 |
| 8 | 20 | 160 | 64 | 400 |
| 25 | 73 | 399 | 145 | 1125 |
(a) Pearson's r:
n=5, Σx=25, Σy=73, Σxy=399, Σx²=145, Σy²=1125
Numerator: 5(399) − 25(73) = 1995 − 1825 = 170
Denominator: √{[5(145)−625] · [5(1125)−5329]} = √{100 · 296} = √29600 = 172.05
r = 170/172.05 = 0.9881
(b) Regression of y on x:
bᵧₓ = 170/100 = 1.7, a = 73/5 − 1.7(25/5) = 14.6 − 8.5 = 6.1
ŷ = 6.1 + 1.7x
Regression of x on y:
bₓᵧ = 170/296 = 0.5743, a' = 5 − 0.5743(14.6) = 5 − 8.385 = −3.385
x̂ = −3.385 + 0.5743y
(c) R² = r² = 0.9881² = 0.9764 → 97.6% of variance explained
(d) Sᵧₓ: Sᵧ = √[1125/5 − 14.6²] = √[225−213.16] = √11.84 = 3.441. Sᵧₓ = 3.441√(1−0.9764) = 3.441(0.1536) = 0.529
(e) Prediction at x=7: ŷ = 6.1 + 1.7(7) = 6.1 + 11.9 = 18.0
Step 1: Transform — Y = log₁₀(y)
| x | y | Y = log y | xY | x² |
|---|---|---|---|---|
| 1 | 3 | 0.4771 | 0.4771 | 1 |
| 2 | 9 | 0.9542 | 1.9084 | 4 |
| 3 | 27 | 1.4314 | 4.2942 | 9 |
| 4 | 81 | 1.9085 | 7.6340 | 16 |
| 5 | 243 | 2.3856 | 11.9280 | 25 |
| 15 | 7.1568 | 26.2417 | 55 |
B = [5(26.2417) − 15(7.1568)] / [5(55) − 225] = [131.2085 − 107.352] / 50 = 23.8565/50 = 0.47713
A = 7.1568/5 − 0.47713(3) = 1.43136 − 1.43139 ≈ 0
a = 10⁰ = 1, b = 10^0.47713 = 3
ŷ = 1 × 3ˣ = 3ˣ (perfect fit — the data is exactly 3ˣ!)
At x = 6: ŷ = 3⁶ = 729
| Student | R₁ | R₂ | d = R₁−R₂ | d² |
|---|---|---|---|---|
| A | 1 | 2 | −1 | 1 |
| B | 3 | 1 | +2 | 4 |
| C | 2 | 3 | −1 | 1 |
| D | 5 | 5 | 0 | 0 |
| E | 4 | 4 | 0 | 0 |
| F | 7 | 6 | +1 | 1 |
| G | 6 | 7 | −1 | 1 |
| Σd² = | 8 | |||
rₛ = 1 − 6(8) / [7(49−1)] = 1 − 48/336 = 1 − 0.1429 = 0.8571
Interpretation: Strong positive agreement between the two judges' rankings.
Partial correlation:
r₁₂·₃ = (0.85 − 0.70×0.60) / √[(1−0.49)(1−0.36)]
= (0.85 − 0.42) / √[0.51 × 0.64]
= 0.43 / √0.3264 = 0.43 / 0.5713 = 0.7526
Multiple correlation:
R²₁·₂₃ = (0.85² + 0.70² − 2(0.85)(0.70)(0.60)) / (1 − 0.60²)
= (0.7225 + 0.49 − 0.714) / (1 − 0.36)
= 0.4985 / 0.64 = 0.7789
R₁·₂₃ = √0.7789 = 0.8826
Interpretation: After controlling for X₃, the correlation drops from 0.85 to 0.75 — some confounding, but the relationship remains strong. Together, X₂ and X₃ explain 77.9% of the variance in X₁.
Regression coefficients:
bᵧₓ = r·Sᵧ/Sₓ = 0.75 × 8/5 = 1.2
bₓᵧ = r·Sₓ/Sᵧ = 0.75 × 5/8 = 0.46875
Regression equations:
y on x: ŷ = ȳ − bᵧₓx̄ + bᵧₓx = 50 − 1.2(30) + 1.2x = 14 + 1.2x
x on y: x̂ = x̄ − bₓᵧȳ + bₓᵧy = 30 − 0.46875(50) + 0.46875y = 6.5625 + 0.46875y
Verification: bᵧₓ × bₓᵧ = 1.2 × 0.46875 = 0.5625 = r² = 0.75² = 0.5625 ✓
R² = r² = 0.5625 = 56.25%
Sᵧₓ = Sᵧ√(1−r²) = 8√(1−0.5625) = 8√0.4375 = 8(0.6614) = 5.29
tan θ = |1.2 − 0.46875| / (1 + 0.5625) = 0.73125 / 1.5625 = 0.468
θ = arctan(0.468) = 25.1°
These are perfect squares: y = x². Let's verify the parabola fit captures this.
| x | y | xy | x²y | x² | x³ | x⁴ |
|---|---|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 2 | 4 | 8 | 16 | 4 | 8 | 16 |
| 3 | 9 | 27 | 81 | 9 | 27 | 81 |
| 4 | 16 | 64 | 256 | 16 | 64 | 256 |
| 5 | 25 | 125 | 625 | 25 | 125 | 625 |
| 15 | 55 | 225 | 979 | 55 | 225 | 979 |
Normal equations:
55 = 5a + 15b + 55c …(1)
225 = 15a + 55b + 225c …(2)
979 = 55a + 225b + 979c …(3)
Solving: a = 0, b = 0, c = 1
ŷ = x² — Perfect fit with SSE = 0!
This makes sense: the data was generated by y = x², and the parabola fit recovers this exactly.
1. Using Pearson's r for non-linear data (use scatter plot first!)
2. Concluding causation from correlation
3. Confusing r = 0 with "no relationship" (only no linear relationship)
4. Forgetting that r has no units while regression coefficients do
5. Computing r² and calling it r (or vice versa)
1. Using the wrong regression line (y on x vs x on y)
2. Thinking bₓᵧ = 1/bᵧₓ (they are NOT reciprocals!)
3. Forgetting that both lines pass through (x̄, ȳ)
4. Extrapolating far beyond the data range
5. Confusing explained and unexplained variation
1. Taking log of zero or negative values in exponential fitting
2. Using wrong normal equations (check: j-th equation has Σxʲy on left)
3. Forgetting to transform back (a = 10ᴬ, not a = A)
4. Overfitting with too high a polynomial degree
5. Not checking: does the curve make sense for the data?
Create computation tables with all columns. This earns method marks even if the final answer has an error.
Check: Σ residuals ≈ 0. Line passes through (x̄,ȳ). bᵧₓ × bₓᵧ = r². −1 ≤ r ≤ 1.
Curve fitting problems take longer (more sums). Correlation/regression are faster once you know the formulas.
Don't just give the number. Say what it means: "r = 0.85 means strong positive linear correlation."
1. Compute Pearson's r from a data table and interpret. (Almost every exam)
2. Find both regression equations from data or from given statistics (x̄, ȳ, Sₓ, Sᵧ, r). (Very common)
3. Compute Spearman's rₛ from ranked data. (Common)
4. Fit y = abˣ or y = axᵇ to given data. (Common)
5. State and explain properties of regression lines or correlation. (Theory question)
6. Given r₁₂, r₁₃, r₂₃, find partial or multiple correlation. (Occasional)
7. Given regression equation, make predictions and interpret coefficients. (Common)
8. Prove that both regression lines pass through (x̄, ȳ). (Occasional proof)
Q1: The least squares method minimises:
Q2: To fit a second-degree polynomial, you need at least:
Q3: To fit y = abˣ, we transform by taking:
Q4: If r = −0.9, then r² = ?
Q5: Spearman's rₛ = 1 − 6Σd²/[n(n²−1)]. If Σd² = 0, then rₛ = ?
Q6: The regression of y on x and x on y both pass through:
Q7: bᵧₓ · bₓᵧ = ?
Q8: If r = 1, the two regression lines:
Q9: For the power curve y = axᵇ, the linearised form is:
Q10: The standard error of estimate Sᵧₓ is related to Sᵧ by:
Q11: Partial correlation r₁₂·₃ measures:
Q12: Which is always true?
Q13: Overfitting in polynomial regression means:
Q14: Anscombe's Quartet shows that:
Q15: If bᵧₓ = 1.5 and bₓᵧ = 0.6, then r = ?
| Situation | Method | Key Formula | Lecture |
|---|---|---|---|
| Data follows a straight line | Fit y = a + bx | Normal equations (2) | L24 |
| Data has one curve (U-shape) | Fit y = a + bx + cx² | Normal equations (3) | L25 |
| Data grows/decays exponentially | Fit y = abˣ or y = aeᵇˣ | Log transform → linear | L26 |
| Data follows a power law | Fit y = axᵇ | Log-log transform → linear | L26 |
| Measure linear association (quantitative) | Pearson's r | r = [nΣxy−ΣxΣy]/√{…} | L28 |
| Measure association (ordinal/non-linear) | Spearman's rₛ | rₛ = 1−6Σd²/[n(n²−1)] | L30 |
| Control for a third variable | Partial correlation | r₁₂·₃ formula | L29 |
| Combined predictive power | Multiple correlation | R²₁·₂₃ formula | L29 |
| Predict y from x | Regression y on x | ŷ = a + bᵧₓx | L31 |
| Predict x from y | Regression x on y | x̂ = a' + bₓᵧy | L31 |
| Assess prediction accuracy | Standard error / R² | Sᵧₓ = Sᵧ√(1−r²) | L32 |
Probability — Random experiments, sample spaces, events, axioms of probability, conditional probability, Bayes' theorem
Unit IV · Lecture 33 · BDS113 Introduction to Statistics · B.Sc. Data Science, Semester II