📝 Problem
Given bᵧₓ = 0.6 and bₓᵧ = 1.0 (from our earlier example). Find the angle between the regression lines.
Unit IV · Lecture 32 of 40 · B.Sc. Data Science, 1st Year, Semester II
y − ȳ = bᵧₓ(x − x̄)
bᵧₓ = Cov(x,y)/Var(x)
bᵧₓ = r·(Sᵧ/Sₓ)
x − x̄ = bₓᵧ(y − ȳ)
bₓᵧ = Cov(x,y)/Var(y)
bₓᵧ = r·(Sₓ/Sᵧ)
r² = bᵧₓ × bₓᵧ
r = ±√(bᵧₓ · bₓᵧ)
Both pass through (x̄,ȳ)
In this lecture, we go deeper into the properties and applications of regression coefficients: the angle between regression lines, the standard error of estimate, the coefficient of determination in regression context, and comprehensive worked examples that tie together everything from Lectures 27–32.
If θ is the acute angle between the two regression lines:
tan θ = |bᵧₓ − bₓᵧ| / (1 + bᵧₓ · bₓᵧ)
Since bᵧₓ · bₓᵧ = r²:
tan θ = |bᵧₓ − bₓᵧ| / (1 + r²)
If r = ±1: bᵧₓ = bₓᵧ, so tan θ = 0, θ = 0°. The lines coincide.
If r = 0: bᵧₓ = bₓᵧ = 0. One line is horizontal (y = ȳ), the other is vertical (x = x̄). θ = 90°.
As |r| increases from 0 to 1: θ decreases from 90° to 0°. The lines get closer together.
The angle θ is a visual measure of correlation strength:
Small angle (near 0°): Lines nearly overlap → strong correlation.
Large angle (near 90°): Lines nearly perpendicular → weak correlation.
θ = 45°: Moderate correlation. This occurs when r² ≈ 0.38 (r ≈ 0.62).
Given bᵧₓ = 0.6 and bₓᵧ = 1.0 (from our earlier example). Find the angle between the regression lines.
r² = bᵧₓ × bₓᵧ = 0.6 × 1.0 = 0.60
tan θ = |bᵧₓ − bₓᵧ| / (1 + r²)
= |0.6 − 1.0| / (1 + 0.60)
= 0.4 / 1.6
= 0.25
θ = tan⁻¹(0.25) = 14.04°
The angle is relatively small, consistent with a moderately strong correlation (r = 0.775). The two regression lines are fairly close together.
The standard error of estimate (Sᵧₓ) measures the scatter of observed y-values around the regression line. It is analogous to the standard deviation — while SD measures scatter around the mean, Sᵧₓ measures scatter around the regression line.
Sᵧₓ = √[Σ(yᵢ − ŷᵢ)² / n]
Shortcut formula (using r):
Sᵧₓ = Sᵧ · √(1 − r²)
Where Sᵧ is the standard deviation of y.
Interpretation: About 68% of observed y-values fall within ±Sᵧₓ of the regression line (if errors are normally distributed). A smaller Sᵧₓ means a better fit.
The total variation in y can be decomposed:
Total SS = Regression SS + Residual SS
Σ(yᵢ − ȳ)² = Σ(ŷᵢ − ȳ)² + Σ(yᵢ − ŷᵢ)²
R² = Regression SS / Total SS
= 1 − (Residual SS / Total SS)
= 1 − [Σ(yᵢ − ŷᵢ)² / Σ(yᵢ − ȳ)²]
For simple linear regression: R² = r²
R² = 0.85: 85% of the variation in y is explained by the regression on x. Only 15% is unexplained (residual).
R² = 0.30: Only 30% explained. The model is weak — other factors dominate.
R² = 1.00: Perfect fit. All points on the line.
R² = 0.00: x explains none of the variation in y.
In multiple regression, adjusted R² penalises adding unnecessary predictors:
R²adj = 1 − [(1−R²)(n−1)/(n−k−1)]
Where k = number of predictors. This prevents R² from artificially increasing when more variables are added, even if they are irrelevant.
(For simple regression with one predictor, adjusted R² is very close to R².)
Given: n = 6, Σx = 21, Σy = 42, Σxy = 161, Σx² = 91, Σy² = 322. Find: r, both regression lines, R², Sᵧₓ, and the angle between the lines.
x̄ = 21/6 = 3.5, ȳ = 42/6 = 7.0
S²ₓ = Σx²/n − x̄² = 91/6 − 12.25 = 15.167 − 12.25 = 2.917 → Sₓ = 1.708
S²ᵧ = Σy²/n − ȳ² = 322/6 − 49 = 53.667 − 49 = 4.667 → Sᵧ = 2.160
Cov(x,y) = Σxy/n − x̄ȳ = 161/6 − 24.5 = 26.833 − 24.5 = 2.333
r = Cov(x,y) / (Sₓ · Sᵧ) = 2.333 / (1.708 × 2.160) = 2.333 / 3.689 = 0.6324
r² = 0.3999 ≈ 0.40 (40% of variance explained)
bᵧₓ = r · (Sᵧ/Sₓ) = 0.6324 × (2.160/1.708) = 0.6324 × 1.2646 = 0.7998
bₓᵧ = r · (Sₓ/Sᵧ) = 0.6324 × (1.708/2.160) = 0.6324 × 0.7907 = 0.5000
Check: r² = bᵧₓ × bₓᵧ = 0.7998 × 0.5000 = 0.3999 ✓
y on x: y − 7.0 = 0.80(x − 3.5) → y = 4.20 + 0.80x
x on y: x − 3.5 = 0.50(y − 7.0) → x = 0.00 + 0.50y
Sᵧₓ = Sᵧ · √(1 − r²) = 2.160 × √(1 − 0.40) = 2.160 × √0.60 = 2.160 × 0.7746 = 1.673
tan θ = |bᵧₓ − bₓᵧ| / (1 + r²) = |0.80 − 0.50| / (1 + 0.40) = 0.30 / 1.40 = 0.2143
θ = tan⁻¹(0.2143) = 12.1°
| Quantity | Formula | Interpretation |
|---|---|---|
| Pearson's r | [nΣxy−ΣxΣy] / √{[nΣx²−(Σx)²][nΣy²−(Σy)²]} | Strength & direction of linear relationship |
| Spearman's ρ | 1 − 6Σd²/[n(n²−1)] | Strength of monotonic (rank) relationship |
| bᵧₓ (y on x) | Cov(x,y)/Var(x) = r·(Sᵧ/Sₓ) | Avg change in y per unit change in x |
| bₓᵧ (x on y) | Cov(x,y)/Var(y) = r·(Sₓ/Sᵧ) | Avg change in x per unit change in y |
| r² = R² | bᵧₓ × bₓᵧ | % of variance in y explained by x |
| Sᵧₓ | Sᵧ·√(1−r²) | Scatter around regression line |
| Angle θ | tan⁻¹[|bᵧₓ−bₓᵧ|/(1+r²)] | Angle between regression lines |
| r₁₂.₃ | (r₁₂−r₁₃r₂₃)/√[(1−r₁₃²)(1−r₂₃²)] | Correlation controlling for x₃ |
| R₁.₂₃ | √[(r₁₂²+r₁₃²−2r₁₂r₁₃r₂₃)/(1−r₂₃²)] | Multiple correlation (x₂,x₃ predict x₁) |
Q1: If r = 0.8 and Sᵧ = 10, the standard error of estimate is:
Q2: When r = ±1, the angle between regression lines is:
Q3: If bᵧₓ = 2 and bₓᵧ = 0.5, then r equals:
Q4: Sᵧₓ = 0 means:
Q5: R² = 0.64 means:
Q6: The total sum of squares equals:
Lecture 24: Principle of Least Squares
Lecture 25: Fitting of Polynomials
Lecture 26: Fitting of Exponential Curves
Lecture 27: Bivariate Data & Scatter Diagrams
Lecture 28: Karl Pearson's Coefficient of Correlation
Lecture 29: Partial & Multiple Correlation
Lecture 30: Spearman's Rank Correlation
Lecture 31: Simple Linear Regression
Lecture 32: Coefficient of Regression
1. For the data x = 1,3,5,7,9 and y = 2,6,10,14,18: find r, both regression lines, R², Sᵧₓ, and the angle between the lines.
2. Explain the difference between correlation and regression. When would you use each?
3. A dataset has r = 0.95 between x and y. Does this mean x causes y? Explain.
4. Compare Pearson's r and Spearman's ρ. When would you prefer one over the other?
5. What is the relationship between the coefficient of determination and the standard error of estimate?
Unit IV · Lectures 24–32 · BDS113 Introduction to Statistics · B.Sc. Data Science, Semester II