Unit IV · Lecture 29 of 40 · B.Sc. Data Science, 1st Year, Semester II
Pearson's r measures the relationship between two variables. But in reality, a variable y may be influenced by several variables simultaneously (x₁, x₂, x₃, …). Simple correlation between y and x₁ may be inflated because x₁ is correlated with x₂, which also affects y. We need tools to isolate the effect of one variable while controlling for others.
You notice that restaurants with more staff get higher ratings (positive correlation). But restaurants with more staff also tend to be more expensive. Is it the staff count or the price that drives ratings? Partial correlation answers this: it measures the correlation between staff count and ratings after removing the effect of price. It isolates the "direct" relationship.
Let x₁, x₂, x₃ be three variables.
r₁₂ = simple correlation between x₁ and x₂
r₁₃ = simple correlation between x₁ and x₃
r₂₃ = simple correlation between x₂ and x₃
These are called zero-order correlations (no variables controlled).
Partial correlation: The correlation between two variables after removing the linear effect of one or more other variables.
Multiple correlation: The correlation between one variable (y) and a set of predictors (x₁, x₂, …, xₖ) jointly. It measures how well the combination of predictors explains y.
The partial correlation r₁₂.₃ measures the correlation between x₁ and x₂ after removing the linear effect of x₃ from both. It is a first-order partial correlation (one variable controlled).
r₁₂.₃ = (r₁₂ − r₁₃ · r₂₃) / √[(1 − r₁₃²)(1 − r₂₃²)]
Where r₁₂, r₁₃, r₂₃ are the three zero-order (simple) correlations among the three variables.
Interpretation: r₁₂.₃ tells us the correlation between x₁ and x₂ that remains after we have "factored out" (controlled for) the influence of x₃ on both.
Three variables x₁ (income), x₂ (spending), x₃ (age) have the following correlations:
r₁₂ = 0.80 (income & spending), r₁₃ = 0.60 (income & age), r₂₃ = 0.50 (spending & age).
Find the partial correlation between income and spending, controlling for age.
r₁₂.₃ = (r₁₂ − r₁₃ · r₂₃) / √[(1 − r₁₃²)(1 − r₂₃²)]
Numerator: r₁₂ − r₁₃ · r₂₃ = 0.80 − (0.60)(0.50) = 0.80 − 0.30 = 0.50
Denominator: √[(1 − 0.36)(1 − 0.25)] = √[(0.64)(0.75)] = √0.48 = 0.6928
r₁₂.₃ = 0.50 / 0.6928 = 0.7217
The simple correlation (r₁₂ = 0.80) dropped to 0.722 after controlling for age. The correlation between income and spending is still strong, but part of the original association was due to the shared influence of age.
The multiple correlation coefficient R₁.₂₃ measures how well both x₂ and x₃ together predict x₁. It is the correlation between x₁ and the best linear combination of x₂ and x₃.
R²₁.₂₃ = (r₁₂² + r₁₃² − 2·r₁₂·r₁₃·r₂₃) / (1 − r₂₃²)
R₁.₂₃ = √R²₁.₂₃ (always taken as positive)
Properties:
• 0 ≤ R₁.₂₃ ≤ 1 (always non-negative)
• R₁.₂₃ ≥ max(|r₁₂|, |r₁₃|) — adding predictors never decreases R
• R²₁.₂₃ = proportion of variance in x₁ explained by x₂ and x₃ jointly
Using the same correlations: r₁₂ = 0.80, r₁₃ = 0.60, r₂₃ = 0.50. Find R₁.₂₃.
R²₁.₂₃ = (r₁₂² + r₁₃² − 2·r₁₂·r₁₃·r₂₃) / (1 − r₂₃²)
= (0.64 + 0.36 − 2(0.80)(0.60)(0.50)) / (1 − 0.25)
= (0.64 + 0.36 − 0.48) / 0.75
= 0.52 / 0.75
= 0.6933
R₁.₂₃ = √0.6933 = 0.8327
Age and spending together explain 69.3% of the variation in income (R² = 0.693). This is better than either variable alone (r₁₂² = 0.64, r₁₃² = 0.36).
The coefficient of alienation measures the lack of association — the proportion of variance not explained.
For simple correlation: k = √(1 − r²)
For multiple correlation: K = √(1 − R²)
k = 0: Perfect correlation (r = ±1). No alienation.
k = 1: Zero correlation (r = 0). Complete alienation.
Example: If r = 0.8, then k = √(1−0.64) = √0.36 = 0.6. The "alienated" portion is 0.6.
r² + k² = 1 (for simple correlation)
R² + K² = 1 (for multiple correlation)
The coefficient of alienation is the complement of the coefficient of determination in the Pythagorean sense. As one increases, the other decreases.
Q1: Partial correlation r₁₂.₃ measures:
Q2: The multiple correlation R₁.₂₃ is always:
Q3: If r₁₂ = 0.80 and r₁₂.₃ = 0.10, this suggests:
Q4: The coefficient of alienation when r = 0.6 is:
Q5: R₁.₂₃ is always greater than or equal to:
1. Given r₁₂ = 0.70, r₁₃ = 0.50, r₂₃ = 0.40. Find r₁₂.₃ and R₁.₂₃.
2. Given r₁₂ = 0.90, r₁₃ = 0.85, r₂₃ = 0.95. Find r₁₂.₃. What does the result suggest?
3. If R₁.₂₃ = 0.90, what percentage of variance in x₁ is explained by x₂ and x₃ jointly? What is the coefficient of alienation?
4. Explain the difference between simple, partial, and multiple correlation with an example.
5. BONUS: Show that if r₁₃ = 0 or r₂₃ = 0, then r₁₂.₃ = r₁₂. Interpret this result.
Lecture 30: Spearman's Rank Correlation — a non-parametric alternative to Pearson's r.