STAT 337 · Activity 8 · Simple Linear Regression

Activity 8 — Relationships Between Two Quantitative Variables

In this activity you will use three real data sets to describe, measure, model, and make inferences about the relationship between two quantitative variables. Keep your Activity 8 sheet open as you work: each problem on this page matches a section of the sheet, and the games ask you to record your points round by round.

🐧 Palmer Penguins 225 penguins
Researchers at Palmer Station, Antarctica measured adult Adelie, Chinstrap, and Gentoo penguins nesting on islands in the Palmer Archipelago during the 2007/08, 2008/09, and 2009/10 breeding seasons. Before egg laying began, researchers selected and marked nests where a breeding pair of adult penguins was present; the paper does not state how the particular study nests were selected. When a nest reached the one-egg stage, both adults in the breeding pair were captured and measured. Our version contains 225 of these penguins.
  • species — Adelie, Chinstrap, or Gentoo
  • flipper_length_mm — flipper length (mm)
  • bill_length_mm — bill length (mm)
  • bill_depth_mm — bill depth (mm)
  • body_mass_g — body mass (g)
Gorman KB, Williams TD, Fraser WR (2014). Ecological sexual dimorphism and environmental variability within a community of Antarctic penguins (genus Pygoscelis). PLOS ONE 9(3): e90081.
💎 Diamonds 500 diamonds
A random sample of 500 round-cut diamonds from the 53,940 diamonds in the diamonds data set that ships with the ggplot2 package in R.
  • carat — weight of the diamond (carats)
  • price — price in US dollars
  • depth_pct — total depth as a percentage of average width
  • table — width of the top face relative to the widest point (%)
  • length, width — dimensions (mm)
  • cut — Fair, Good, Very Good, Premium, or Ideal
Wickham H (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York.
🚗 Cars 50 cars
The speed of 50 cars and the distance each took to stop, recorded in the 1920s. The data set comes with R as cars.
  • speed — speed of the car (mph)
  • dist — stopping distance (ft)
Ezekiel M (1930). Methods of Correlation Analysis. New York: Wiley.
Focus: describe a relationship from a scatterplot, measure it with r, model it with the least-squares line, judge the fit with R², test it with a permutation test, and check the conditions that inference depends on.
STAT 337 · Activity 8 · Problem 1

Problem 1 — Scatterplots: 6 Things to Look For

Choose a data set and a pair of variables, then describe what you see using the six features in the reference card. Use the sliders and toggles below the plot to see simulated examples of each feature.

REAL DATA
Dataset
Explanatory (x)
Response (y)
Species
Select a dataset and variable pair above
SIMULATED EXAMPLES ONLY — NOT REAL DATA
DEMONSTRATION
Move the sliders and toggles to see a few examples of what each of the six features can look like. These are not the only ways the features can appear: in real data, a curve, changing spread, an outlier, or groups can take many other shapes, and several features often show up together. This plot is made up for practice — use the real-data plot above when you answer questions about the data.
positive
negative positive
strong
weak strong
linear
concave ↓ concave ↑
constant
decreasing increasing
5 · OUTLIERS
6 · DISTINCT GROUPS
6 Things to Look For Reference
  • 1
    Direction
    Is the overall association positive (y tends to increase as x increases), negative (y tends to decrease as x increases), or no association?
  • 2
    Strength
    How tightly do the points cluster around the overall pattern? Strong — points stay close to the pattern. Moderate — the pattern is clear but there is noticeable scatter. Weak — the pattern is faint.
  • 3
    Form / Linearity
    Does the pattern follow a straight line or is it curved / non-linear? Pearson's r measures only the strength of a linear relationship, so a curved relationship may be strong visually even when |r| is small.
  • 4
    Changing Variability
    Does the vertical spread of the points stay about constant, or does it increase or decrease across x?
  • 5
    Outliers
    An outlier is a point that does not follow the overall pattern of the other observations. A point that is far away but still follows the same pattern is not necessarily an outlier.
  • 6
    Distinct Groups
    Do the points form two or more separate clusters? Look for clearly separated bands or clouds. Distinct groups can sometimes be mistaken for one wide, noisy pattern, so check for visible gaps or subgroup structure.
STAT 337 · Activity 8 · Problem 2

Problem 2 — Correlation: Guess the Correlation

Select a data set and a pair of variables, drag the slider to your best estimate of the correlation coefficient r, and submit. r ranges from −1 (perfect negative linear relationship) to +1 (perfect positive linear relationship). Each pair you submit is one round.

TOTAL PTS
0
GUESSES
0
AVG ERROR
—
BULLSEYES
0
SELECT DATA — choose a variable pair then guess r
Dataset
Explanatory (x)
Response (y)
Species
Select a dataset above to begin
Interpreting r reference

Pearson's r measures the strength and direction of a linear association. It always falls between −1 and +1.

+1.00 Perfect positive +0.70 Strong positive +0.30 Weak positive 0.00 No association −0.30 Weak negative −0.70 Strong negative −1.00 Perfect negative

Key warnings:

r only measures the strength and direction of a linear association between two quantitative variables.

Scoring per plot
5 ptsError ≤ 0.05  (bullseye)
4 ptsError ≤ 0.10
3 ptsError ≤ 0.15
2 ptsError ≤ 0.25
1 ptError ≤ 0.35
0 ptsError > 0.35

Error = |your guess − sample r|. Record each round's guess, sample r, and points in the Problem 2 table on your activity sheet.

HOW r IS CALCULATED (CARS DATA)
THE IDEA

The correlation coefficient is built from z-scores: how many standard deviations each value is from its mean, z = (value − mean) / SD. For each car we multiply its two z-scores. The mean lines split the plot into quadrants I–IV (counterclockwise from the upper right). Cars in quadrants I and III (both above average or both below) have a positive product and contribute positively to r; cars in quadrants II and IV have a negative product and contribute negatively. Then

r = Σ (zx × zy) / (n − 1)

Each product is drawn as a rectangle from the center point (x, y) to the car: green if positive, red if negative. Data: speed (mph) and stopping distance (ft) of 50 cars from the 1920s. Use Five activity cars only to see cars A–E from your activity sheet, then place an outlier, car F, and watch everything recalculate.

speed → stopping distance0 / 50 cars
EACH CAR'S CONTRIBUTION
#z speedz distproduct
Press Next Car to begin
RUNNING SUM Σ zxzy
0
THE CAR
Click Next Car or hover any point.
→
① z FOR x
(x − x) / sx
→
② z FOR y
(y − y) / sy
→
③ PRODUCT zx × zy
sign tells whether it contributes positively or negatively to r
CARS INCLUDED
0
r = Σ / (n − 1)
—
LEAST-SQUARES SLOPE b₁
—
MEANS & SDs
—
STAT 337 · Activity 8 · Problem 3

Problem 3 — Least-Squares Line: Minimize the Squared Residuals

Drag the handles on the line to make your SSE as small as you can. Each pair you submit is one round.

TOTAL PTS
0
ATTEMPTS
0
AVG SSE RATIO
—
BULLSEYES
0
SELECT DATA — drag the line to minimize SSE
Dataset
Explanatory (x)
Response (y)
Species
Select a dataset above to begin
🎯 Goal: drag the line until the orange squares are as small as possible
Your Goal
Smallest possible SSE — match this number
—
Get your SSE as close to this number as you can.
The Idea key idea
Scoring per plot

Scored on the ratio your SSE ÷ smallest possible SSE:

5 ptsWithin 2× the smallest possible SSE
4 ptsWithin 3×
3 ptsWithin 5×
2 ptsWithin 10×
1 ptWithin 20×
0 ptsMore than 20×

Record each round's SSE, the smallest SSE, and your points in the Problem 3 table on your activity sheet.

STAT 337 · Activity 8 · Problem 4

Problem 4 — Coefficient of Determination: R²

Drag the line to make R² as large as you can. Each pair you submit is one round.

TOTAL PTS
0
ATTEMPTS
0
AVG R² GAP
—
BULLSEYES
0
SELECT DATA — drag the line to maximize R²
Dataset
Explanatory (x)
Response (y)
Species
Select a dataset above to begin
🎯 Goal: drag the line to make R² as large as possible
Your Goal
Largest possible R² — match this number
—
Drag to get your R² as close to this number as you can.
The Idea key idea
Scoring per plot

Scored on how close your R² is to the largest possible R²:

5 ptsWithin 0.04 of the largest possible R²
4 ptsWithin 0.08
3 ptsWithin 0.15
2 ptsWithin 0.25
1 ptWithin 0.40
0 ptsMore than 0.40 off

Record each round's R², the largest possible R², and your points in the Problem 4 table on your activity sheet.

STAT 337 · Activity 8 · Problem 5

Problem 5 — Permutation Test for the Slope: How is the null distribution built?

If there were no linear relationship between x and y, how likely is a sample slope at least as extreme as ours? Work through the five steps: fit the least-squares line to the observed data, break the pairing by shuffling x, refit the line to the shuffled data, and then repeat the shuffle 1000 times to build the null distribution.

WHAT IS THIS?

A residual is the vertical distance between a point and the line: e = y − ŷ. Squaring the residuals turns them into positive areas and makes large misses count more; adding them gives the sum of squared errors (SSE). The least-squares line is the line with the smallest SSE, and R² = 1 − SSE/SST is the fraction of the variability in y it explains. If x and y were unrelated, shuffling the x values would not change anything important — so the slopes from shuffled data show us what chance alone produces.

1Choose a data set and click Use These Data, then click Draw Residuals — each residual appears from left to right and the list on the right builds up.
2Click Show Squares — each residual is squared and drawn as a box, and the running SSE adds up.
3Click Shuffle x — the x values are randomly reassigned to different y values, breaking the original pairing.
4Click Draw Shuffled Residuals and Show Squares — see how well (or poorly) a line fits the shuffled data.
5Click Run 1000 Permutations — watch the null distribution of b₁* build up, then toggle to R²* to see that it gives the same p-value.
SELECT DATA — choose a variable pair for the test
Dataset
Explanatory (x)
Response (y)
Species
Penguins · flipper_length_mm → body_mass_g · Chinstrap is loaded.
12 THE LEAST-SQUARES LINE & SQUARED RESIDUALS
Click “Draw Residuals” to begin.
RESIDUALS DRAWN
0
RUNNING SSE
0
R²
—
SLOPE b₁
—
RESIDUALS — e = y − ŷ
#ee²
Press Draw Residuals to begin
STAT 337 · Activity 8 · Problem 6

Problem 6 — Bootstrap Confidence Interval: how much could the slope change from sample to sample?

We have one random sample of 30 diamonds from the 53,940 in the ggplot2 data. Its least-squares slope is b₁ = $8,012 per carat — but a different sample of 30 would give a different slope. The bootstrap uses the one sample we have to estimate how much the slope varies, and turns that into a confidence interval.

HOW IT WORKS

Treat the sample as a stand-in for the population. Randomly pick one (carat, price) pair from the 30 diamonds, put it back, and pick again — 30 times in all. Those 30 draws are one bootstrap resample: the same size as the original sample, but with some diamonds appearing more than once and some not at all. Fit the least-squares line to the resample and record its slope, b₁*.

Repeat this 1000 times. The spread of the b₁* values mimics the sample-to-sample variability we would see if we could take many real samples of 30 diamonds. The middle 95% of the b₁* values is a 95% bootstrap confidence interval for the true slope β₁.

When is it useful? When the normality condition is in doubt but the other conditions — linearity, equal variability, independence — are reasonable. The bootstrap does not fix problems with those other conditions.

1 RESAMPLING WITH REPLACEMENT
Why with replacement? If we drew 30 diamonds without replacement we would always get the same 30 back. Putting each one back means some are picked twice and some are skipped, so every resample is a different data set.

Click Draw One to pick a diamond. Blue = picked once, orange = picked again, faded = skipped.

Ready — draw 30 times to complete one resample.
The resample is a new data set. These 30 draws (with repeats) are one bootstrap resample. Fit the least-squares line to them and record the slope b₁*.
RESAMPLE PILE (0/30)
Empty — start drawing
PICKED 2+ TIMES
0
SKIPPED
—
━━ true line (all diamonds)   ┅┅ our sample   ━━ this resample
SAMPLE b₁
$8,012
RESAMPLE b₁*
—
2 BOOTSTRAP DISTRIBUTION & CONFIDENCE INTERVAL
Each resample gives a new b₁*. The left plot shows every bootstrap line on the scatterplot — together they form a band around our sample line. The right plot is the bootstrap distribution of b₁*. The middle portion of that distribution, set by the confidence level, is the bootstrap confidence interval. It is centered near our sample slope ($8,012), not the true slope, because the sample is all we have.
BOOTSTRAP LINES AROUND OUR SAMPLE
Faint blue = one resample's line each. Dashed blue = lines with the slopes at the edges of the interval. Red = true line. Click a dot on the right to highlight that resample.
ALL b₁* VALUES — PERCENTILE NUMBER LINE
CONFIDENCE LEVEL
95%
Drag to see how the confidence level changes the width of the interval and how many b₁* values land in each tail.
BOOTSTRAP DISTRIBUTION OF b₁*
Each dot = one resample's b₁*. Click a dot to see that resample's line on the left.
95% BOOTSTRAP CONFIDENCE INTERVAL FOR β₁
—
RESAMPLES
0
LATEST RESAMPLE b₁*
—
SAMPLE b₁
$8,012
TRUE β₁
$7,753
STAT 337 · Activity 8 · Problem 7

Problem 7 — Residuals: building the residual vs fitted plot

Follow individual cars from the scatterplot to the residual vs fitted plot to see where each point comes from.

THE DATA & THE IDEA

The cars data set in R records the speed (mph) and stopping distance (ft) of 50 cars from the 1920s. The least-squares line is predicted distance = −17.58 + 3.93 × speed.

Every car gives two numbers we can plot. Its fitted value ŷ is where the line is at that car's speed. Its residual e = y − ŷ is how far the car is above (+) or below (−) the line. The residual vs fitted plot puts each car at the point (ŷ, e). Watch one car at a time travel from the scatterplot to the residual plot.

Scatterplot with least-squares linespeed → stopping distance
Residual vs fitted plotx-axis: ŷ · y-axis: e = y − ŷ
THE CAR
Click Next Car or hover any point.
→
① FITTED VALUE ŷ
go up from the car's speed to the line
→
② RESIDUAL e = y − ŷ
vertical gap from the line to the car
→
③ PLOT (ŷ, e)
ŷ goes across, e goes up or down from 0
0 of 50 cars plotted. Green = above the line (e > 0). Red = below the line (e < 0).
Tilt view — the residual plot is the scatterplot with the line laid flat0% tilted
SCATTERPLOT RESIDUAL PLOT
STAT 337 · Activity 8 · Extra (optional exploration)

Extra — The Normal Tunnel: the distribution of y at each x

The simple linear regression model says that at every x, the responses are normally distributed around the population line μy = β₀ + β₁x with the same standard deviation σ. This problem simulates diamonds that follow that model exactly, so you can see what data look like when every condition holds. Simulate a few hundred, then thousands, and watch the tunnel fill in.

🎲 Simulated diamonds — not real data. Each simulated diamond gets a carat value picked at random from our 500 real diamonds, and a price drawn from the model: price = −$2,254 + $7,753·carat + error, with error ~ N(0, $1,548). Real diamonds are checked against this model in Problems 6 and 7.
Simulated diamonds drawn from the modeln = 0 simulated
Estimating σ from the data — the residual standard error
LEAST-SQUARES LINE FOR THE SIMULATED DIAMONDS
—
RESIDUAL STANDARD ERROR σ
—
MODEL σ
$1,548
RATIO σ ÷ σ
—
Cross-section — simulated prices near 0.80 ct
MODEL σ
$1,548
OBSERVED SD
—
RATIO SD ÷ σ
—
n NEAR SLICE
—
Spread by carat — observed SD of simulated prices around the line
How to read it the model

The model: price = β₀ + β₁·carat + ε, with ε ~ N(0, σ²). At every carat value, prices are normal, centered on the red line, with the same spread σ. The line and σ come from all 53,940 ggplot2 diamonds.

The curves: each sideways red curve is the model's distribution of price at one carat value. They move up with the line but never get wider — that is the equal-variability condition. The orange curve follows the slider.

Estimating σ: the residual standard error σ = √(Σeᵢ²/(n − 2)) uses the residuals from the least-squares line to estimate σ. It is our estimate of how far individual prices typically fall from the true mean price at each carat — the width of the tunnel.

Simulated, not real: every point here was generated from the model, so every condition holds by construction. This is what data look like when the model is exactly right.

Try: compare the cross-section and the observed SD ÷ σ ratio with 500 simulated diamonds and again with 5,000 or more. More simulations make the picture match the model more closely.