Working with experimental data in R
Tables, transformations, and comparisons
Denis O’Meally
City of Hope
By the end
You should be able to:
state what one row represents;
inspect a table before changing it;
choose an operation that matches the biological question;
inspect a derived variable before comparing it;
distinguish a result, its interpretation and the evidence supporting it.
Purpose: prepare the reasoning used in the exercise without completing it.
Ask: If every line runs, which outcome is already established?
Answer: none; execution is not verification.
Build source: BIOSCI504 commit 3730f8b813668f317689ac56fabb299c9a1b072e.
How this session works
LOOK → PREDICT → TRY IN RSTUDIO → DISCUSS → REVEAL
The slides contain the code and precomputed output.
Short pauses are marked Now in RStudio .
Work with your neighbour; the TAs will circulate around the room.
You present from this deck; no instructor RStudio window is needed.
Students use RStudio only at the marked pauses.
Advance to the following slide for the precomputed answer.
The mouse experiment
Before
48 mice
Baseline weight
→
Assigned group
Control
Nutritional treatment
→
sex · cage · batch provide experimental context for each mouse.
Did treated mice gain more weight than control mice?
Treatment varies within cages; treatment and cage are not the same grouping.
Context: 48 mice, one assigned group, weights before and after treatment.
Clarify: treatment varies within cages; cage is context, not the treatment group.
Ask: What information must the table retain to answer the question?
Do not discuss the observed treatment result.
Before seeing the table
Predict:
What should one row represent?
Which columns are needed?
Which column should identify a mouse?
Where should the unit of weight be recorded?
Take answers before advancing.
Listen for mouse ID, treatment, baseline weight and final weight.
Accept sex, cage and batch as useful experimental context.
Before seeing the table
Predict:
What should one row represent?
Which columns are needed?
Which column should identify a mouse?
Where should the unit of weight be recorded?
one mouse
mouse_id
treatment
baseline and final weight
sex, cage, batch
One value per cell. Units belong in metadata or the column name.
Reveal this as a defensible answer, not the only imaginable layout.
Takeaway: table structure should preserve experimental structure.
Transition: inspect the actual representation.
Four rows from the real table
M001
treatment
male
26.9
29.0
C01
B02
M002
treatment
male
26.8
29.4
C01
B03
M003
control
male
26.0
26.8
C01
B01
M004
treatment
male
27.7
30.2
C01
B02
This is a genuine preview from mouse_trial.
Ask learners to locate the identifier, group, measurements and context.
A preview does not establish completeness, uniqueness or plausible ranges.
What does one row represent?
Choose one:
one weight measurement;
one mouse;
one cage;
one treatment group.
Ask for a show of hands before advancing.
Do not resolve the choice on this slide.
What does one row represent?
Choose one:
one weight measurement;
one mouse;
one cage;
one treatment group.
One mouse.
The two measurements occupy different columns on the same mouse row.
Answer: one mouse.
Takeaway: identifiers, duplicates, sample size and change depend on this choice.
Return to the observational unit after transformations and summaries.
Rows, columns and cells
row
one observed mouse
column
one recorded or derived variable
cell
one value for one mouse and variable
mouse_id
the identifier for the observational unit
One value per cell; units belong in metadata or the column name.
Connect the vocabulary directly to the displayed table.
Takeaway: rows are observations, columns are variables, cells are values.
Source: https://datacarpentry.github.io/spreadsheet-ecology-lesson/01-format-data.html
Variables have different roles
identifier
mouse_id
assigned group
treatment
biological/context variables
sex, cage, batch
continuous measurements
baseline_weight_g, final_weight_g
The role follows from the experiment, not merely the R storage type.
Connect to Lecture 2: character and double describe storage.
Treatment, sex, cage and batch are categorical variables stored as text.
Weight is a continuous measurement.
The dictionary is part of the data
mouse_id
Unique mouse identifier.
—
—
treatment
Treatment assignment.
—
control; treatment
sex
Recorded sex.
—
female; male
baseline_weight_g
Weight before treatment.
g
—
final_weight_g
Weight after treatment.
g
—
cage
Housing cage identifier.
—
C01-C06
batch
Experimental batch identifier.
—
B01-B03
A name alone is not a complete variable definition.
Ask: could the R type tell us whether 27500 is a plausible weight?
Answer: no; the dictionary supplies units and biological meaning.
Checkpoint: read the representation
For the cell in row 2 under final_weight_g:
What does its row represent?
What does its column represent?
What does the value represent?
Where does its unit come from?
Take answers before advancing.
Check that row and measurement are no longer being conflated.
Checkpoint: read the representation
For the cell in row 2 under final_weight_g:
What does its row represent?
What does its column represent?
What does the value represent?
Where does its unit come from?
Row: mouse M002.
Column: post-treatment weight.
Cell: M002’s recorded final weight.
Unit: grams, established by _g and the data dictionary.
Reveal and correct any mismatch in the room’s answers.
Takeaway: meaning comes from table structure plus metadata.
Now in RStudio
Run once in the Console:
library (tidyverse)
data (mouse_trial, package = "BIOSCI504" )
Keep mouse_trial unchanged. We will build temporary pipelines from it.
Pause while everyone loads the package data.
The TAs circulate; confirm that mouse_trial appears in the Environment.
The next slides contain precomputed output if anyone falls behind.
A preview is not an inspection
can show examples of values.
It cannot establish:
the total number of records;
whether every ID is unique;
missingness elsewhere;
complete group counts;
the full range of a measurement.
Ask: what could be wrong in row 40 that head() cannot reveal?
Takeaway: a preview samples rows; inspection checks the complete object.
Source: https://datacarpentry.github.io/R-ecology-lesson/instructor/how-r-thinks-about-data.html
Inspection starts with expectations
Is the table complete?
dimensions
missing or extra records
Does one row equal one mouse?
distinct IDs
duplicates
Are measurements present?
missingness
incomplete data
Are groups represented?
counts
imbalance or miscoding
Are values plausible?
ranges + units
impossible or mis-scaled values
Inspection is a sequence of questions, not one command.
Ask learners to state an expected result before each check.
Takeaway: a check is useful when a failure would change the analysis.
Now in RStudio: inspect structure
Predict what each command will tell you, then run it:
dim (mouse_trial)
names (mouse_trial)
glimpse (mouse_trial)
Allow two minutes; pairs can divide the three commands.
Ask for dimensions and one observation about the schema.
Advance for the precomputed output; do not open instructor RStudio.
Now in RStudio: inspect structure
Predict what each command will tell you, then run it:
[1] "mouse_id" "treatment" "sex"
[4] "baseline_weight_g" "final_weight_g" "cage"
[7] "batch"
Rows: 48
Columns: 7
$ mouse_id <chr> "M001", "M002", "M003", "M004", "M005", "M006", "M00…
$ treatment <chr> "treatment", "treatment", "control", "treatment", "c…
$ sex <chr> "male", "male", "male", "male", "female", "female", …
$ baseline_weight_g <dbl> 26.9, 26.8, 26.0, 27.7, 24.1, 24.1, 23.4, 22.9, 23.2…
$ final_weight_g <dbl> 29.0, 29.4, 26.8, 30.2, 25.3, 25.3, 25.1, 26.1, 25.5…
$ cage <chr> "C01", "C01", "C01", "C01", "C01", "C01", "C01", "C0…
$ batch <chr> "B02", "B03", "B01", "B02", "B01", "B03", "B03", "B0…
Answer: 48 rows and seven columns.
Point out character identifiers/groups and double weight measurements.
glimpse() does not test every data-quality condition.
Check the observational unit
mouse_trial |>
summarise (
rows = n (),
distinct_mice = n_distinct (mouse_id)
)
What relationship do you expect between the two results?
Ask for the expected relationship before students run it.
Allow one minute, then advance for the precomputed answer.
The identifiers support the row claim
mouse_trial |>
summarise (
rows = n (),
distinct_mice = n_distinct (mouse_id)
)
rows distinct_mice
1 48 48
Rows equal distinct mouse IDs in this table.
Answer: both are 48.
A mismatch would trigger investigation; it would not identify the cause.
Do not introduce the challenge fixture here.
Now in RStudio: groups and missingness
Before running, predict the number of groups and what 0 would mean:
mouse_trial |>
count (treatment)
mouse_trial |>
summarise (
across (
c (baseline_weight_g, final_weight_g),
~ sum (is.na (.x))
)
)
Take predictions, then allow two minutes to run both checks.
Advance for the precomputed output.
Now in RStudio: groups and missingness
Before running, predict the number of groups and what 0 would mean:
mouse_trial |>
count (treatment)
treatment n
1 control 26
2 treatment 22
mouse_trial |>
summarise (
across (
c (baseline_weight_g, final_weight_g),
~ sum (is.na (.x))
)
)
baseline_weight_g final_weight_g
1 1 1
Group counts establish denominators before comparison.
There is one missing baseline weight and one missing final weight.
These need not occur in the same mouse; the derived change has two missing values.
Later checks may examine missingness within groups.
Now in RStudio: check ranges against units
Predict a plausible weight range in grams, then run:
mouse_trial |>
summarise (
across (
ends_with ("_weight_g" ),
~ list (range (.x, na.rm = TRUE ))
)
)
The suffix _g makes the expected scale testable.
Ask for a rough plausible range before execution.
Advance for the precomputed output.
Now in RStudio: check ranges against units
Predict a plausible weight range in grams, then run:
mouse_trial |>
summarise (
across (
ends_with ("_weight_g" ),
~ list (range (.x, na.rm = TRUE ))
)
)
baseline_weight_g final_weight_g
1 22.3, 28.3 23.1, 34.1
Compare the output with the dictionary and biological expectation.
A valid double can still use the wrong unit.
Takeaway: plausible storage is not plausible biology.
Checkpoint: choose the revealing check
Which check is most direct?
the same mouse appears twice
?
one weight uses another unit
?
final weights are missing mainly in one group
?
Ask pairs to supply one check for each concern.
Require them to name the evidence each check would produce.
Checkpoint: choose the revealing check
Which check is most direct?
the same mouse appears twice
?
one weight uses another unit
?
final weights are missing mainly in one group
?
Duplicate mouse: compare rows with distinct IDs, then inspect duplicates.
Wrong unit: compare ranges or plots with the documented unit.
Group-dependent missingness: count missing values within treatment.
Reveal these as defensible checks, not magic commands.
Takeaway: each concern should map to evidence capable of exposing it.
The question determines the operation
Final weight
Are treated mice heavier at the end?
Weight change
Did treated mice gain more weight?
These are different biological questions.
This is the conceptual centrepiece.
Final weight combines starting weight and change during the study.
Neither target is inherently invalid; the operation must match the question.
Same final weight, different change
Mouse A gained 5 g . Mouse B gained 1 g . Both finished at 25 g .
Ask: which quantity answers the study question?
Takeaway: paired measurements retain information the final value alone cannot.
This illustration does not reveal the treatment comparison.
Valid code can answer the wrong question
Question: Did treated mice gain more weight?
mouse_trial |>
group_by (treatment) |>
summarise (
n = sum (! is.na (final_weight_g)),
mean_final_g = mean (final_weight_g, na.rm = TRUE )
)
This code runs. Which question does it actually answer?
Ask learners to name the mismatch before advancing.
The code is valid and its output could look reasonable.
Do not run or reveal the treatment means.
Valid code can answer the wrong question
Question: Did treated mice gain more weight?
mouse_trial |>
group_by (treatment) |>
summarise (
n = sum (! is.na (final_weight_g)),
mean_final_g = mean (final_weight_g, na.rm = TRUE )
)
This code runs. Which question does it actually answer?
It compares mean final weight between treatment groups—not weight gain.
“It ran” does not establish that the code implemented the scientific question.
Answer: it asks whether groups differ in final weight.
Takeaway: compare the specification with the executed operation.
Name the quantity before coding
For each mouse:
\[
\text{weight change (g)} = \text{final weight (g)} - \text{baseline weight (g)}
\]
Expected sign: positive for gain, negative for loss.
Move from words to arithmetic to an R name.
Units remain grams because grams are subtracted from grams.
Ask for the expected sign before showing the transformation.
Now in RStudio: derive the variable
mouse_trial |>
mutate (
weight_change_g = ____________________
)
What should remain unchanged after mutate()?
Give pairs one minute to complete the expression.
Answer: final_weight_g - baseline_weight_g.
Invariant: still 48 rows, each representing one mouse.
The derived table state
mouse_trial |>
mutate (
weight_change_g = final_weight_g - baseline_weight_g
) |>
select (mouse_id, baseline_weight_g, final_weight_g, weight_change_g) |>
slice_head (n = 6 ) |>
knitr:: kable ()
M001
26.9
29.0
2.1
M002
26.8
29.4
2.6
M003
26.0
26.8
0.8
M004
27.7
30.2
2.5
M005
24.1
25.3
1.2
M006
24.1
25.3
1.2
mutate() adds a column; it does not change what one row represents.
Ask students to verify one subtraction from the displayed values.
Takeaway: inspect the new table state before continuing.
Source: https://datacarpentry.github.io/R-ecology-lesson/instructor/working-with-data.html#making-new-columns-with-mutate
Inspect the new state
mouse_trial |>
mutate (
weight_change_g = final_weight_g - baseline_weight_g
) |>
summarise (
type = typeof (weight_change_g),
missing = sum (is.na (weight_change_g)),
minimum = min (weight_change_g, na.rm = TRUE ),
maximum = max (weight_change_g, na.rm = TRUE )
)
type missing minimum maximum
1 double 2 0.4 6.1
Ask what each check contributes: type, missingness and plausible range.
Stop before grouping the real changes by treatment.
Takeaway: inspect → transform → inspect again.
Plot individual observations first
Neutral illustration: both groups have the same mean but different spread.
Ask what is visible before the mean: sample size, values, spread and extremes.
A summary deliberately removes detail
Same mean; different evidence.
Ask what is lost if only the diamonds are shown.
Takeaway: retain observations beside summaries.
Predict the grouped summary
tibble (
comparison_group = rep (c ("A" , "B" ), each = 5 ),
outcome = c (1 , 2 , 3 , 4 , 5 , 2.8 , 2.9 , 3 , 3.1 , 3.2 )
) |>
group_by (comparison_group) |>
summarise (
n = n (),
mean = mean (outcome),
.groups = "drop"
)
Before running: how many rows should the result contain?
Ask for the row count and new observational unit before advancing.
Do not substitute the real treatment comparison.
Predict the grouped summary
tibble (
comparison_group = rep (c ("A" , "B" ), each = 5 ),
outcome = c (1 , 2 , 3 , 4 , 5 , 2.8 , 2.9 , 3 , 3.1 , 3.2 )
) |>
group_by (comparison_group) |>
summarise (
n = n (),
mean = mean (outcome),
.groups = "drop"
)
# A tibble: 2 × 3
comparison_group n mean
<chr> <int> <dbl>
1 A 5 3
2 B 5 3
One row now represents one group.
Answer: two rows, one per group.
Keep n beside the mean so the denominator remains visible.
Source: https://datacarpentry.github.io/R-ecology-lesson/instructor/working-with-data.html#the-split-apply-combine-approach
Uncertainty belongs with an estimate
observed difference
size and direction in these data
confidence interval
effect sizes compatible with the data and model under the stated procedure
p-value
how unusual the result would be if a specified null model were true
None establishes that the design, data handling or code are correct.
Keep the statistical treatment modest.
A confidence interval is not the range containing 95% of mice.
A p-value is neither the probability of the null nor biological importance.
Do not calculate the exercise’s final values.
Before choosing a test, record the assumptions
What is the observational unit?
Are observations independent enough for the proposed comparison?
How were treatment and measurements assigned or collected?
Are missing values plausibly ignorable?
Do spread, shape or unusual values challenge the summary?
Are all measurements expressed in the documented unit?
Shared cages can challenge a simple independence assumption.
Cage and batch may contribute variation; do not turn this into a mixed-model lecture.
Test choice follows the question, design and observed data.
Visualization is verification evidence
A plot can reveal:
spread hidden by a mean;
an observation on the wrong scale;
a group with few usable values;
an unexpected relationship between baseline and final weight;
a transformation whose sign or range is implausible.
The figure should be checked before it is interpreted.
A plot is an analytical result and a diagnostic artifact.
In the exercise, retain observations and check agreement with numerical results.
Checkpoint: what does the summary row represent?
After group_by(treatment) and summarise():
Does one row still represent one mouse?
What does n establish?
Which feature of the individual data can the mean conceal?
Which assumption could shared cages challenge?
One row represents one treatment group—not one mouse.
n records the denominator used by the summary.
A mean can conceal spread, overlap, clusters or outliers.
Shared cages can challenge simple independence.
Take answers before advancing.
Use the response to decide whether the class is ready for the exercise.
Checkpoint: what does the summary row represent?
After group_by(treatment) and summarise():
Does one row still represent one mouse?
What does n establish?
Which feature of the individual data can the mean conceal?
Which assumption could shared cages challenge?
One row represents one treatment group—not one mouse.
n records the denominator used by the summary.
A mean can conceal spread, overlap, clusters or outliers.
Shared cages can challenge simple independence.
Reveal each answer and correct the observational-unit language first.
Takeaway: summaries change representation and remove detail.
Result is not interpretation
Result
observed difference
uncertainty
sample size
visual pattern
Interpretation
relation to the biological question
plausible explanations
design limits
remaining uncertainty
Write the numerical and visual result before the biological explanation.
Avoid causal language beyond the design.
The design limits the claim
The table records:
assigned treatment;
paired weights;
sex, cage and batch;
missing measurements.
It does not automatically establish:
the cause of every individual difference;
independence within cages;
why a measurement is missing;
generality beyond the studied mice and conditions.
Treat limits as properties of the design, not a ritual disclaimer.
Ask what additional evidence would change one conclusion.
Why I trust this
rows match distinct mouse IDs
observational unit represented once
missingness recorded by group
effective sample is visible
ranges agree with units
scale is plausible
derived values inspected
transformation behaved as specified
individuals agree with summary
result is not a summary artifact
uncertainty and assumptions stated
limits remain visible
Trust points to evidence. Remaining doubt should also be recorded.
Evidence must be specific and inspectable.
“I checked carefully” is not evidence.
Ask for one remaining doubt after every listed check passes.
AI can explain one step
Give it the local context:
Here is one inspection command and its output.
1. Explain what the command checks.
2. Name one concerning result and why it matters.
3. Do not repair the data or continue the analysis.
Then test the explanation against the table or a small code change.
Generated code can reduce typing. The question, context, output and assumptions still require scientific evaluation.
Keep this bounded to explanation or debugging.
Retain the command and output beside the AI response; check the explanation with R.
Use Teams Copilot; do not send sensitive data to an unapproved service.
The exercise follows the same reasoning sequence
Situation
→ question and prediction
→ how the data arose
→ analysis specification
→ inspect
→ compare final weight
→ derive and compare change
→ verify
→ result
→ interpretation
→ why I trust this
This is a map, not a solution.
The exercise asks about final weight and change so students can distinguish the targets.
The Quarto document remains the executable record.
Begin with the question
Open:
exercises/tabular-comparison/analysis.qmd
Start with:
question and prediction;
how the data arose;
analysis specification;
inspection.
Do not begin by requesting the complete analysis from AI.
If needed: BIOSCI504::copy_template("lecture-3") from the course Project.
Do not overwrite an existing exercise directory.
Give students time to commit to a prediction before comparison.
Questions
Then move into the tabular-comparison exercise.
Take questions about the reasoning sequence.
Handle workstation-specific problems individually during the exercise.
Further reading
Adapted from Data Carpentry and Software Carpentry instructional materials, Copyright © The Carpentries, CC BY 4.0 . Changes were made for BIOSCI 504; no endorsement is implied.
Tables and figures use the public BIOSCI504 package commit recorded in the source.