Working with experimental data in R

Tables, transformations, and comparisons

Denis O’Meally

City of Hope

By the end

You should be able to:

  1. state what one row represents;
  2. inspect a table before changing it;
  3. choose an operation that matches the biological question;
  4. inspect a derived variable before comparing it;
  5. distinguish a result, its interpretation and the evidence supporting it.

How this session works

LOOK → PREDICT → TRY IN RSTUDIO → DISCUSS → REVEAL
  • The slides contain the code and precomputed output.
  • Short pauses are marked Now in RStudio.
  • Work with your neighbour; the TAs will circulate around the room.

The mouse experiment

Before

48 mice

Baseline weight

Assigned group

Control

Nutritional treatment

After

Final weight

sex · cage · batch provide experimental context for each mouse.

Did treated mice gain more weight than control mice?

Treatment varies within cages; treatment and cage are not the same grouping.

Before seeing the table

Predict:

  • What should one row represent?
  • Which columns are needed?
  • Which column should identify a mouse?
  • Where should the unit of weight be recorded?

Before seeing the table

Predict:

  • What should one row represent?
  • Which columns are needed?
  • Which column should identify a mouse?
  • Where should the unit of weight be recorded?
One row Identifier Assigned group Measurements Context
one mouse mouse_id treatment baseline and final weight sex, cage, batch

One value per cell. Units belong in metadata or the column name.

Four rows from the real table

mouse_id treatment sex baseline_weight_g final_weight_g cage batch
M001 treatment male 26.9 29.0 C01 B02
M002 treatment male 26.8 29.4 C01 B03
M003 control male 26.0 26.8 C01 B01
M004 treatment male 27.7 30.2 C01 B02

What does one row represent?

Choose one:

  1. one weight measurement;
  2. one mouse;
  3. one cage;
  4. one treatment group.

What does one row represent?

Choose one:

  1. one weight measurement;
  2. one mouse;
  3. one cage;
  4. one treatment group.

One mouse.

The two measurements occupy different columns on the same mouse row.

Rows, columns and cells

Part Meaning in this table
row one observed mouse
column one recorded or derived variable
cell one value for one mouse and variable
mouse_id the identifier for the observational unit

One value per cell; units belong in metadata or the column name.

Variables have different roles

Role Variables
identifier mouse_id
assigned group treatment
biological/context variables sex, cage, batch
continuous measurements baseline_weight_g, final_weight_g

The role follows from the experiment, not merely the R storage type.

The dictionary is part of the data

variable description units allowed_values
mouse_id Unique mouse identifier.
treatment Treatment assignment. control; treatment
sex Recorded sex. female; male
baseline_weight_g Weight before treatment. g
final_weight_g Weight after treatment. g
cage Housing cage identifier. C01-C06
batch Experimental batch identifier. B01-B03

Checkpoint: read the representation

For the cell in row 2 under final_weight_g:

  • What does its row represent?
  • What does its column represent?
  • What does the value represent?
  • Where does its unit come from?

Checkpoint: read the representation

For the cell in row 2 under final_weight_g:

  • What does its row represent?
  • What does its column represent?
  • What does the value represent?
  • Where does its unit come from?
  • Row: mouse M002.
  • Column: post-treatment weight.
  • Cell: M002’s recorded final weight.
  • Unit: grams, established by _g and the data dictionary.

Now in RStudio

Run once in the Console:

library(tidyverse)
data(mouse_trial, package = "BIOSCI504")

Keep mouse_trial unchanged. We will build temporary pipelines from it.

A preview is not an inspection

head(mouse_trial)

can show examples of values.

It cannot establish:

  • the total number of records;
  • whether every ID is unique;
  • missingness elsewhere;
  • complete group counts;
  • the full range of a measurement.

Inspection starts with expectations

Question Check Concerning result
Is the table complete? dimensions missing or extra records
Does one row equal one mouse? distinct IDs duplicates
Are measurements present? missingness incomplete data
Are groups represented? counts imbalance or miscoding
Are values plausible? ranges + units impossible or mis-scaled values

Inspection is a sequence of questions, not one command.

Now in RStudio: inspect structure

Predict what each command will tell you, then run it:

dim(mouse_trial)
names(mouse_trial)
glimpse(mouse_trial)

Now in RStudio: inspect structure

Predict what each command will tell you, then run it:

dim(mouse_trial)
[1] 48  7
names(mouse_trial)
[1] "mouse_id"          "treatment"         "sex"              
[4] "baseline_weight_g" "final_weight_g"    "cage"             
[7] "batch"            
glimpse(mouse_trial)
Rows: 48
Columns: 7
$ mouse_id          <chr> "M001", "M002", "M003", "M004", "M005", "M006", "M00…
$ treatment         <chr> "treatment", "treatment", "control", "treatment", "c…
$ sex               <chr> "male", "male", "male", "male", "female", "female", …
$ baseline_weight_g <dbl> 26.9, 26.8, 26.0, 27.7, 24.1, 24.1, 23.4, 22.9, 23.2…
$ final_weight_g    <dbl> 29.0, 29.4, 26.8, 30.2, 25.3, 25.3, 25.1, 26.1, 25.5…
$ cage              <chr> "C01", "C01", "C01", "C01", "C01", "C01", "C01", "C0…
$ batch             <chr> "B02", "B03", "B01", "B02", "B01", "B03", "B03", "B0…

Check the observational unit

mouse_trial |>
  summarise(
    rows = n(),
    distinct_mice = n_distinct(mouse_id)
  )

What relationship do you expect between the two results?

The identifiers support the row claim

mouse_trial |>
  summarise(
    rows = n(),
    distinct_mice = n_distinct(mouse_id)
  )
  rows distinct_mice
1   48            48

Rows equal distinct mouse IDs in this table.

Now in RStudio: groups and missingness

Before running, predict the number of groups and what 0 would mean:

mouse_trial |>
  count(treatment)

mouse_trial |>
  summarise(
    across(
      c(baseline_weight_g, final_weight_g),
      ~ sum(is.na(.x))
    )
  )

Now in RStudio: groups and missingness

Before running, predict the number of groups and what 0 would mean:

mouse_trial |>
  count(treatment)
  treatment  n
1   control 26
2 treatment 22
mouse_trial |>
  summarise(
    across(
      c(baseline_weight_g, final_weight_g),
      ~ sum(is.na(.x))
    )
  )
  baseline_weight_g final_weight_g
1                 1              1

Now in RStudio: check ranges against units

Predict a plausible weight range in grams, then run:

mouse_trial |>
  summarise(
    across(
      ends_with("_weight_g"),
      ~ list(range(.x, na.rm = TRUE))
    )
  )

The suffix _g makes the expected scale testable.

Now in RStudio: check ranges against units

Predict a plausible weight range in grams, then run:

mouse_trial |>
  summarise(
    across(
      ends_with("_weight_g"),
      ~ list(range(.x, na.rm = TRUE))
    )
  )
  baseline_weight_g final_weight_g
1        22.3, 28.3     23.1, 34.1

Checkpoint: choose the revealing check

Which check is most direct?

Concern Candidate check
the same mouse appears twice ?
one weight uses another unit ?
final weights are missing mainly in one group ?

Checkpoint: choose the revealing check

Which check is most direct?

Concern Candidate check
the same mouse appears twice ?
one weight uses another unit ?
final weights are missing mainly in one group ?
  • Duplicate mouse: compare rows with distinct IDs, then inspect duplicates.
  • Wrong unit: compare ranges or plots with the documented unit.
  • Group-dependent missingness: count missing values within treatment.

The question determines the operation

Final weight

Are treated mice heavier at the end?

Weight change

Did treated mice gain more weight?

These are different biological questions.

Same final weight, different change

Two mice both finish at 25 grams. Mouse A rises from 20 to 25 grams, while Mouse B rises from 24 to 25 grams.

Mouse A gained 5 g. Mouse B gained 1 g. Both finished at 25 g.

Valid code can answer the wrong question

Question: Did treated mice gain more weight?

mouse_trial |>
  group_by(treatment) |>
  summarise(
    n = sum(!is.na(final_weight_g)),
    mean_final_g = mean(final_weight_g, na.rm = TRUE)
  )

This code runs. Which question does it actually answer?

Valid code can answer the wrong question

Question: Did treated mice gain more weight?

mouse_trial |>
  group_by(treatment) |>
  summarise(
    n = sum(!is.na(final_weight_g)),
    mean_final_g = mean(final_weight_g, na.rm = TRUE)
  )

This code runs. Which question does it actually answer?

It compares mean final weight between treatment groups—not weight gain.

“It ran” does not establish that the code implemented the scientific question.

Name the quantity before coding

For each mouse:

\[ \text{weight change (g)} = \text{final weight (g)} - \text{baseline weight (g)} \]

Expected sign: positive for gain, negative for loss.

Now in RStudio: derive the variable

mouse_trial |>
  mutate(
    weight_change_g = ____________________
  )

What should remain unchanged after mutate()?

The derived table state

mouse_trial |>
  mutate(
    weight_change_g = final_weight_g - baseline_weight_g
  ) |>
  select(mouse_id, baseline_weight_g, final_weight_g, weight_change_g) |>
  slice_head(n = 6) |>
  knitr::kable()
mouse_id baseline_weight_g final_weight_g weight_change_g
M001 26.9 29.0 2.1
M002 26.8 29.4 2.6
M003 26.0 26.8 0.8
M004 27.7 30.2 2.5
M005 24.1 25.3 1.2
M006 24.1 25.3 1.2

mutate() adds a column; it does not change what one row represents.

Inspect the new state

mouse_trial |>
  mutate(
    weight_change_g = final_weight_g - baseline_weight_g
  ) |>
  summarise(
    type = typeof(weight_change_g),
    missing = sum(is.na(weight_change_g)),
    minimum = min(weight_change_g, na.rm = TRUE),
    maximum = max(weight_change_g, na.rm = TRUE)
  )
    type missing minimum maximum
1 double       2     0.4     6.1

Missing inputs propagate

final weight     baseline weight     change
    25.3 g    −       NA          =    NA
      NA      −      24.2 g       =    NA

The mouse remains a row. The derived measurement is missing.

Checkpoint: predict the transformation

For one mouse:

baseline = 24.1 g
final    = 25.3 g

Before calculating:

  • What sign should the change have?
  • What is its value and unit?
  • What does the row represent after adding the column?

Checkpoint: predict the transformation

For one mouse:

baseline = 24.1 g
final    = 25.3 g

Before calculating:

  • What sign should the change have?
  • What is its value and unit?
  • What does the row represent after adding the column?
  • Sign: positive.
  • Value: 1.2 g.
  • Row: still the same mouse.

Adding a derived variable changes columns, not the observational unit.

Plot individual observations first

Two dot plots with five observations each. Group A spans outcomes 1 to 5, while group B is tightly clustered around 3.

A summary deliberately removes detail

The same two dot plots with diamond mean markers. Both means equal 3 despite the much wider spread in group A.

Same mean; different evidence.

Predict the grouped summary

tibble(
  comparison_group = rep(c("A", "B"), each = 5),
  outcome = c(1, 2, 3, 4, 5, 2.8, 2.9, 3, 3.1, 3.2)
) |>
  group_by(comparison_group) |>
  summarise(
    n = n(),
    mean = mean(outcome),
    .groups = "drop"
  )

Before running: how many rows should the result contain?

Predict the grouped summary

tibble(
  comparison_group = rep(c("A", "B"), each = 5),
  outcome = c(1, 2, 3, 4, 5, 2.8, 2.9, 3, 3.1, 3.2)
) |>
  group_by(comparison_group) |>
  summarise(
    n = n(),
    mean = mean(outcome),
    .groups = "drop"
  )
# A tibble: 2 × 3
  comparison_group     n  mean
  <chr>            <int> <dbl>
1 A                    5     3
2 B                    5     3

One row now represents one group.

Uncertainty belongs with an estimate

Quantity What it reports
observed difference size and direction in these data
confidence interval effect sizes compatible with the data and model under the stated procedure
p-value how unusual the result would be if a specified null model were true

None establishes that the design, data handling or code are correct.

Before choosing a test, record the assumptions

  • What is the observational unit?
  • Are observations independent enough for the proposed comparison?
  • How were treatment and measurements assigned or collected?
  • Are missing values plausibly ignorable?
  • Do spread, shape or unusual values challenge the summary?
  • Are all measurements expressed in the documented unit?

Visualization is verification evidence

A plot can reveal:

  • spread hidden by a mean;
  • an observation on the wrong scale;
  • a group with few usable values;
  • an unexpected relationship between baseline and final weight;
  • a transformation whose sign or range is implausible.

The figure should be checked before it is interpreted.

Checkpoint: what does the summary row represent?

After group_by(treatment) and summarise():

  1. Does one row still represent one mouse?
  2. What does n establish?
  3. Which feature of the individual data can the mean conceal?
  4. Which assumption could shared cages challenge?
  1. One row represents one treatment group—not one mouse.
  2. n records the denominator used by the summary.
  3. A mean can conceal spread, overlap, clusters or outliers.
  4. Shared cages can challenge simple independence.

Checkpoint: what does the summary row represent?

After group_by(treatment) and summarise():

  1. Does one row still represent one mouse?
  2. What does n establish?
  3. Which feature of the individual data can the mean conceal?
  4. Which assumption could shared cages challenge?
  1. One row represents one treatment group—not one mouse.
  2. n records the denominator used by the summary.
  3. A mean can conceal spread, overlap, clusters or outliers.
  4. Shared cages can challenge simple independence.

Result is not interpretation

Result

  • observed difference
  • uncertainty
  • sample size
  • visual pattern

Interpretation

  • relation to the biological question
  • plausible explanations
  • design limits
  • remaining uncertainty

The design limits the claim

The table records:

  • assigned treatment;
  • paired weights;
  • sex, cage and batch;
  • missing measurements.

It does not automatically establish:

  • the cause of every individual difference;
  • independence within cages;
  • why a measurement is missing;
  • generality beyond the studied mice and conditions.

Why I trust this

Evidence What it supports
rows match distinct mouse IDs observational unit represented once
missingness recorded by group effective sample is visible
ranges agree with units scale is plausible
derived values inspected transformation behaved as specified
individuals agree with summary result is not a summary artifact
uncertainty and assumptions stated limits remain visible

Trust points to evidence. Remaining doubt should also be recorded.

AI can explain one step

Give it the local context:

Here is one inspection command and its output.

1. Explain what the command checks.
2. Name one concerning result and why it matters.
3. Do not repair the data or continue the analysis.

Then test the explanation against the table or a small code change.

Generated code can reduce typing. The question, context, output and assumptions still require scientific evaluation.

The exercise follows the same reasoning sequence

Situation
  → question and prediction
  → how the data arose
  → analysis specification
  → inspect
  → compare final weight
  → derive and compare change
  → verify
  → result
  → interpretation
  → why I trust this

Begin with the question

Open:

exercises/tabular-comparison/analysis.qmd

Start with:

  1. question and prediction;
  2. how the data arose;
  3. analysis specification;
  4. inspection.

Do not begin by requesting the complete analysis from AI.

Questions

Then move into the tabular-comparison exercise.

Further reading

Adapted from Data Carpentry and Software Carpentry instructional materials, Copyright © The Carpentries, CC BY 4.0. Changes were made for BIOSCI 504; no endorsement is implied.