AP* Statistics Tutorial: Experimental Design

The term experimental design refers to a plan for assigning subjects to treatment conditions. A good experimental design serves three purposes.

• Causation. It allows the experimenter to make causal inferences about the relationship between independent variables and a dependent variable.
  
  
• Control. It allows the experimenter to rule out alternative explanations due to the confounding effects of extraneous variables (i.e., variables other than the independent variables).
  
  
• Variability. It reduces variability within treatment conditions, which makes it easier to detect differences in treatment outcomes.

An Experimental Design Example

Consider the following hypothetical experiment. Acme Medicine is conducting an experiment to test a new vaccine, developed to immunize people against the common cold. To test the vaccine, Acme has 1000 volunteer subjects - 500 men and 500 women. The subjects range in age from 21 to 70.

In this lesson, we describe three experimental designs - a completely randomized design, a randomized block design, and a matched pairs design. And we show how each design might be applied by Acme Medicine to understand the effect of the vaccine, while ruling out confounding effects of other factors.

Completely Randomized Design

The completely randomized design is probably the simplest experimental design, in terms of data analysis and convenience. With this design, subjects are randomly assigned to treatments.

A completely randomized design layout for the Acme Experiment is shown in the table to the right. In this design, the experimenter randomly assigned subjects to one of two treatment conditions. They received a placebo or they received the vaccine. The same number of subjects (500) were assigned to each treatment condition (although this is not required). The dependent variable is the number of colds reported in each treatment condition. If the vaccine is effective, subjects in the "vaccine" condition should report significantly fewer colds than subjects in the "placebo" condition.

A completely randomized design relies on randomization to control for the effects of extraneous variables. The experimenter assumes that, on averge, extraneous factors will affect treatment conditions equally; so any significant differences between conditions can fairly be attributed to the independent variable.

Randomized Block Design

With a randomized block design, the experimenter divides subjects into subgroups called blocks, such that the variability within blocks is less than the variability between blocks. Then, subjects within each block are randomly assigned to treatment conditions. Because this design reduces variability and potential confounding, it produces a better estimate of treatment effects.

The table to the right shows a randomized block design for the Acme experiment. Subjects are assigned to blocks, based on gender. Then, within each block, subjects are randomly assigned to treatments. For this design, 250 men get the placebo, 250 men get the vaccine, 250 women get the placebo, and 250 women get the vaccine.

It is known that men and women are physiologically different and react differently to medication. This design ensures that each treatment condition has an equal proportion of men and women. As a result, differences between treatment conditions cannot be attributed to gender. This randomized block design removes gender as a potential source of variability and as a potential confounding variable.

In this Acme example, the randomized block design is an improvement over the completely randomized design. Both designs use randomization to implicitly guard against confounding. But only the randomized block design explicitly controls for gender.

Note 1: In some blocking designs, individual subjects may receive multiple treatments. This is called using the subject as his own control. Using the subject as his own control is desirable in some experiments (e.g., research on learning or fatigue). But it can also be a problem (e.g., medical studies where the medicine used in one treatment might interact with the medicine used in another treatment).

Note 2: Blocks perform a similar function in experimental design as strata perform in sampling. Both divide observations into subgroups. However, they are not the same. Blocking is associated with experimental design, and stratification is associated with survey sampling.

Matched Pairs Design

A matched pairs design is a special case of the randomized block design. It is used when the experiment has only two treatment conditions; and subjects can be grouped into pairs, based on some blocking variable. Then, within each pair, subjects are randomly assigned to different treatments.

The table to the right shows a matched pairs design for the Acme experiment. The 1000 subjects are grouped into 500 matched pairs. Each pair is matched on gender and age. For example, Pair 1 might be two women, both age 21. Pair 2 might be two women, both age 22, and so on.

For the Acme example, the matched pairs design is an improvement over the completely randomized design and the randomized block design. Like the other designs, the matched pairs design uses randomization to control for confounding. However, unlike the others, this design explicitly controls for two potential lurking variables - age and gender.

Test Your Understanding of This Lesson

Problem

Which of the following statements are true?

I. A completely randomized design offers no control for lurking variables.
II. A randomized block design controls for the placebo effect.
III. In a matched pairs design, subjects within each pair receive the same treatment.

(A) I only
(B) II only
(C) III only
(D) All of the above.
(E) None of the above.

Solution

The correct answer is (E). In a completely randomized design, subjects are randomly assigned to treatment conditions. Randomization provides some control for lurking variables. By itself, a randomized block design does not control for the placebo effect. To control for the placebo effect, the experimenter must include a placebo in one of the treatment levels. In a matched pairs design, subjects within each pair are assigned to different treatment levels.