How to Choose the Best Sampling Method
What is the Best Sampling Method?
The best sampling method
is the sampling method that most effectively meets the particular goals of the
study in question. The effectiveness of a sampling method
depends on many factors. Because these factors interact in complex ways,
the "best" sampling method is seldom obvious. Good researchers use the
following strategy to identify the best sampling method.
Choose the method that does the best job of achieving the goals.
The next section presents an example that illustrates this strategy.
Sample Planning Wizard
The computations involved in comparing different sampling methods can be complex
and time-consuming. Stat Trek's Sample Planning Wizard can help. The Wizard computes
survey precision, sample size requirements, costs, etc., allowing you to compare
alternative sampling methods quickly, easily, and accurately. The Wizard creates
a summary report that lists key findings and documents analytical techniques.
Whenever you want to find the most precise, cost-effective sampling method, consider
using the Sample Planning Wizard. The Sample Planning Wizard is a premium tool
available only to registered users.
How to Choose the Best Sampling Method
In this section, we illustrate how to choose the best sampling method by working
through a sample problem. Here is the problem:
At the end of every school year, the
state administers a reading test to a sample of third graders. The school
system has 20,000 third graders, half boys and half girls. There are 1000
third-grade classes, each with 20 students.
The maximum budget for this research is
$3600. The only expense is the cost to proctor each test session. This amounts
to $100 per session.
The purpose of the study is to estimate
the reading proficiency of third graders, based on sample data. School
administrators want to maximize the precision of this estimate without
exceeding the $3600 budget. What sampling method should they use?
As noted earlier, finding the "best" sampling method is a four-step process. We
work through each step below.
List goals. This study has two main goals: (1) maximize precision and (2) stay
Identify potential sampling methods. This tutorial has covered three basic
sampling methods - simple random sampling, stratified sampling, and cluster
sampling. In addition, we've described some variations on the basic methods
(e.g., proportionate vs. disproportionate stratification,
one-stage vs. two-stage
cluster sampling, sampling with replacement versus
sampling without replacement).
Because one of the main goals is to maximize precision, we can eliminate some of
these alternatives. Sampling without replacement always provides equal or
better precision than sampling with replacement, so we will focus only on
sampling without replacement. Also, as long as the same clusters are sampled,
one-stage cluster sampling always provides equal or better precision than
two-stage cluster sampling, so we will focus only on one-stage cluster
sampling. (Note: For cluster sampling in this example, the cost is the same
whether we sample all students or only some students from a particular cluster;
so in this example, two-stage sampling offers no cost advantage over one-stage
This leaves us with four potential sampling methods - simple random sampling,
proportionate stratified sampling, disproportionate stratified sampling, and
one-stage cluster sampling. Each of these uses sampling without replacement.
Because of the need to maximize precision, we will use
Neyman allocation with our disproportionate stratified sample.
Test methods. A key part of the analysis is to test the ability of each
potential sampling method to satisfy the research goals. Specifically, we will
want to know the level of precision and the cost associated with each potential
method. For our test, we use the
standard error to measure precision. The smaller the standard error,
the greater the precision.
To avoid getting bogged down in the computational details of the analysis, we
will use results from sample problems that have appeared in previous lessons.
Those results are summarized in the table below. (To review the analyses that
produced this output, click the "See analysis" links in the last column of the
|Simple random sampling
|Proportionate stratified sampling
|Disproportionate stratified sampling
|One-stage cluster sampling
Because the budget is $3600 and because each test session costs $100 (for the
proctor), there can be at most 36 test sessions. For the first three methods,
students in the sample might come from 36 different schools, which would mean
that each test
session could have only one student. Thus, for simple random sampling and
stratified sampling, the sample size might be only 36 students. For cluster
sampling, in contrast, each of the 36 test sessions will have a full class of
20 students; so the sample size will be 36 * 20 = 720 students.
Choose best method. In this example, the cost of each sampling method is
identical, so none of the methods has an advantage on cost. However, the
methods do differ with respect to precision (as measured by standard error).
Cluster sampling provides the most precision (i.e., the smallest standard
error); so cluster sampling is the best method.
Although cluster sampling was "best" in this example, it may not be the best
solution in other situations. Other sampling methods may be best in other
situations. Use the four-step process described above to determine which method
is best in any situation.