Randomization Tests, Fourth Edition
Purchasing Options
Features
Summary
The number of innovative applications of randomization tests in various fields and recent developments in experimental design, significance testing, computing facilities, and randomization test algorithms have necessitated a new edition of Randomization Tests.
Updated, reorganized, and revised, the text emphasizes the irrelevance and implausibility of the random sampling assumption for the typical experiment in three completely rewritten chapters. It also discusses factorial designs and interactions and combines repeated-measures and randomized block designs in one chapter. The authors focus more attention on the practicality of N-of-1 randomization tests and the availability of user-friendly software to perform them. In addition, they provide an overview of free and commercial computer programs for all of the tests presented in the book.
Building on the previous editions that have served as standard textbooks for more than twenty-five years, Randomization Tests, Fourth Edition includes a CD-ROM of up-to-date randomization test programs that facilitate application of the tests to experimental data. This CD-ROM enables students to work out problems that have been added to the chapters and helps professors teach the basics of randomization tests and devise tasks for assignments and examinations.
Table of Contents
Statistical Tests That Do Not Require Random Sampling
Randomization Tests
Numerical Examples
Randomization Tests and Nonrandom Samples
The Prevalence of Nonrandom Samples in Experiments
The Irrelevance of Random Samples for the Typical Experiment
Generalizing from Nonrandom Samples
Intelligibility
Respect for the Validity of Randomization Tests
Versatility
Practicality
Precursors of Randomization Tests
Other Applications of Permutation Tests
Questions and Exercises
Notes
References
Randomized Experiments
Unique Benefits of Experiments
Experimentation without Manipulation of Treatments
Matching: A Precursor of Randomization
Randomization of Experimental Units
Experimental Units
Groups as Experimental Units
Control over Confounding Variables
Between-Subject and Within-Subject Randomization
Conventional Randomization Procedures
Randomization Procedures for Randomization Tests
Further Reading
Questions and Exercises
Calculating P-Values
Introduction
Systematic Reference Sets
Criteria of Validity for Randomization Tests
Randomization Test Null Hypotheses
Permuting Data for Experiments with Equal Sample Sizes
Monte Carlo Randomization Tests
Equivalent Test Statistics
Randomization Test Computer Programs
Writing Programs for Randomization Tests
How to Test Systematic Data Permutation Programs
How to Test Random Data Permutation Programs
Nonexperimental Applications of the Programs
Questions and Exercises
References
Between-Subjects Designs
Introduction
One-Way ANOVA with Systematic Reference Sets
A Simpler Test Statistic Equivalent to F
One-Way ANOVA with Equal Sample Sizes
One-Way ANOVA with Random Reference Sets
Analysis of Covariance
One-Tailed t Tests and Predicted Direction of Difference
Simpler Equivalent Test Statistics to t
Tests of One-Tailed Null Hypotheses for t Tests
Unequal-N One-Tailed Null Hypotheses
Fast Alternatives to Systematic Data Permutation for Independent t Tests
Independent t Test with Random Reference Sets
Independent t Test and Planned Comparisons
Independent t Test and Multiple Comparisons
Loss of Experimental Subjects
Ranked Data
Dichotomous Data
Outliers
Questions and Exercises
References
Factorial Designs
Advantages of Randomization Tests for Factorial Designs
Factorial Designs for Completely Randomized Experiments
Proportional Cell Frequencies
Program for Tests of Main Effects
Completely Randomized Two-Factor Experiments
Completely Randomized Three-Factor Experiments
Interactions in Completely Randomized Experiments
Randomization Test Null Hypotheses and Test Statistics
Designs with Factor-Specific Dependent Variables
Dichotomous and Ranked Data
Fractional Factorial and Response Surface Designs
Questions and Exercises
References
Repeated-Measures and Randomized Block Designs
Carry-Over Effects in Repeated-Measures Designs
The Power of Repeated-Measures Tests
Systematic Listing of Data Permutations
A Nonredundant Listing Procedure
Σt2 as an Equivalent Test Statistic to F
Repeated-Measures ANOVA with Systematic Data Permutation
Repeated-Measures ANOVA with Random Data Permutation
Correlated t Test with Systematic Data Permutation
Fast Alternatives to Systematic Data Permutation for Correlated t Tests
Correlated t Test with Random Data Permutation
Correlated t Test and Planned Comparisons
Correlated t Test and Multiple Comparisons
Rank Tests
Dichotomous Data
Counterbalanced Designs
Outliers
Factorial Experiments with Repeated Measures
Interactions in Repeated-Measures Experiments
Randomized Block Designs
Randomized Complete Blocks
Incomplete Blocks
Treatments-by-Subjects Designs
Disproportional Cell Frequencies
Test Statistic for Disproportional Cell Frequencies
Data Adjustment for Disproportional Cell Frequency Designs
Restricted-Alternatives Random Assignment
Combining P-Values
Additive Method of Combining P-Values
Combining One-Tailed and Two-Tailed P-Values
Questions and Exercises
References
Multivariate Designs
Importance of Parametric Assumptions Underlying MANOVA
Randomization Tests for Conventional MANOVA
Custom-Made Multivariate Randomization Tests
Effect of Units of Measurement
Multivariate Tests Based on Composite z Scores
Combining t or F Values over Dependent Variables
A Geometrical Model
Tests of Differences in Composition
Evaluation of Three MANOVA Tests
Multivariate Factorial Designs
Combining Univariate and Multivariate P-Values
Questions and Exercises
References
Correlation
Determining P-Values by Data Permutation
Computer Program for Systematic Data Permutation
Correlation with Random Data Permutation
Multivariate Correlation
Point-Biserial Correlation
Correlation between Dichotomous Variables
Spearman’s Rank Correlation Procedure
Kendall’s Rank Correlation Procedure
Questions and Exercises
References
Trend Tests
Goodness-of-Fit Trend Test
Power of the Goodness-of-Fit Trend Test
Test Statistic for the Goodness-of-Fit Trend Test
Computation of Trend Means
Computer Program for Goodness-of-Fit Trend Test
Modification of the Goodness-of-Fit Trend Test Statistic
Correlation Trend Test
Correlation Trend Test for Factorial Designs
Disproportional Cell Frequencies
Data Adjustment for Disproportional Cell Frequency Designs
Combining of P-Values for Trend Tests for Factorial Experiments
Repeated-Measures Trend Tests
Differences in Trends
Correlation Trend Test and Simple Correlation
Ordered Levels of Treatments
Ranked and Dichotomous Data
Questions and Exercises
References
Matching and Proximity Experiments
Randomization Tests for Matching
Randomization Tests of Proximity
Matching and Proximity Tests Based on Random Selection of Treatment Levels
Questions and Exercises
References
N-of-1 Designs
The Importance of N-of-1 Designs
Fisher’s Lady-Tasting-Tea Experiment
The Concept of Choosing as a Random Process
Limitations of the Random Sampling Model for N-of-1 Experiments
Random Assignment Model
Carry-Over Effects
The N-of-1 Randomization Test: An Early Model
Factorial Experiments
Randomized Blocks
Correlation
Operant Research and Treatment Blocks
ABAB Design
Random Assignment of Treatment Blocks to Treatments
Randomization Tests for Treatment Intervention
Effects of Trends
Randomization Tests for Intervention and Withdrawal
Multiple Schedule Experiments
Power of N-of-1 Randomization Tests
Replicated N-of-1Experiments
N-of-1 Clinical Trial Facilities
Single-Cell and Other Single-Unit Neuroscience Experiments
Books on N-of-1 Design and Analysis
Software for N-of-1 Randomization Tests
Questions and Exercises
References
Tests of Quantitative Laws
Generic and Specific Null Hypotheses
The Referent of a Law or Model
Test of Incremental Effects
Weber’s Law
Other Psychophysical Laws
Foraging Behavior of Hawks
Complications
Questions and Exercises
References
Tests of Direction and Magnitude of Effect
Tests of One-Tailed Null Hypotheses for Correlated t Tests
Other Tests of One-Tailed Null Hypotheses Using ta or (Ā -B) as Test Statistics
Tests of One-Tailed Null Hypotheses about Differences in Variability
Tests of One-Tailed Null Hypotheses for Correlation
Testing Null Hypotheses about Magnitude of Effect
Testing Null Hypotheses about Specific Additive Effects
Questions and Exercises
References
Fundamentals of Validity
Randomization Tests as Distribution-Free Tests
Differences between Randomization Test Theory and Permutation Test Theory
Parametric Tests as Approximations to Randomization Tests
Randomization Test Theory
Systematically Closed Reference Sets Permutation Groups
Data-Permuting and Randomization-Referral Procedures
Invariance of Measurements under the Null Hypothesis
General and Restricted Null Hypotheses
Reference Sets for General Null Hypotheses
Reference Subsets for General Null Hypotheses
Reference Subsets for Restricted Null Hypotheses
Reference Subsets for Planned and Multiple Comparisons
Reference Subsets for Factorial Designs
Open Reference Sets: Treatment Intervention and Withdrawal
Closed Reference Sets: Dropouts
Open Reference Sets: Permuting Residuals
Sampling a List of Randomizations
Random Data Permutation: Hypothesis Testing vs. Estimation
Stochastic Closure When Assignments Are Equally Probable
Systematic Expansion of a Random Reference Set
Random Ordering of Measurements within Treatments
Fixed, Mixed, and Random Models
Deriving One-Tailed P-Values from Two-Tailed P-Values with Unequal N
Test Statistics and Adaptive Tests
Stochastic Closure When Assignments Are Not Equally Probable
Questions and Exercises
References
General Guidelines and Software Availability
Randomization: Multistage Model
Permuting Data: Data-Exchanging Model
Maximizing Power
Randomization Test Computer Programs on the CD
Other Computer Programs
References
Editorial Reviews
“…Overall, this is an interesting and well-written book that provides a useful discussion of the theory, design, and application of randomization tests, illustrated with appropriate examples using experimental data. The end-of-chapter questions and exercises make it useful also as a textbook for college students. It should be of interest for every experimenter who is interested in randomization or permutation tests or is skeptical about the reliability of the assumptions of parametric tests.”
—Andreas Karlsson (Uppsala University), Journal of the Royal Statistical Society
"I believe Randomization Tests will be useful for those researchers who often work with data that are not compatible with the usual parametric assumptions (e.g., normality). In these cases, the permutation and randomization tests discussed in the text are more appropriate and more reliable."
– Michael Sherman, Texas A&M University, in Journal of the American Statistical Association, June 2009, Vol. 104, No. 486
