1st Edition

Richly Parameterized Linear Models Additive, Time Series, and Spatial Models Using Random Effects

By James S. Hodges Copyright 2014
    470 Pages 77 B/W Illustrations
    by Chapman & Hall

    470 Pages 77 B/W Illustrations
    by Chapman & Hall

    A First Step toward a Unified Theory of Richly Parameterized Linear Models

    Using mixed linear models to analyze data often leads to results that are mysterious, inconvenient, or wrong. Further compounding the problem, statisticians lack a cohesive resource to acquire a systematic, theory-based understanding of models with random effects.

    Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects takes a first step in developing a full theory of richly parameterized models, which would allow statisticians to better understand their analysis results. The author examines what is known and unknown about mixed linear models and identifies research opportunities.

    The first two parts of the book cover an existing syntax for unifying models with random effects. The text explains how richly parameterized models can be expressed as mixed linear models and analyzed using conventional and Bayesian methods.

    In the last two parts, the author discusses oddities that can arise when analyzing data using these models. He presents ways to detect problems and, when possible, shows how to mitigate or avoid them. The book adapts ideas from linear model theory and then goes beyond that theory by examining the information in the data about the mixed linear model’s covariance matrices.

    Each chapter ends with two sets of exercises. Conventional problems encourage readers to practice with the algebraic methods and open questions motivate readers to research further. Supporting materials, including datasets for most of the examples analyzed, are available on the author’s website.

    Mixed Linear Models: Syntax, Theory, and Methods
    An Opinionated Survey of Methods for Mixed Linear Models
    Mixed linear models in the standard formulation
    Conventional analysis of the mixed linear model
    Bayesian analysis of the mixed linear model
    Conventional and Bayesian approaches compared
    A few words about computing

    Two More Tools: Alternative Formulation, Measures of Complexity
    Alternative formulation: The "constraint-case" formulation
    Measuring the complexity of a mixed linear model fit

    Richly Parameterized Models as Mixed Linear Models
    Penalized Splines as Mixed Linear Models

    Penalized splines: Basis, knots, and penalty
    More on basis, knots, and penalty
    Mixed linear model representation

    Additive Models and Models with Interactions
    Additive models as mixed linear models
    Models with interactions

    Spatial Models as Mixed Linear Models
    Geostatistical models
    Models for areal data
    Two-dimensional penalized splines

    Time-Series Models as Mixed Linear Models
    Example: Linear growth model
    Dynamic linear models in some generality
    Example of a multi-component DLM

    Two Other Syntaxes for Richly Parameterized Models
    Schematic comparison of the syntaxes
    Gaussian Markov random fields
    Likelihood inference for models with unobservables

    From Linear Models to Richly Parameterized Models: Mean Structure
    Adapting Diagnostics from Linear Models
    Preliminaries
    Added variable plots
    Transforming variables
    Case influence
    Residuals

    Puzzles from Analyzing Real Datasets
    Four puzzles
    Overview of the next three chapters

    A Random Effect Competing with a Fixed Effect
    Slovenia data: Spatial confounding
    Kids and crowns: Informative cluster size

    Differential Shrinkage
    The simplified model and an overview of the results
    Details of derivations
    Conclusion: What might cause differential shrinkage?

    Competition between Random Effects
    Collinearity between random effects in three simpler models
    Testing hypotheses on the optical-imaging data and DLM models
    Discussion

    Random Effects Old and New
    Old-style random effects
    New-style random effects
    Practical consequences
    Conclusion

    Beyond Linear Models: Variance Structure
    Mysterious, Inconvenient, or Wrong Results from Real Datasets
    Periodontal data and the ICAR model
    Periodontal data and the ICAR with two classes of neighbor pairs
    Two very different smooths of the same data
    Misleading zero variance estimates
    Multiple maxima in posteriors and restricted likelihoods
    Overview of the remaining chapters

    Re-Expressing the Restricted Likelihood: Two-Variance Models
    The re-expression
    Examples
    A tentative collection of tools

    Exploring the Restricted Likelihood for Two-Variance Models
    Which vj tell us about which variance?
    Two mysteries explained

    Extending the Re-Expressed Restricted Likelihood
    Restricted likelihoods that can and can’t be re-expressed
    Expedients for restricted likelihoods that can’t be re-expressed

    Zero Variance Estimates
    Some observations about zero variance estimates
    Some thoughts about tools

    Multiple Maxima in the Restricted Likelihood and Posterior
    Restricted likelihoods with multiple local maxima
    Posteriors with multiple modes

    Biography

    James S. Hodges

    "Richly Parameterized Linear Models is a strong addition to the literature on mixed models, offering a much more unified treatment and broader scope than many existing texts. … The book also provides an excellent treatment of diagnostics for mixed models. … The literature on mixed models has expanded in recent decades, often in disparate ways. Richly Parameterized Linear Models provides a step toward unifying this growing area of research and serves as an excellent resource for applied researchers with experience and interest in mixed models."
    Journal of the American Statistical Association, Vol. 110, 2015

    "This is a thought-provoking book that stimulates discussion and further research. … It is a refreshing idea to concentrate on poorly understood ‘mysteries’ in the class of linear mixed models. … If you have ever encountered similar issues in linear mixed models, you will appreciate this book. Furthermore, if you are looking for new scientific challenges, then you will find a large resource of still open questions and useful references. Graduate students will develop a deeper understanding of the linear mixed model and critical thinking in the application of this model class. … We recommend this book, as it will stimulate further research, leading to deeper insight into richly parametrized linear models."
    Biometrical Journal, 2015

    "Hodges’ book is a really recommendable reference for mixed models users. Use of random effects models has exponentially grown during the last few decades mainly due to the availability of software making the fit of these models possible. Such software has made mixed models available to a wide community of users, thus complex analyses with complex covariance structures have filled both applied and theoretical journals. Nevertheless, as masterfully described within this book, the hypotheses underneath these models, and the corresponding covariance structures, are not harmless at all but, on the contrary, they may have a large impact on their fit. Hodges’ monograph explores and highlights the repercussion of many aspects involved in the definition of most mixed models that are usually unknown or, even worse, ignored. Hodges makes us aware of these issues that should be kept very in mind by any mixed models user and illustrates them with lots of real case studies.
    This is an extremely clarifying tool for those who usually work with GLMMs in general. Mathematical details underlying GLMMs are usually overwhelming … Hodges makes a journey to the grounds of linear mixed models, where mathematical detail is more accessible, providing a deep insight also of use in GLMMs. The approach described in this book yields some intuition about what can be happening underneath the complex GLMMs that are usually used in practice."
    —Miguel Martinez Beneito, Foundation for the Promotion of Health and Biomedical Research in the Valencian Region

    "I recommend this text to any student/researcher who is interested in mixed models. The book is written in an enthralling and engaging style and is overflowing with interesting observations, has a unique spin, and is very thought provoking. Over the past 20 years there has been a tendency toward the fitting of more and more complex models, with the potential negative implications of this endeavor (which I will loosely term ‘overfitting’) being lost amid the enthusiasm for bigger and allegedly ‘more realistic’ models. Those with such tendencies would definitely benefit from studying this book in order to gain insight into the unexpected consequences that some mixed model choices can have. All of the model types appearing in the title are covered in great detail, including coverage of diagnostics for mixed models, a sorely under-examined topic. Real data analyses are prominent in the discussions, with strange behavior of fitting being highlighted; for example, there is a section entitled, ‘Mysterious, inconvenient, or wrong results from real datasets.’ The author is not afraid to state an opinion, which is very refreshing; see, for example, the critique/gentle destruction of h-likelihood in Section 7.3. Open questions are also advertised and PhD students looking for research problems could do a lot worse than read this book. … an excellent, inspiring, and entertaining statistics book …"
    —Jon Wakefield, Professor of Statistics, University of Washington

    "This new book by James S. Hodges is different. Whereas others describe a polished version of the theme of the book, Hodges takes another approach. In a very personal writing style, Hodges explores what we do and do not understand about mixed linear models. Why does his approach work so well? It makes you think!!! By also discussing mysterious, inconvenient, or plainly wrong results, we simply gain more insight and understanding. This works for me; I have never gained so much (hard to get) insight in so short a time from any other book I have read. I highly recommend it!"
    —Håvard Rue, Norwegian University of Science and Technology

    "This book is a masterpiece, destined to become a classic. Linear mixed models are widely used by statisticians and analysts in many fields, often under other names: hierarchical, longitudinal, dynamic, random effects, multi-level, and others. Statistical packages routinely fit these models. Sometimes the packages return sensible results; sometimes not. Sometimes the user can tell whether the results are sensible; sometimes not. Oddities include ‘zero variance estimates, multiple maxima, counterintuitive outlier effects, odd fits (e.g., a wiggly smooth with one smoother but not with another apparently similar smoother), big changes in fit from apparently modest changes in the model or data, and unpredictable convergence of numerical routines,’ to quote from the book. There is not presently a unified theory, like that for linear regression, to explain how, why, and when our numerical routines give results that should be questioned, or at least examined further. Even so, this book does the best job I have seen of explaining what can go wrong and what the state of the art is.
    The subject is important; the writing is excellent; and the examples are compelling. I am excited by the prospect of teaching a course from this book. Its clarity of thought and presentation are exemplary. I recommend it for anyone who fits complicated models."
    —Michael Lavine, University of Massachusetts Amherst