Measures of Interobserver Agreement and Reliability

Published:
Author(s):

Purchasing Options

Hardback
Not available
in your region
ISBN 9781584883210
Cat# C3219
 

Features

  • Includes a comprehensive review of the interobserver agreement approach to assessing the reliability of a rating system
  • Explains how to reduce measurement error and provide good precise definition of the category being used
  • Provides direction and standards for conducting clinical and other types of studies
  • Contains examples of applications that emphasize the practicality and use of the theory
  • Supplies SAS code for problem solving both within the book and on the CRC Web site
  • Summary

    Agreement among at least two evaluators is an issue of prime importance to statisticians, clinicians, epidemiologists, psychologists, and many other scientists. Measuring interobserver agreement is a method used to evaluate inconsistencies in findings from different evaluators who collect the same or similar information. Highlighting applications over theory, Measure of Interobserver Agreement provides a comprehensive survey of this method and includes standards and directions on how to run sound reliability and agreement studies in clinical settings and other types of investigations.

    The author clearly explains how to reduce measurement error, presents numerous practical examples of the interobserver agreement approach, and emphasizes measures of agreement among raters for categorical assessments. The models and methods are considered in two different but closely related contexts: 1) assessing agreement among several raters where the response variable is continuous and 2) where there is a prior decision by the investigators to use categorical scales to judge the subjects enrolled in the study. While the author thoroughly discusses the practical and theoretical issues of case 1, a major portion of this book is devoted to case 2. He explores issues such as two raters randomly judging a group of subjects, interrater bias and its connection to marginal homogeneity, and statistical issues in determining sample size.

    Statistical analysis of real and hypothetical datasets are presented to demonstrate the various applications of the models in repeatability and validation studies. To help with problem solving, the monograph includes SAS code, both within the book and on the CRC Web site. The author presents information with the right amount mathematical details, making this a cohesive book that reflects new research and the latest developments in the field.

    Table of Contents

    INTRODUCTION
    RELIABILITY FOR CONTINUOUS SCALE MEASUREMENTS
    Model for Reliability Studies
    Inference Procedures on the Index of Reliability for Case (1)
    Analysis of Method - Comparison Studies
    Comparing Reliability Coefficients
    MEASURES OF 2x2 ASSOCIATION AND AGREEMENT OF CROSS CLASSIFIED DATA
    Introduction
    Indices of Adjusted Agreement
    Cohen's Kappa =Chance Corrected Measure of Agreement
    Intraclass Kappa
    The 2x2 Kappa in the Context of Association
    Stratified Kappa
    Conceptual issues
    COEFFICIENTS OF AGREEMENT FOR MULTIPLE RATERS AND MULTIPLE CATEGORIES
    Introduction
    Multiple Categories and Two Raters
    Agreement for Multiple Raters and Dichotomous Classification
    Multiple Raters and Multiple Categories
    Testing the Homogeneity of Kappa Statistic from Independent Studies
    ASSESSING AGREEMENT FROM DEPENDENT
    Introduction
    Dependent Dichotomous Assessments
    Adjusting for Covariates
    Likelihood Based Approach
    Estimating Equations Approach
    Loglinear and Association Models
    Appendix I: Joint probability distribution of repeated dichotomous assessments
    Appendix II: Correlation between estimated kappas
    SAMPLE SIZE REQUIREMENTS FOR THE DESIGN OF RELIABILITY STUDY
    Introduction
    The Case of Continuous Measurements
    The Non-Normal Case
    Cost Implications
    The Case of Dichotomous Assessments
    Bibliography

    Editorial Reviews

    "… an important strength of this text is that it brings together technical results on agreement statistics from a variety of sources, and it often provides information about the precision of the estimates and interval estimates … this text provides a handy compilation of statistical approaches to interobserver agreement … biostatisticians …are likely to find the summary and results to be helpful, and they will want to add this book to their collection."
    -Patrick E. Shrout, New York University, USA, in Statistics in Medicine, Vol. 25, 2006

    "Mohamed Shoukri is well placed to write this book, as he is one of the foremost current researchers in this field. I recommend it."
    - Journal of the Royal Statistical Society

    "The well written specialized book by Shoukri illustrates the meaning and importance of the measures of interobserver agreement with fun examples. The author is known for clear presentation."
    -Journal of Statistical Computation and Simulation, Vol. 75, No. 4, April 2005

    "This book can be recommended…"
    -Biometrics

    Downloads Updates


    Resource OS Platform Updated Description Instructions
    SASCODES.pdf Cross Platform June 11, 2003 SAS Codes SAS codes in pdf format

    Related Titles