1st Edition
Statistical and Methodological Myths and Urban Legends Doctrine, Verity and Fable in Organizational and Social Sciences
This book provides an up-to-date review of commonly undertaken methodological and statistical practices that are sustained, in part, upon sound rationale and justification and, in part, upon unfounded lore. Some examples of these "methodological urban legends", as we refer to them in this book, are characterized by manuscript critiques such as: (a) "your self-report measures suffer from common method bias"; (b) "your item-to-subject ratios are too low"; (c) "you can’t generalize these findings to the real world"; or (d) "your effect sizes are too low".
Historically, there is a kernel of truth to most of these legends, but in many cases that truth has been long forgotten, ignored or embellished beyond recognition. This book examines several such legends. Each chapter is organized to address: (a) what the legend is that "we (almost) all know to be true"; (b) what the "kernel of truth" is to each legend; (c) what the myths are that have developed around this kernel of truth; and (d) what the state of the practice should be. This book meets an important need for the accumulation and integration of these methodological and statistical practices.
Part 1. Statistical Issues. Daniel A. Newman, Missing Data Techniques and Low Response Rates: The Role of Systematic Nonresponse Parameters. Michael J. Zickar, Alison A. Broadfoot, The Partial Revival of a Dead Horse? Comparing Classical Test Theory and Item Response Theory. Deborah L. Bandalos, Meggen R. Boehm, Four Common Misconceptions in Exploratory Factor Analysis. Adam W. Meade, Tara S. Behrend, Charles E. Lance, Dr. StrangeLOVE, or: How I Learned to Stop Worrying and Love Omitted Variables. James M. LeBreton, Jane Wu, Mark N. Bing, The Truth(s) on Testing for Mediation in the Social and Organizational Sciences. Jeffrey R. Edwards, Seven Deadly Myths of Testing Moderation in Organizational Research. Robert J. Vandenberg, Darrin M. Grelle, Alternative Model Specifications in Structural Equation Modeling: Facts, Fictions, and Truth. Ronald S. Landis, Bryan D. Edwards, Jose M. Cortina, On the Practice of Allowing Correlated Residuals Among Indicators in Structural Equation Models. Part 2. Methodological Issues. Lillian T. Eby, Carrie S. Hurst, Marcus M. Butts, Qualitative Research: The Red-Headed Stepchild in Organizational and Social Science Research? Scott Highhouse, Jennifer Z. Gillespie, Do Samples Really Matter That Much? Herman Aguinis, Erika E. Harden, Sample Size Rules of Thumb: Evaluating Three Common Practices. Jose M. Cortina, Ronald S. Landis, When Small Effect Sizes Tell a Big Story, and When Large Effect Sizes Don’t. David Chan, Why Ask Me? Are Self-report Data Really that Bad? Charles E. Lance, Lisa E. Baranik, Abby R. Lau, Elizabeth A. Scharlau, If It Ain’t Trait It Must Be Method: (Mis)application of the Multitrait Multimethod Design in Organizational Research. Marcus M. Butts, Thomas W. H. Ng, Chopped Liver? OK. Chopped Data? Not OK .
Biography
Charles E. Lance is a Professor of Industrial and Organizational Psychology at The University of Georgia. His work in the areas of performance measurement, assessment center validity, research methods, and structural equation modeling has appeared in such journals as Psychological Methods, Organizational Research Methods (ORM), Journal of Applied Psychology, Organizational Behavior and Human Decision Processes, Journal of Management and Multivariate Behavioral Research. His 2000 ORM article with Bob Vandenberg on measurement invariance is the most often cited article in ORM’s history and won the 2005 Research Methods Division’s Robert McDonald Advancement of Organizational Research Methodology Award. His 2006 ORM article on the origin and evolution of four statistical cutoff criteria won the Research Methods Division of the Academy of Management Best Paper of the Year Award. Also, his 2008 article "Why Assessment Centers (ACs) Do Not Work the Way They’re Supposed to" was one of the two inaugural focal articles in Industrial and Organizational Psychology: An Exchange of Perspectives on Science and Practice. Dr. Lance is also co-editor of Performance Measurement: Current Perspectives and Future Challenges (with Wink Bennett and Dave Woehr). Dr. Lance is a Fellow of the Society for Industrial and Organizational Psychology (SIOP) and the American Psychological Association, former President of the Atlanta Society for Applied Psychology, is a member of the Society for Organizational Behavior and is a licensed psychologist in the State of Georgia. He is currently Associate Editor of ORM, and on the editorial boards of Personnel Psychology, Human Performance, and Group & Organization Management.
Robert J. Vandenberg is a Professor of Management in the Terry College of Business at the University of Georgia, Athens, GA (USA). Bob's primary substantive research focuses are on organizational commitment, and high involvement work processes. His methodological research stream includes measurement invariance, latent growth modeling, and multilevel structural equation modeling. Bob's articles on these topics have appeared in the Journal of Applied Psychology, Journal of Management, Journal of Organizational Behavior, Human Resource Management, Organization Sciences, Group and Organization Management, Journal of Managerial Psychology, Organizational Behavior and Human Decision Processes, and Organizational Research Methods. Since 1999, both his substantive and methodological work has been integral to three funded grants totaling $4 million from the Centers for Disease Control, and the National Institute of Occupational Safety and Health. Bob's measurement invariance article co-authored with Chuck Lance received the 2005 Robert McDonald Award for the Best Published Article to Advance Research Methods given by the Research Methods Division of the Academy of Management. He has served on the editorial boards of the British Journal of Management, Journal of Applied Psychology, Journal of Management, Organizational Behavior and Human Decision Processes, and Organizational Research Methods. He is currently the editor of Organizational Research Methods. He is past division chair of the Research Methods Division of the Academy of Management. In addition, he is a fellow of the American Psychological Association, the Society for Industrial and Organizational Psychology, and the Southern Management Association. He is also a fellow in the Center for the Advancement of Research Methods and Analysis at Virginia Commonwealth University in which he conducts annual short courses in advanced structural equation modeling techniques.
"It is an easy-to carry size and has clear print and particularly important, clear writing. The text includes many good charts and graphics that could come in handy as examples to be used in classes, especially high-level graduate classes in statistics or research methods. … Most doctoral students should have a copy of Lance and Vandenberg before they set words to paper in their dissertation proposals … The information in the many chapters can be very important in considering results and especially in considering limitations. It can serve as a rich source of primary articles and coherent discussion of many important topics." - Malcolm James Ree, Professor of Leadership, Our Lady of the Lake University, Texas, in Personnel Psychology
"Owning a copy of this book is essential for any researcher interested in issues regarding statistics and methods. It is beneficial as well for instructors." – Stephen A. Truhon, PsycCritiques
“The chapters in this book address topics with which every researcher has grappled with such as missing data, tests of moderating and mediating effects, dependence on self report data, and the importance of the sample of respondents available. With the increased interest and use of qualitative research, this volume should also be of wide interest. The book’s chapters promise to be standard citations in research papers for many years to come. Students are likely to find this book invaluable as they begin research careers. More experienced researchers will likely relate to these chapters as well, since they will have experienced the frustration associated with many of these problems firsthand." -Neal Schmitt,Chair, Michigan State University
"Misconceptions about methods are all too common in the organizational and social sciences and they impede research progress. Until now, researchers lacked a comprehensive guidebook that separates fact from fiction regarding appropriate methodological choices. Drawing on a world class team of authors, Lance and Vandenberg close the gap between 'what we know' and 'what we need to know' about missing data, factor analysis, moderation, mediation, structural equation modeling, sampling and many other essential methodological issues." -Dave Ketchen, Lowder Eminent Scholas and Professor of Management, Auburn University
"For every management researcher, this is a GREAT book. The chapters improve one's technical skills but also explain the many, many details that one always wanted to know. I recommend this book for all researchers and graduate and doctoral students. It's as close to a 'must have' as there can be." -Tom Lee, Michael G. Foster School of Business, University of Washington
