PRAGMATIC Security Metrics

PRAGMATIC Security Metrics: Applying Metametrics to Information Security

Free Standard Shipping

Purchasing Options

ISBN 9781439881521
Cat# K13838



SAVE 20%

eBook (VitalSource)
ISBN 9781439881538
Cat# KE13996



SAVE 30%

eBook Rentals

Other eBook Options:


  • Provides practical advice on how to specify, develop, use, and maintain a more meaningful and useful system of metrics
  • Offers guidance on using metrics to identify problem areas and drive security improvements
  • Discusses metrics that support an information security management system that complies with ISO/IEC 27001
  • Introduces capability maturity metrics that can be used to measure and drive continuous improvement in information security
  • Presents the PRAGMATIC mnemonic to help practitioners choose better metrics
  • Includes a Foreword by Mich Kabay


Other books on information security metrics discuss number theory and statistics in academic terms. Light on mathematics and heavy on utility, PRAGMATIC Security Metrics: Applying Metametrics to Information Security breaks the mold. This is the ultimate how-to-do-it guide for security metrics.

Packed with time-saving tips, the book offers easy-to-follow guidance for those struggling with security metrics. Step by step, it clearly explains how to specify, develop, use, and maintain an information security measurement system (a comprehensive suite of metrics) to help:

  • Security professionals systematically improve information security, demonstrate the value they are adding, and gain management support for the things that need to be done
  • Management address previously unsolvable problems rationally, making critical decisions such as resource allocation and prioritization of security relative to other business activities
  • Stakeholders, both within and outside the organization, be assured that information security is being competently managed

The PRAGMATIC approach lets you hone in on your problem areas and identify the few metrics that will generate real business value. The book:

  • Helps you figure out exactly what needs to be measured, how to measure it, and most importantly, why it needs to be measured
  • Scores and ranks more than 150 candidate security metrics to demonstrate the value of the PRAGMATIC method
  • Highlights security metrics that are widely used and recommended, yet turn out to be rather poor in practice
  • Describes innovative and flexible measurement approaches such as capability maturity metrics with continuous scales
  • Explains how to minimize both measurement and security risks using complementary metrics for greater assurance in critical areas such as governance and compliance

In addition to its obvious utility in the information security realm, the PRAGMATIC approach, introduced for the first time in this book, has broader application across diverse fields of management including finance, human resources, engineering, and production—in fact any area that suffers a surplus of data but a deficit of useful information.

Visit Security Metametrics. Security Metametrics supports the global community of professionals adopting the innovative techniques laid out in PRAGMATIC Security Metrics. If you, too, are struggling to make much sense of security metrics, or searching for better metrics to manage and improve information security, Security Metametrics is the place.

Table of Contents

Why Have We Written This Book?
What’s Different about This Metrics Book?
Who Are We Writing This For?
Who Are We?
     Krag Brotby
     Gary Hinson
What We’ll Be Talking About
Defining Our Terminology
What We Expect of You, the Reader

Why Measure Information Security?
To Answer Awkward Management Questions
To Improve Information Security, Systematically
For Strategic, Tactical, and Operational Reasons
For Compliance and Assurance Purposes
To Fill the Vacuum Caused by Our Inability to Measure Security
To Support the Information Security Manager
For Profit!
For Various Other Reasons

The Art and Science of Security Metrics
Metrology, the Science of Measurement
Governance and Management Metrics
Information Security Metrics
Financial Metrics (for Information Security)
(Information Security) Risk Management Metrics
     Software Quality (and Security) Metrics
     Information Security Metrics Reference Sources
     Douglas Hubbard "How to Measure Anything" (Hubbard 2010)
     Andrew Jaquith: Security Metrics (Jaquith 2007)
      NIST SP 800-55: Performance Measurement Guide for Information Security (NIST 2008)
     Debra Herrmann: Complete Guide to Security and Privacy Metrics (Herrmann 2007)
     Krag Brotby: Information Security Management Metrics (Brotby 2009a)
     Lance Hayden: IT Security Metrics (Hayden 2010)
     Caroline Wong "Security Metrics: A Beginner’s Guide" (Wong 2012)
     ISO/IEC 27004: Information Security Management–Measurement (ISO/IEC 27004 2009) 3.7.9 CIS Security Metrics (CIS 2010)
Specifying Metrics
Metrics Catalogs and a Serious Warning About SMD
Other (Information Security) Metrics Resources

Audiences for Security Metrics
Metrics Audiences Within the Organization
     Senior Management
     Middle and Junior Management
     Security Operations
     Others with Interest in Information Security
Metrics Audiences From Without the Organization

Finding Candidate Metrics
Preexisting/Current Information Security Metrics
Other Corporate Metrics
Metrics Used in Other Fields and Organizations
Information Security Metrics Reference Sources
Other Sources of Inspiration for Security Metrics
     Security Surveys
     Vendor Reports and White Papers
     Security Software
Roll-Your-Own Metrics
Metrics Supply and Demand

Metametrics and the PRAGMATIC Approach
Selecting Information Security Metrics
6.3.1 P = Predictive
6.3.2 R = Relevant
6.3.3 A = Actionable
6.3.4 G = Genuine
6.3.5 M = Meaningful
6.3.6 A = Accurate
6.3.7 T = Timely
6.3.8 I = Independent
6.3.9 C = Cost
Scoring Information Security Metrics against the PRAGMATIC Criteria
Other Uses for PRAGMATIC Metametrics
Classifying Information Security Metrics
6.6.1 Strategic/Managerial/Operational (SMO)Metrics Classification
6.6.2 Risk/Control Metrics Classification
6.6.3 Input–Process–Output (Outcome) Metrics Classification
6.6.4 Effectiveness and Efficiency Metrics Classification
6.6.5 Maturity Metrics Classification
6.6.6 Directness Metrics Classification
6.6.7 Robustness Metrics Classification
6.6.8 Readiness Metrics Classification
6.6.9 Policy/Practice Metrics Classification

150+ Example Security Metrics
Information Security Risk Management Example Metrics
Information Security Policy Example Metrics
Security Governance, Management, and Organization Example Metrics
     Information Security Financial Management Metrics
     Information Security Control-Related Metrics
     Metrics for Business Alignment and Relevance of Controls
     Control Monitoring and Testing Metrics
     Financial Information Security Metrics
Information Asset Management Example Metrics
Human Resources Security Example Metrics
Physical Security Examples
IT Security Metric Examples
Access Control Example Metrics
Software Security Example Metrics
Incident Management Example Metrics
Business Continuity Management Examples
Compliance and Assurance Metrics Examples

Designing PRAGMATIC Security Measurement System
Brief History of Information Security Metrics
Taking Systems Approach to Metrics
Information Security Measurement System Lifecycle

Advanced Information Security Metrics
High-Reliability Metrics
Indicators and Proxies
Key Indicators
     Key Goal Indicators (KGIs)
     Key Performance Indicators (KPIs)
     Key Risk Indicators (KRIs)
     Critical Success Factors (CSFs)
Targets, Hurdles, Yardsticks, Goals, Objectives, Benchmarks, and Triggers

Downsides of Metrics
Numbers Don’t Always Tell the Whole Story
Scoring Political Points through Metrics
Implausible Deniability
Metrics Gaps
On Being Good Enough
What Not to Measure

Using PRAGMATIC Metrics in Practice
Gathering Raw Data
     Automated Data Sources
     Observations, Surveys, and Interviews
     Online or In-Person Surveys
     Scoring Scales
     Audits, Reviews, and Studies
Data Analysis and Statistics
Data Presentation
     General Considerations
     Analytical Tools and Techniques
     Reporting Tools and Techniques
     Presentational Tools and Techniques
     Graphs, Figures, Diagrams, and Illustrations
     Drawing Attention to Specific Issues
Using, Reacting to, and Responding to Metrics
     Periodic versus Event-Driven Reporting

Case Study
The Context: Acme Enterprises, Inc.
Information Security Metrics for C-Suite
     Information Security Metrics for the CEO
     Information Security Metrics for the CIO
     Information Security Metrics for the CISO
     Information Security Metrics for the CFO
     Information Security Metrics for the VP of Production
     Information Security Metrics for the VP of Marketing
Information Security Metrics for Management and Operations
Information Security Metrics for External Stakeholders
Acme’s Information Security Measurement System

Take-Home Lessons from This Book
     On Pragmatism and Being PRAGMATIC
     On Giving You the Confidence and Skills to Have a Go
     On Improving the Quality of Your Management Information through Metametrics
     On Improving Metrics of All Sorts
Your Chance to Advance the Profession and the Practice of Metrics
An Action Plan to Take Away

Appendix A: PRAGMATIC Criteria
Appendix B: Business Model of Information Security (BMIS)
Appendix C: Capability Maturity Model (CMM)
     Level 1–Initial
     Level 2–Repeatable
     Level 3–Defined
     Level 4–Managed
     Level 5–Optimizing
Appendix D: Example Opinion Survey Form
     Security Awareness Survey on Malware
Appendix E: SABSA Security Attributes Table
Appendix F: Prototype Metrics Catalog
Appendix G: Effect of Weighting the PRAGMATIC Criteria
Appendix H: ISO27k Maturity Scale Metrics
Appendix I: Sample Management Survey
Appendix J: Observer Bias
Appendix K: Observer Calibration
Appendix L: Bibliography

Author Bio(s)

Editorial Reviews

Like all books on metrics, PRAGMATIC Security Metrics: Applying Metametrics to Information Security makes the statement that "you can't manage what you can't measure".  The authors claim that other books on information security metrics discuss number theory and statistics in academic terms. This title promises to be light on mathematics and heavy on utility and is meant as a how-to-do-it guide for security metrics.

As to the title, PRAGMATIC is an acronym for the basis of the method of the book, in using metrics that are predictive, relevant, actionable, genuine, meaningful, timely, independent and cost. After reading the first chapter, PRAGMATIC Security Metrics: Applying Metametrics to Information Security looks like it may live up to its promise of being able to use metrics not only to track and report performance but to identify problem areas and opportunities, and drive information security improvements. If so, this could be the metrics book a lot of information security professionals have been waiting for.
—Ben Rothke, CISSP, CISM, Information Security Manager, Wyndham Worldwide; and author of Computer Security: 20 Things Every Employee Should Know, writing on the RSA Conference Blog,