1st Edition

Testing Complex and Embedded Systems

By Kim H. Pries, Jon M. Quigley Copyright 2011
    320 Pages 100 B/W Illustrations
    by CRC Press

    Many enterprises regard system-level testing as the final piece of the development effort, rather than as a tool that should be integrated throughout the development process. As a consequence, test teams often execute critical test plans just before product launch, resulting in much of the corrective work being performed in a rush and at the last minute.

    Presenting combinatorial approaches for improving test coverage, Testing Complex and Embedded Systems details techniques to help you streamline testing and identify problems before they occur—including turbocharged testing using Six Sigma and exploratory testing methods. Rather than present the continuum of testing for particular products or design attributes, the text focuses on boundary conditions. Examining systems and software testing, it explains how to use simulation and emulation to complement testing.

    • Details how to manage multiple test hardware and software deliveries
    • Examines the contradictory perspectives of testing—including ordered/ random, structured /unstructured, bench/field, and repeatable/non repeatable
    • Covers essential planning activities prior to testing, how to scope the work, and how to reach a successful conclusion
    • Explains how to determine when testing is complete

    Where you find organizations that are successful at product development, you are likely to find groups that practice disciplined, strategic, and thorough testing. Tapping into the authors’ decades of experience managing test groups in the automotive industry, this book provides the understanding to help ensure your organization joins the likes of these groups.

    Does Your Testing Look Like This?
    Last-Minute Flailing
    Fiascos Uncovered Weeks before Launch
    Huge Warranty Problems
    Customer Dissatisfaction

    Benefits of Improved Testing
    Product Problems Revealed Early
    Improved Reliability = Lower Cost
    Happy Customers
    Confidence in a Fine Product
    Cost-Effective Testing Solutions Not Waiting Until
    the Last Minute
    Overview
    Goals of Testing
    Types of Testing
    Levels of Testing

    Basic Principles
    Looking at the Evidence

    The Question
    Not Phrases
    Instead

    Contradictory Perspectives of Testing
    Organic/Inorganic
    Quantitative/Qualitative
    Objective/Subjective
    Deterministic/Probabilistic
    Variable/Attribute
    Continuous/Discrete
    Wide-Ranging/Focused
    Many/Few
    Structure/Unstructured
    Ordered/Random
    Nominal/Overstress
    Single Environment/Multiple Environments
    Compliance/Characterization
    High-Level/Detailed
    Growing/Imposing
    Bench/Field
    Abstraction/Verisimilitude
    Reproducible/Nonreproducible
    Repeatable/Nonrepeatable
    Linear/Nonlinear
    Fine/Coarse
    Combinatorial/Exhaustive/Stochastic
    Focused/Fuzzy
    Instant/Enduring
    Inside System/Outside System
    Ambiguity/Clarity
    Sensitive/Insensitive
    Long-Range/Short-Range
    Costly/Cheap
    Flexible/Inflexible
    Parameter/Tolerance
    Standard/Failure
    "Good"/"Bad" Testing
    Parallel/Sequential
    Fractal/Smooth/Ordered

    The Use of Noise
    Realistic
    Can Use Taguchi Approach
    Sometimes Difficult to Simulate
    Where to Apply Noise?
    Exogenous Shocks
    Self-Generated Noise

    How to Perform "Bad’’ Tests
    Do Not
    Do
    Documenting the Testing
    Components of a Test Plan
    Components of a Test Report
    The DVP&R Format
    Failures

    Test Administration
    Test Management
    Test Scheduling
    Test Human Resources
    Test Device Resources
    Test Quality
    Test Costing
    Test Risk
    Calibration Issues

    Advanced Concepts
    Test Impacts
    Construct Validity
    Types of Bias
    Reliability and Confidence
    Life Testing
    Testing Team Early Involvement

    Software Test Documentation
    IEEE
    Defect Life Cycle

    Configuration Management
    Configuration Management Areas
    Planning
    Elements of Configuration Management
    Importance of Configuration Management
    Links to Testing
    Risks

    Software Testing
    Overview
    Software Testing—The Problem
    Test Metrics
    Software Boundary Testing
    Static Code Analysis
    Dynamic Code Analysis

    Systems Testing
    End-to-End Testing
    Big-Bang Testing
    Top-Down Testing
    Bottom-Up Testing
    Managing Multiple Test Hardware
    and Software Deliveries
    Test Configuration Management
    System Variations and Test Demands
    Functional Testing
    System Response to Multiple System Failures
    Ranges of System Performance

    Simulation and Emulation
    Simulation
    Simulation Levels
    Simulation Activities
    Objectives of Simulation
    Simulation as Verification
    Simulation as Test Preparation
    Conflict between Simulation and Test Results

    Span of Tests
    Software
    Unit Test
    Component Test
    Subsystem-Level Integration Testing
    System
    Production Test
    Static Analysis
    Structural Analysis
    Simulation
    Prototyping and Simulation
    Reliability and Confidence
    Limitations of Reliability and Confidence
    Concept of "Life" and the Product
    Establishing Product Life Exposure

    Exit Criteria
    When Is Enough, Enough?
    Compared to Plan?
    Testing Ethics
    Final Words

    Bibliography
    Index

    Biography

    Kim H. Pries, Jon M. Quigley