1st Edition

Adaptive Filtering Fundamentals of Least Mean Squares with MATLAB®

By Alexander D. Poularikas Copyright 2015
    364 Pages 129 B/W Illustrations
    by CRC Press

    363 Pages
    by CRC Press

    Adaptive filters are used in many diverse applications, appearing in everything from military instruments to cellphones and home appliances. Adaptive Filtering: Fundamentals of Least Mean Squares with MATLAB® covers the core concepts of this important field, focusing on a vital part of the statistical signal processing area—the least mean square (LMS) adaptive filter.

    This largely self-contained text:

    • Discusses random variables, stochastic processes, vectors, matrices, determinants, discrete random signals, and probability distributions
    • Explains how to find the eigenvalues and eigenvectors of a matrix and the properties of the error surfaces
    • Explores the Wiener filter and its practical uses, details the steepest descent method, and develops the Newton’s algorithm
    • Addresses the basics of the LMS adaptive filter algorithm, considers LMS adaptive filter variants, and provides numerous examples
    • Delivers a concise introduction to MATLAB®, supplying problems, computer experiments, and more than 110 functions and script files

    Featuring robust appendices complete with mathematical tables and formulas, Adaptive Filtering: Fundamentals of Least Mean Squares with MATLAB® clearly describes the key principles of adaptive filtering and effectively demonstrates how to apply them to solve real-world problems.

    Preface

    Author

    Abbreviations

    MATLAB® Functions

    Vectors

    Introduction

    Multiplication by a Constant and Addition and Subtraction

    Unit Coordinate Vectors

    Inner Product

    Distance between Two Vectors

    Mean Value of a Vector

    Direction Cosines

    The Projection of a Vector

    Linear Transformations

    Linear Independence, Vector Spaces, and Basis Vectors

    Orthogonal Basis Vectors

    Problems

    Hints–Suggestions–Solutions

    Matrices

    Introduction

    General Types of Matrices

    Diagonal, Identity, and Scalar Matrices

    Upper and Lower Triangular Matrices

    Symmetric and Exchange Matrices

    Toeplitz Matrix

    Hankel and Hermitian

    Matrix Operations

    Determinant of a Matrix

    Definition and Expansion of a Matrix

    Trace of a Matrix

    Inverse of a Matrix

    Linear Equations

    Square Matrices (n × n)

    Rectangular Matrices (n < m)

    Rectangular Matrices (m < n)

    Quadratic and Hermitian Forms

    Eigenvalues and Eigenvectors

    Eigenvectors

    Properties of Eigenvalues and Eigenvectors

    Problems

    Hints–Suggestions–Solutions

    Processing of Discrete Deterministic Signals: Discrete Systems

    Discrete-Time Signals

    Time-Domain Representation of Basic Continuous and Discrete Signals

    Transform-Domain Representation of Discrete Signals

    Discrete-Time Fourier Transform

    The Discrete FT

    Properties of DFT

    The z-Transform

    Discrete-Time Systems

    Linearity and Shift Invariant

    Causality

    Stability

    Transform-Domain Representation

    Problems

    Hints–Suggestions–Solutions

    Discrete-Time Random Processes

    Discrete Random Signals, Probability Distributions, and Averages of Random Variables

    Stationary and Ergodic Processes

    Averages of RV

    Stationary Processes

    Autocorrelation Matrix

    Purely Random Process (White Noise)

    Random Walk

    Special Random Signals and pdf’s

    White Noise

    Gaussian Distribution (Normal Distribution)

    Exponential Distribution

    Lognormal Distribution

    Chi-Square Distribution

    Wiener–Khinchin Relations

    Filtering Random Processes

    Special Types of Random Processes

    Autoregressive Process

    Nonparametric Spectra Estimation

    Periodogram

    Correlogram

    Computation of Periodogram and Correlogram Using FFT

    General Remarks on the Periodogram

    Proposed Book Modified Method for Better Frequency Resolution

    Bartlett Periodogram

    The Welch Method

    Proposed Modified Welch Methods

    Problems

    Hints–Solutions–Suggestions

    The Wiener Filter

    Introduction

    The LS Technique

    Linear LS

    LS Formulation

    Statistical Properties of LSEs

    The LS Approach

    Orthogonality Principle

    Corollary

    Projection Operator

    LS Finite Impulse Response Filter

    The Mean-Square Error

    The FIR Wiener Filter

    The Wiener Solution

    Orthogonality Condition

    Normalized Performance Equation

    Canonical Form of the Error-Performance Surface

    Wiener Filtering Examples

    Minimum MSE

    Optimum Filter (wo)

    Linear Prediction

    Problems

    Additional Problems

    Hints–Solutions–Suggestions

    Additional Problems

    Eigenvalues of Rx: Properties of the Error Surface

    The Eigenvalues of the Correlation Matrix

    Karhunen–Loeve Transformation

    Geometrical Properties of the Error Surface

    Problems

    Hints–Solutions–Suggestions

    Newton’s and Steepest Descent Methods

    One-Dimensional Gradient Search Method

    Gradient Search Algorithm

    Newton’s Method in Gradient Search

    Steepest Descent Algorithm

    Steepest Descent Algorithm Applied to Wiener Filter

    Stability (Convergence) of the Algorithm

    Transient Behavior of MSE

    Learning Curve

    Newton’s Method

    Solution of the Vector Difference Equation

    Problems

    Edition Problems

    Hints–Solutions–Suggestions

    Additional Problems

    The Least Mean-Square Algorithm

    Introduction

    The LMS Algorithm

    Examples Using the LMS Algorithm

    Performance Analysis of the LMS Algorithm

    Learning Curve

    The Coefficient-Error or Weighted-Error Correlation Matrix

    Excess MSE and Misadjustment

    Stability

    The LMS and Steepest Descent Methods

    Complex Representation of the LMS Algorithm

    Problems

    Hints–Solutions–Suggestions

    Variants of Least Mean-Square Algorithm

    The Normalized Least Mean-Square Algorithm

    Power NLMS

    Self-Correcting LMS Filter

    The Sign-Error LMS Algorithm

    The NLMS Sign-Error Algorithm

    The Sign-Regressor LMS Algorithm

    Self-Correcting Sign-Regressor LMS Algorithm

    The Normalized Sign-Regressor LMS Algorithm

    The Sign–Sign LMS Algorithm

    The Normalized Sign–Sign LMS Algorithm

    Variable Step-Size LMS

    The Leaky LMS Algorithm

    The Linearly Constrained LMS Algorithm

    The Least Mean Fourth Algorithm

    The Least Mean Mixed Norm LMS Algorithm

    Short-Length Signal of the LMS Algorithm

    The Transform Domain LMS Algorithm

    Convergence

    The Error Normalized Step-Size LMS Algorithm

    The Robust Variable Step-Size LMS Algorithm

    The Modified LMS Algorithm

    Momentum LMS

    The Block LMS Algorithm

    The Complex LMS Algorithm

    The Affine LMS Algorithm

    The Complex Affine LMS Algorithm

    Problems

    Hints–Solutions–Suggestions

    Appendix 1: Suggestions and Explanations for MATLAB Use

    Suggestions and Explanations for MATLAB Use

    Creating a Directory

    Help

    Save and Load

    MATLAB as Calculator

    Variable Names

    Complex Numbers

    Array Indexing

    Extracting and Inserting Numbers in Arrays

    Vectorization

    Windowing

    Matrices

    Producing a Periodic Function

    Script Files

    Functions

    Complex Expressions

    Axes

    2D Graphics

    3D Plots

    General Purpose Commands

    Managing Commands and Functions

    Managing Variables and Workplace

    Operators and Special Characters

    Control Flow

    Elementary Matrices and Matrix Manipulation

    Elementary Matrices and Arrays

    Matrix Manipulation

    Elementary Mathematical Functions

    Elementary Functions

    Numerical Linear Algebra

    Matrix Analysis

    Data Analysis

    Basic Operations

    Filtering and Convolution

    Fourier Transforms

    2D Plotting

    2D Plots

    Appendix 2: Matrix Analysis

    Definitions

    Special Matrices

    Matrix Operation and Formulas

    Eigendecomposition of Matrices

    Matrix Expectations

    Differentiation of a Scalar Function with respect to a Vector

    Appendix 3: Mathematical Formulas

    Trigonometric Identities

    Orthogonality

    Summation of Trigonometric Forms

    Summation Formulas

    Finite Summation Formulas

    Infinite Summation Formulas

    Series Expansions

    Logarithms

    Some Definite Integrals

    Appendix 4: Lagrange Multiplier Method

    Bibliography

    Index

    Biography

    Alexander D. Poularikas is chairman of the electrical and computer engineering department at the University of Alabama in Huntsville, USA. He previously held positions at University of Rhode Island, Kingston, USA and the University of Denver, Colorado, USA. He has published, coauthored, and edited 14 books and served as an editor-in-chief of numerous book series. A Fulbright scholar, lifelong senior member of the IEEE, and member of Tau Beta Pi, Sigma Nu, and Sigma Pi, he received the IEEE Outstanding Educators Award, Huntsville Section in 1990 and 1996. Dr. Poularikas holds a Ph.D from the University of Arkansas, Fayetteville, USA.