1st Edition

Zeroing Dynamics, Gradient Dynamics, and Newton Iterations

    340 Pages 294 B/W Illustrations
    by CRC Press

    Neural networks and neural dynamics are powerful approaches for the online solution of mathematical problems arising in many areas of science, engineering, and business. Compared with conventional gradient neural networks that only deal with static problems of constant coefficient matrices and vectors, the authors’ new method called zeroing dynamics solves time-varying problems.

    Zeroing Dynamics, Gradient Dynamics, and Newton Iterations is the first book that shows how to accurately and efficiently solve time-varying problems in real-time or online using continuous- or discrete-time zeroing dynamics. The book brings together research in the developing fields of neural networks, neural dynamics, computer mathematics, numerical algorithms, time-varying computation and optimization, simulation and modeling, analog and digital hardware, and fractals.

    The authors provide a comprehensive treatment of the theory of both static and dynamic neural networks. Readers will discover how novel theoretical results have been successfully applied to many practical problems. The authors develop, analyze, model, simulate, and compare zeroing dynamics models for the online solution of numerous time-varying problems, such as root finding, nonlinear equation solving, matrix inversion, matrix square root finding, quadratic optimization, and inequality solving.

    Time-Varying Root Finding
    Time-Varying Square Root Finding
    Introduction
    Problem Formulation and Continuous-Time (CT) Models
    S-DTZD Model and Newton Iteration
    Illustrative Examples

    Time-Varying Cube Root Finding
    Introduction
    ZD Models for Time-Varying Case
    Simplified ZD Models for Constant Case and Newton Iteration
    Illustrative Examples

    Time-Varying 4th Root Finding
    Introduction
    Problem Formulation and ZD Models
    GD Model
    Illustrative Examples

    Time-Varying 5th Root Finding
    Introduction
    ZD Models for Time-Varying Case
    Simplified ZD Models for Constant Case and Newton Iteration
    Illustrative Examples
    Appendix: Extension to Time-Varying pth Root Finding

    Nonlinear Equation Solving
    Time-Varying Nonlinear Equation Solving
    Introduction
    Problem Formulation and Solution Models
    Convergence Analysis
    Illustrative Example

    Static Nonlinear Equation Solving
    Problem Formulation and Continuous-Time Models
    DTZD Models
    Comparison between CTZD Model and Newton Iteration
    Further Discussion to Avoid Local Minimum

    System of Nonlinear Equations Solving
    Problem Formulation and CTZD Model
    Discrete-Time Models

    Matrix Inversion
    ZD Models and Newton Iteration
    Introduction
    ZD Models
    Choices of Initial State X0
    Choices of Step Size h
    Illustrative Examples
    New DTZD Models Aided with Line-Search Algorithm

    Moore–Penrose Inversion
    Introduction
    Preliminaries
    ZD Models for Moore–Penrose Inverse
    Comparison between ZD and GD Models
    Simulation and Verification
    Application to Robot Arm

    Matrix Square Root Finding
    ZD Models and Newton Iteration
    Introduction
    Problem Formulation and ZD Models
    Link and Explanation to Newton Iteration
    Line-Search Algorithm
    Illustrative Examples

    ZD Model Using Hyperbolic Sine Activation Functions
    Model and Activation Functions
    Convergence Analysis
    Robustness Analysis
    Illustrative Examples

    Time-Varying Quadratic Optimization
    ZD Models for Quadratic Minimization
    Introduction
    Problem Formulation and CTZD Model
    DTZD Models
    GD Models
    Illustrative Example

    ZD Models for Quadratic Programming
    Introduction
    CTZD Model
    DTZD Models
    Illustrative Examples

    Simulative and Experimental Application to Robot Arms
    Problem Formulation and Reformulation
    Solution Models
    Computer Simulations
    Hardware Experiments

    Time-Varying Inequality Solving
    Linear Inequality Solving
    Introduction
    Time-Varying Linear Inequality
    Constant Linear Inequality
    Illustrative Examples
    System of Time-Varying Linear Inequalities
    Illustrative Examples

    System of Time-Varying Nonlinear Inequalities Solving
    Introduction
    Problem Formulation
    CZD Model and Convergence Analysis
    MZD Model and Convergence Analysis
    Illustrative Example

    Application to Fractal
    Fractals Yielded via Static Nonlinear Equation
    Introduction
    Complex-Valued ZD Models
    Illustrative Examples

    Fractals Yielded via Time-Varying Nonlinear Equation
    Introduction
    Complex-Valued ZD Models
    Illustrative Examples

    A summary appears at the end of each chapter.

    Biography

    Yunong Zhang is a professor in the School of Information Science and Technology at Sun Yat-sen University. He is also with the SYSU-CMU Shunde International Joint Research Institute for cooperative research. He has published more than 375 scientific works of various types and has been a winner of the Best Paper Award of ISSCAA and the Best Paper Award of ICAL. He was among the 2014 Highly Cited Scholars of China. His main research interests include neural networks, robotics, computation, and optimization. He earned a PhD from the Chinese University of Hong Kong.

    Lin Xiao is a lecturer in the College of Information Science and Engineering at Jishou University. His current research interests include neural networks, intelligent information processing, robotics, and related areas. He earned a PhD from Sun Yat-sen University.

    Zhengli Xiao is currently pursuing an MS in the Department of Computer Science in the School of Information Science and Technology at Sun Yat-sen University. He is also with the SYSU-CMU Shunde International Joint Research Institute for cooperative research. His current research interests include neural networks, intelligent information processing, and learning machines. He earned a BS in software engineering from Changchun University of Science and Technology.

    Mingzhi Mao is an associate professor in the School of Information Science and Technology at Sun Yat-sen University. His main research interests include intelligence algorithms, software engineering, and management information systems. He earned a PhD from the Department of Computer Science at Sun Yat-sen University.