1st Edition

Subspace Learning of Neural Networks

By Jian Cheng Lv, Zhang Yi, Jiliu Zhou Copyright 2011
    256 Pages 84 B/W Illustrations
    by CRC Press

    256 Pages 84 B/W Illustrations
    by CRC Press

    Using real-life examples to illustrate the performance of learning algorithms and instructing readers how to apply them to practical applications, this work offers a comprehensive treatment of subspace learning algorithms for neural networks. The authors summarize a decade of high quality research offering a host of practical applications. They demonstrate ways to extend the use of algorithms to fields such as encryption communication, data mining, computer vision, and signal and image processing to name just a few. The brilliance of the work lies with how it coherently builds a theoretical understanding of the convergence behavior of subspace learning algorithms through a summary of chaotic behaviors.

    Preface

    Chapter 1. Introduction

    1.1 Introduction

    1.1.1 Linear Neural Networks

    1.1.2 Subspace Learning

    1.2 Subspace Learning Algorithms

    1.2.1 PCA Learning Algorithms

    1.2.2 MCA Learning Algorithms

    1.2.3 ICA Learning Algorithms

    1.3 Methods for Convergence Analysis

    1.3.1 SDT Method

    1.3.2 DCT Method

    1.3.3 DDT Method

    1.4 Block Algorithms

    1.5 Simulation Data Set and Notation

    1.6 Conclusions

    Chapter 2. PCA Learning Algorithms with Constants Learning Rates

    2.1 Oja’s PCA Learning Algorithms

    2.1.1 The Algorithms

    2.1.2 Convergence Issue

    2.2 Invariant Sets

    2.2.1 Properties of Invariant Sets

    2.2.2 Conditions for Invariant Sets

    2.3 Convergence analysis via DDT Method

    2.3.1 Problem Formulation

    2.3.2 Proof of Convergence

    2.4 Convergence analysis of Xu’s LMSER algorithm

    2.5 Discussions

    2.5.1 Learning Rates Selection

    2.5.2 Initial Points Selection

    2.6 Conclusions

    Chapter 3. PCA Learning Algorithms with Adaptive Learning Rates

    3.1 Introduction

    3.2 Adaptive Learning Rates

    3.3 Oja’s Algorithm with Adaptive Learning Rates

    3.4 Convergence Analysis of Oja’s Algorithm with Adaptive Learning Rates

    3.4.1 Boundedness

    3.4.2 Global Convergence

    3.5 Simulations and Discussions

    3.6 Conclusions

    Chapter 4. GHA PCA Learning Algorithms

    4.1 GHA PCA Learning Alogrithms

    4.1.1 The Algorithms

    4.1.2 Convergence Issue

    4.2 Problem Formulation

    4.3 Convergence Analysis via DDT Method

    4.3.1 Outline of Proof

    4.3.2 Detail of Proof

    4.4 Discussions and Simulations

    4.4.1 Example 1

    4.4.2 Example 2

    4.4.3 Example 3

    4.5 Conclusions

    Chapter 5. MCA Learning Algorithms

    5.1 MCA Learning Algorithms

    5.1.1 The Algorithms

    5.1.2 Convergence Issue

    5.2 Invariant Sets

    5.2.1 Properties of Invariant Sets

    5.2.2 Conditions for Invariant Sets

    5.3 Convergence Analysis via DDT Methods

    5.3.1 Problem Formulation

    5.3.2 Proof of Convergence

    5.4 Simulations and Discussions

    5.5 Conclusions

    Chapter 6. ICA Learning Algorithms

    6.1 Hyvarinen-Oja Algorithm

    6.1.1 The Algorithm

    6.1.2 Convergence Issue

    6.2 Invariant Sets

    6.2.1 Properties of Invariant Sets

    6.2.2 Conditions for Invariant Sets

    6.3 Convergence Analysis via DDT Method

    6.3.1 Problem Formulation

    6.3.2 Proof of Convergence

    6.4 Simulations and Discussions

    6.5 Conclusions

    Chapter 7. Chaotic Behaviors Arising from Learning Algorithms

    7.1 Introduction to Chaotic Behaviors

    7.1.1 Chaotic Behaviors

    7.1.2 Lyapunov Exponents

    7.2 Chaotic Behaviors Arising from PCA Learning Algorithms

    7.2.1 Computing of Lyapunov Exponents

    7.2.2 Simulation Results

    7.3 Chaotic Behaviors Arising from MCA Learning Algorithms

    7.3.1 Computing of Lyapunov Exponents

    7.3.2 Simulation Results

    7.4 Chaotic Behaviors Arising from ICA Learning Algorithms

    7.4.1 Computing of Lyapunov Exponents

    7.4.2 Simulation Results

    7.5 Conclusions

    Chapter 8. Multi-Block-Based MCA for Nonlinear Surface Fitting

    8.1 Introduction

    8.2 MCA Neural Network for Nonlinear Surface Fitting

    8.3 Multi-Block-Based MCA

    8.4 Multi-Block-Based MCA for Nonlinear Surface Fitting

    8.5 Conclusions

    Chapter 9. A ICA Algorithm for Extracting Fetal Electrocardiogram

    9.1 Introduction

    9.2 Problem Formulation

    9.3 The Proposed Algorithm

    9.4 Extracting Fetal Electrocardiogram

    9.5 Conclusions

    Chapter 10. Some Applications of PCA Neural Networks

    10.1 Introduction

    10.2 Rigid Medical Image Registration

    10.2.1 Introduction

    10.2.2 Method

    10.2.3 Simulation

    10.2.4 Conclusions

    10.3 A Chaotic Encryption System Based on PCA Algorithm

    10.3.1 Chaos and Encryption

    10.3.2 A Chaotic Encryption System

    10.3.3 Simulation

    10.3.4 Conclusion

    10.4 Conclusion

    Biography

    Jian Cheng LV and Zhang Yi are affiliated with the Machine Intelligence Lab of the College of Computer Science at Sichuan University. Jiliu Zhou is affiliated with the College of Computer Science at Sichuan University.