1st Edition

Support Vector Machines Optimization Based Theory, Algorithms, and Extensions

    364 Pages 60 B/W Illustrations
    by Chapman & Hall

    Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions presents an accessible treatment of the two main components of support vector machines (SVMs)—classification problems and regression problems. The book emphasizes the close connection between optimization theory and SVMs since optimization is one of the pillars on which SVMs are built.

    The authors share insight on many of their research achievements. They give a precise interpretation of statistical leaning theory for C-support vector classification. They also discuss regularized twin SVMs for binary classification problems, SVMs for solving multi-classification problems based on ordinal regression, SVMs for semi-supervised problems, and SVMs for problems with perturbations.

    To improve readability, concepts, methods, and results are introduced graphically and with clear explanations. For important concepts and algorithms, such as the Crammer-Singer SVM for multi-class classification problems, the text provides geometric interpretations that are not depicted in current literature.

    Enabling a sound understanding of SVMs, this book gives beginners as well as more experienced researchers and engineers the tools to solve real-world problems using SVMs.

    Optimization
    Optimization Problems in Euclidian Space
    Convex Programming in Euclidean Space
    Convex Programming in Hilbert Space
    Convex Programming with Generalized Inequality Constraints in Rn
    Convex Programming with Generalized Inequality Constraints in Hilbert Space

    Linear Classification Machines
    Presentation of Classification Problems
    Support Vector Classification (SVC) for Linearly Separable Problems
    Linear Support Vector Classification

    Linear Regression Machines
    Regression Problems and Linear Regression Problems
    Hard ε-Band Hyperplane
    Linear Hard ε-Band Support Vector Regression
    Linear ε-Support Vector Regression

    Kernels and Support Vector Machines
    From Linear Classification to Nonlinear Classification
    Kernels
    Support Vector Machines and Their Properties
    Meaning of Kernels

    Basic Statistical Learning Theory of C-Support Vector Classification
    Classification Problems on Statistical Learning Theory
    Empirical Risk Minimization
    Vapnik Chervonenkis (VC) Dimension
    Structural Risk Minimization
    An Implementation of Structural Risk Minimization
    Theoretical Foundation of C-Support Vector Classification on Statistical Learning Theory

    Model Construction
    Data Generation
    Data Preprocessing
    Model Selection
    Rule Extraction

    Implementation
    Stopping Criterion
    Chunking
    Decomposing
    Sequential Minimal Optimization
    Software

    Variants and Extensions of Support Vector Machines
    Variants of Binary Classification
    Variants of Regression
    Multi-Class Classification
    Semi-Supervised Classification
    Universum Classification
    Privileged Classification
    Knowledge-Based Classification
    Robust Classification
    Multi-Instance Classification
    Multi-Label Classification

    Bibliography

    Index

    Biography

    Naiyang Deng, Yingjie Tian, Chunhua Zhang

    This book provides a concise overview of support vector machines (SVMs), starting from the basics and connecting to many of their most significant extensions. Starting from an optimization perspective provides a new way of presenting the material, including many of the technical details that are hard to find in other texts. And since it includes a discussion of many practical issues important for the effective use of SVMs (e.g., feature construction), the book is valuable as a reference for researchers and practitioners alike.
    —Thorsten Joachims, Associate Professor, Department of Computer Science, Cornell University

    The books on support vector machines (SVMs) in Chinese written by the same authors are very popular in China. It is really great that the authors have translated the books into English and made further extensions on them. One thing which makes the book very unique from the other books is that the authors try to shed light on SVM from the viewpoint of optimization. I believe that the comprehensive and systematic explanation on the basic concepts, fundamental principles, algorithms, and theories of SVM will help readers have a really in-depth understanding of the space. It is really a great book, which many researchers, students, and engineers in computer science and related fields will want to carefully read and routinely consult.
    —Dr. Hang Li, Chief Scientist of Noah’s Ark Lab, Huawei Technologies Co., Ltd

    This book comprehensively covers many topics of support vector machines (SVMs). In particular, it gives a nice connection between optimization theory and support vector machines. In my experience of developing the popular SVM software LIBSVM, I found that many users lack a good understanding of the optimization concept behind SVM. This book starts with explaining basic knowledge of convex optimization and then introduces linear support vector classification and regression. Next, it discusses kernel SVM and the practical implementation. The setting allows readers to easily learn how optimization techniques are used in a machine learning technique such as SVM.
    —Chih-Jen Lin, Professor, Department of Computer Science, National Taiwan University