Modeling and Analysis of Stochastic Systems, Second Edition
Purchasing Options
Features
- Presents a systematic and logical treatment of each class of stochastic process, including discrete- and continuous-time Markov chains and Poisson, renewal, regenerative, semi-Markov, Markov regenerative, and diffusion processes
- Provides a detailed account of Markov models
- Explains how first passage times play an important role in the applications and theory of limiting behavior
- Covers the important topic of phase-type distributions
- Demonstrates the use of queuing models in several applications
- Contains modeling, computational, and conceptual exercises
- Offers a collection of MATLAB®-based programs that can be downloaded from the author’s website
Solutions manual available for qualifying instructors
Summary
Based on the author’s more than 25 years of teaching experience, Modeling and Analysis of Stochastic Systems, Second Edition covers the most important classes of stochastic processes used in the modeling of diverse systems, from supply chains and inventory systems to genetics and biological systems. For each class of stochastic process, the text includes its definition, characterization, applications, transient and limiting behavior, first passage times, and cost/reward models. Along with reorganizing the material, this edition revises and adds new exercises and examples.
New to the Second Edition
- A new chapter on diffusion processes that gives an accessible and non-measure-theoretic treatment with applications to finance
- A more streamlined, application-oriented approach to renewal, regenerative, and Markov regenerative processes
- Two appendices that collect relevant results from analysis and differential and difference equations
Rather than offer special tricks that work in specific problems, this book provides thorough coverage of general tools that enable the solution and analysis of stochastic models. After mastering the material in the text, students will be well-equipped to build and analyze useful stochastic models for various situations.
A collection of MATLAB®-based programs can be downloaded from the author’s website and a solutions manual is available for qualifying instructors.
Table of Contents
Introduction
What in the World Is a Stochastic Process?
How to Characterize a Stochastic Process
What Do We Do with a Stochastic Process?
Discrete-Time Markov Chains: Transient Behavior
Definition and Characterization
Examples
DTMCs in Other Fields
Marginal Distributions
Occupancy Times
Computation of Matrix Powers
DTMCs: First Passage Times
Definitions
Cumulative Distribution Function of T
Absorption Probabilities
Expectation of T
Generating Function and Higher Moments of T
DTMCs: Limiting Behavior
Exploring the Limiting Behavior by Examples
Irreducibility and Periodicity
Recurrence and Transience
Determining Recurrence and Transience: Infinite DTMCs
Limiting Behavior of Irreducible DTMCs
Examples: Limiting Behavior of Infinite State-Space Irreducible DTMCs
Limiting Behavior of Reducible DTMCs
DTMCs with Costs and Rewards
Reversibility
Poisson Processes
Exponential Distributions
Poisson Process: Definitions
Event Times in a Poisson Process
Superposition and Splitting of Poisson Processes
Non-Homogenous Poisson Process
Compound Poisson Process
Continuous-Time Markov Chains
Definitions and Sample Path Properties
Examples
Transient Behavior: Marginal Distribution
Transient Behavior: Occupancy Times
Computation of P(t): Finite State-Space
Computation of P(t): Infinite State-Space
First-Passage Times
Exploring the Limiting Behavior by Examples
Classification of States
Limiting Behavior of Irreducible CTMCs
Limiting Behavior of Reducible CTMCs
CTMCs with Costs and Rewards
Phase-Type Distributions
Reversibility
Queueing Models
Introduction
Properties of General Queueing Systems
Birth and Death Queues
Open Queueing Networks
Closed Queueing Networks
Single Server Queues
Retrial Queue
Infinite Server Queue
Renewal Processes
Introduction
Properties of N(t)
The Renewal Function
Renewal-Type Equation
Key Renewal Theorem
Recurrence Times
Delayed Renewal Processes
Alternating Renewal Processes
Semi-Markov Processes
Renewal Processes with Costs/Rewards
Regenerative Processes
Markov Regenerative Processes
Definitions and Examples
Markov Renewal Process and Markov Renewal Function
Key Renewal Theorem for MRPs
Extended Key Renewal Theorem
Semi-Markov Processes: Further Results
Markov Regenerative Processes
Applications to Queues
Diffusion Processes
Brownian Motion
Sample Path Properties of BM
Kolmogorov Equations for Standard Brownian Motion
First Passage Times
Reflected SBM
Reflected BM and Limiting Distributions
BM and Martingales
Cost/Reward Models
Stochastic Integration
Stochastic Differential Equations
Applications to Finance
Epilogue
Appendix A: Probability of Events
Appendix B: Univariate Random Variables
Appendix C: Multivariate Random Variables
Appendix D: Generating Functions
Appendix E: Laplace–Stieltjes Transforms
Appendix F: Laplace Transforms
Appendix G: Modes of Convergence
Appendix H: Results from Analysis
Appendix I: Difference and Differential Equations
Answers to Selected Problems
References
Index
Exercises appear at the end of each chapter.
Author Bio(s)
Editorial Reviews
This book provides insight on the mathematical methods for modelling and understanding real-world processes. … for the teacher, this book provides an abundant resource of examples for class exercises and student homework. The modelling examples include many that would be of interest for advanced undergraduate courses on inventory management, reliability, operations management and queuing networks. … This is a book for the few who would like to know almost everything about stochastic processes and for the many who need to know something, and would like to have their eyes opened to the many models and analytical results available on stochastic processes.
—J. Smart, Journal of the Operational Research Society, Vol. 62, 2011
This book is written by a well-experienced scientist and teacher. The material is carefully chosen and presented in a systematic way, starting with the basics and reaching more advanced topics. … It is very useful to see a large number of examples worked out in detail. Thus, besides university courses, the book can be successfully used for self-education. … This book is a good example of how to write on sophisticated topics about stochastic processes without involving the heavy measure theoretic machinery. Another advantage of the book is the application-oriented approach that is followed by the author. … This book can be strongly recommended for introductory, intermediate or advanced university courses on stochastic processes. Both students and their teachers may benefit considerably from it.
—Journal of the Royal Statistical Society: Series A, July 2011
… an accessible, well paced, and very nicely presented book. The publishers are also to be commended on its nice production: it is the sort of book which is a pleasure to read. In all, it is an excellent textbook for use in introductory courses on stochastic processes.
—International Statistical Review (2010), 78, 3
This book may be recommended for all readers who want to understand, build and analyse stochastic models and relevant problems.
—Zentralblatt MATH 1191
Praise for the First Edition:
… a beautiful introductory textbook on stochastic processes … Plenty of applications, examples, and exercises bring the reader to the intuitive understanding of the subject.
—Zentralblatt MATH
Well-chosen examples and interesting exercises make this text a good choice for a first course in stochastic processes for a broad class of students.
—Journal of the American Statistical Association
Downloads Updates
| Resource | OS Platform | Updated | Description | Instructions |
|---|---|---|---|---|
| Cross Platform | June 11, 2010 | Link to Errata | click on http://www.unc.edu/~vkulkarn/gradcorrections2.pdf |
