Controlled Queueing Systems
Purchasing Options
Features
Summary
This is the first book completely devoted to controlled queueing systems. The book gathers the newest results of the theory of Markov decision processes related to queueing models and demonstrates their applications to main types of control in queueing systems, including control of arrivals, control of service mechanism, and control of service discipline. Emphasis is placed on conditions providing further "good" structural properties of Markov optimal strategies such as monotonicity, threshold or hysteretic character, and priority.
Each chapter is followed by exercises, most of which allow the reader to complete technical fragments of proofs. The text assumes the reader is familiar with standard courses of analysis, probability theory, and queueing theory.
Table of Contents
Semi-Regenerative Decision Models
Description of Basic Decision Model
Rigorous Definitions and Assumptions
Examples of Controlled Queues
Optimization Problems
Renewal Kernels of the Decision Model
Special Classes of Strategies
Sufficiency of Markov Strategies
Dynamic Programming
Discounting in Continuous Time
Dynamic Programming Equation
Bellman Functions
Finite-Horizon Problem
Infinite-Horizon Discounted-Cost Problem
Random-Horizon Problem
Average Cost Criterion
Preliminaries: Weak Topology, Limit Passages
Preliminaries: Taboo Probabilities, Limit Theorems for Markov Renewal Processes
Notation, Recurrence-Communication Assumptions, Examples
Existence of Optimal Policies
Existence of Optimal Strategies: General Criterion
Existence of Optimal Strategies: Sufficient Conditions
Optimality Equation
Constrained Average-Cost Problem
Average-Cost Optimality as Limiting Case of Discounted-Cost Optimality
Continuously Controlled Markov Jump Processes
Facts About Measurability of Stochastic Processes
Marked Point Processes and Random Measures
The Predictable s-Algebra
Dual Predictable Projections of Random Measures
Definition of Controlled Markov Jump Process
An M/M/1 Queue With Controllable Input and Service Rate
Dynamic Programming
Optimization Problems
Structured Optimization Problems for Decision Processes
Convex Regularization
Submodular and Supermodular Functions
Existence of Monotone Solutions for Optimization Problems
Processes with Bounded Drift
Birth and Death Processes
Control of Arrivals
The Model Description
Finite-Horizon Discounted-Cost Problem
Cost Functionals
Infinite-Horizon Case with and without Discounting
Optimal Dynamic Pricing Policy: Model; Results
Control of Service Mechanism
Description of the System
Static Optimization Problem
Optimal Policies for the Queueing Process
Service System with Two Interacting Servers
Analysis of Optimality Equation
Optimal Control in Models with Several Classes of Customers
Description of Models and Processes
Associated Controlled Processes
Existence of Optimal Simple Strategies for the Systems with Alternating Priority
Existence of Optimal Simple Strategy for the System with Feedback
Equations for Stationary Distributions
Stationary Characteristics of the Systems with Alternating Priority
Stationary Characteristics of the System with Feedback
Models with Alternating Priority: Linear Programming Problem
Linear Programming Problem in the Model with Feedback
Model with Periods of Idleness and Discounted-Cost Criterion
Basic Formulas
Construction of Optimal Modified Priority Discipline
Bibliography
Index
Each chapter also includes an Introduction, and a Remarks and Exercises section
