Although valued for its ability to allow teams to collaborate and foster coalitional behaviors among the participants, game theory’s application to networking systems is not without challenges. Distributed Strategic Learning for Wireless Engineers illuminates the promise of learning in dynamic games as a tool for analyzing network evolution and underlines the potential pitfalls and difficulties likely to be encountered.
Establishing the link between several theories, this book demonstrates what is needed to learn strategic interaction in wireless networks under uncertainty, randomness, and time delays. It addresses questions such as:
- How much information is enough for effective distributed decision making?
- Is having more information always useful in terms of system performance?
- What are the individual learning performance bounds under outdated and imperfect measurement?
- What are the possible dynamics and outcomes if the players adopt different learning patterns?
- If convergence occurs, what is the convergence time of heterogeneous learning?
- What are the issues of hybrid learning?
- How can one develop fast and efficient learning schemes in scenarios where some players have more information than the others?
- What is the impact of risk-sensitivity in strategic learning systems?
- How can one construct learning schemes in a dynamic environment in which one of the players do not observe a numerical value of its own-payoffs but only a signal of it?
- How can one learn "unstable" equilibria and global optima in a fully distributed manner?
The book provides an explicit description of how players attempt to learn over time about the game and about the behavior of others. It focuses on finite and infinite systems, where the interplay among the individual adjustments undertaken by the different players generates different learning dynamics, heterogeneous learning, risk-sensitive learning, and hybrid dynamics.
Introduction to Learning in Games
Basic Elements of Games
Robust Games in Networks
Basic Robust Games
Basics of Robust Cooperative Games
Distributed Strategic Learning
Distributed Strategic Learning in Wireless Networks
Strategy Learning
Introduction
Strategy Learning under Perfect Action Monitoring
Fully Distributed Strategy-Learning
Stochastic Approximations
Chapter Review
Discussions and Open Issues
Payoff Learning and Dynamics
Introduction
Learning Equilibrium Payoffs
Payoff Dynamics
Routing Games with Parallel Links
Numerical Values of Payoffs Are Not Observed
Introduction
Model and Notations
Pseudo-Trajectory
Hybrid and Combined Dynamics
Learning in Games with Continuous Action Spaces
CODIPAS for Stable Games with Continuous Action Spaces 183
CODIPAS-RL via Extremum-Seeking
Designer and Users in an Hierarchical System
From Fictitious Play with Inertia to CODIPAS-RL . . . . . 191
CODIPAS-RL with Random Number of Active Players
CODIPAS for Multi-Armed Bandit Problems
CODIPAS and Evolutionary Game Dynamics
Fastest Learning Algorithms
Learning under Delayed Measurement
Introduction
Learning under Delayed Imperfect Payoffs
Reacting to the Interference
Learning in Constrained Robust Games
Introduction
Constrained One-Shot Games
Quality of Experience
Relevance in QoE and QoS satisfaction
Satisfaction Levels as Benchmarks
Satisfactory Solution
Efficient Satisfactory Solution
Learning a Satisfactory Solution
From Nash Equilibrium to Satisfactory Solution
Mixed and Near-Satisfactory Solution
CODIPAS with Dynamic Satisfaction Level
Random Matrix Games
Mean-Variance Response and Demand Satisfaction
Learning under Random Updates
Introduction
Description of the Random Update Model
Fully Distributed Learning
Dynamic Routing Games with Random Traffic
Extensions
Mobility-Based Learning in Cognitive Radio Networks
Hybrid Strategic Learning
Quiz
Chapter Review
Fully Distributed Learning for Global Optima
Introduction
Resource Selection Games
Frequency Selection Games
User-Centric Network Selection
Markov Chain Adjustment
Pareto Optimal Solutions
Learning in Risk-Sensitive Games
Introduction
Risk-Sensitive in Dynamic Environment
Risk-sensitive CODIPAS
Risk-Sensitivity in Networking and Communications
Risk-Sensitive Mean Field Learning
Extensions
Chapter Review
Appendix
Basics of Dynamical Systems
Basics of Stochastic Approximations
Differential Inclusion
Markovian Noise
Biography
Hamidou Tembine