Competitive Markov Decision Processes -- Copyright -- Preface -- Contents -- 1 Introduction -- Part I Mathematical Programming Perspective -- 2 Markov Decision Processes: The Noncompetitive Case -- 3 Stochastic Games via Mathematical Programming -- Part II Existence, Structure and Applications -- 4 Summable Stochastic Games -- 5 Average Reward Stochastic Games -- 6 Applications and Special Classes of Stochastic Games -- Appendix G Matrix and Bimatrix Games, and Mathematical Programming -- Appendix H A Theorem of Hardy and Littlewood -- Appendix M Markov Chains -- Appendix P Complex Varieties and the Limit Discount Equation -- References -- Index.