Ross / Birnbaum / Lukacs Introduction to Stochastic Dynamic Programming

1. Auflage 2014
ISBN: 978-1-4832-6909-2
Verlag: Elsevier Science & Techn.
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 178 Seiten, Web PDF

Introduction to Stochastic Dynamic Programming presents the basic theory and examines the scope of applications of stochastic dynamic programming. The book begins with a chapter on various finite-stage models, illustrating the wide range of applications of stochastic dynamic programming. Subsequent chapters study infinite-stage models: discounting future returns, minimizing nonnegative costs, maximizing nonnegative returns, and maximizing the long-run average return. Each of these chapters first considers whether an optimal policy need exist-providing counterexamples where appropriate-and then presents methods for obtaining such policies when they do. In addition, general areas of application are presented. The final two chapters are concerned with more specialized models. These include stochastic scheduling models and a type of process known as a multiproject bandit. The mathematical prerequisites for this text are relatively few. No prior knowledge of dynamic programming is assumed and only a moderate familiarity with probability- including the use of conditional expectation-is necessary.

Dr. Sheldon M. Ross is a professor in the Department of Industrial and Systems Engineering at the University of Southern California. He received his PhD in statistics at Stanford University in 1968. He has published many technical articles and textbooks in the areas of statistics and applied probability. Among his texts are A First Course in Probability, Introduction to Probability Models, Stochastic Processes, and Introductory Statistics. Professor Ross is the founding and continuing editor of the journal Probability in the Engineering and Informational Sciences. He is a Fellow of the Institute of Mathematical Statistics, a Fellow of INFORMS, and a recipient of the Humboldt US Senior Scientist Award.

Autoren/Hrsg.

Ross, Sheldon M.

Birnbaum, Z. W.

Lukacs, E.

Inhaltsverzeichnis

1;Front Cover;1
2;Introduction to Stochastic Dynamic Programming;4
3;Copyright Page;5
4;Table of Contents;8
5;Dedication;6
6;Preface;12
7;Chapter I. Finite-Stage Models;14
7.1;1. Introduction;14
7.2;2. A Gambling Model;15
7.3;3. A Stock-Option Model;17
7.4;4. Modular Functions and Monotone Policies;18
7.5;5. Accepting the Best Offer;24
7.6;6. A Sequential Allocation Model;27
7.7;7. The Interchange Argument in Sequencing;30
7.8;Problems;34
7.9;Notes and References;40
8;Chapter II. Discounted Dynamic Programming;42
8.1;1. Introduction;42
8.2;2. The Optimality Equation and Optimal Policy;43
8.3;3. Method of Successive Approximations;48
8.4;4. Policy Improvement;51
8.5;5. Solution by Linear Programming;53
8.6;6. Extension to Unbounded Rewards;55
8.7;Problems;57
8.8;References;61
9;Chapter III. Minimizing Costs—Negative Dynamic Programming;62
9.1;1. Introduction and Some Theoretical Results;62
9.2;2. Optimal Stopping Problems;64
9.3;3. Bayesian Sequential Analysis;71
9.4;4. Computational Approaches;73
9.5;5. Optimal Search;76
9.6;Problems;81
9.7;References;84
10;Chapter IV. Maximizing Rewards—Positive Dynamic Programming;86
10.1;1. Introduction and Main Theoretical Results;86
10.2;2. Applications to Gambling Theory;89
10.3;3. Computational Approaches to Obtaining V;96
10.4;Problems;98
10.5;Notes and References;101
11;Chapter V. Average Reward Criterion;102
11.1;1. Introduction and Counterexamples;102
11.2;2. Existence of an Optimal Stationary Policy;106
11.3;3. Computational Approaches;111
11.4;Problems;116
11.5;Notes and References;118
12;Chapter VI. Stochastic Scheduling;120
12.1;1. Introduction;120
12.2;2. Maximizing Finite-Time Returns—Single Processor;121
12.3;3. Minimizing Expected Makespan—Processors in Parallel;122
12.4;4. Minimizing Expected Makespan—Processors in Series;127
12.5;5. Maximizing Total Field Life;131
12.6;6. A Stochastic Knapsack Model;135
12.7;7. A Sequential-Assignment Problem;137
12.8;Problems;140
12.9;Notes and References;142
13;Chapter VII. Bandit Processes;144
13.1;1. Introduction;144
13.2;2. Single-Project Bandit Processes;144
13.3;3. Multiproject Bandit Processes;146
13.4;4. An Extension and a Nonextension;156
13.5;5. Generalizations of the Classical Bandit Problem;158
13.6;Problems;163
13.7;Notes and References;164
14;Appendix: Stochastic Order Relations;166
14.1;1. Stochastically Larger;166
14.2;2. Coupling;167
14.3;3. Hazard-Rate Ordering;169
14.4;4. Likelihood-Ratio Ordering;170
14.5;Problems;173
14.6;Reference;174
15;Index;176