Kirkwood Markov Processes

1. Auflage 2015
ISBN: 978-1-4822-4074-0
Verlag: Taylor & Francis
Format: PDF
Kopierschutz: Adobe DRM (»Systemvoraussetzungen)

E-Book, Englisch, 340 Seiten

Reihe: Advances in Applied Mathematics

Clear, rigorous, and intuitive, Markov Processes provides a bridge from an undergraduate probability course to a course in stochastic processes and also as a reference for those that want to see detailed proofs of the theorems of Markov processes. It contains copious computational examples that motivate and illustrate the theorems. The text is designed to be understandable to students who have taken an undergraduate probability course without needing an instructor to fill in any gaps.

The book begins with a review of basic probability, then covers the case of finite state, discrete time Markov processes. Building on this, the text deals with the discrete time, infinite state case and provides background for continuous Markov processes with exponential random variables and Poisson processes. It presents continuous Markov processes which include the basic material of Kolmogorov’s equations, infinitesimal generators, and explosions. The book concludes with coverage of both discrete and continuous reversible Markov chains.

While Markov processes are touched on in probability courses, this book offers the opportunity to concentrate on the topic when additional study is required. It discusses how Markov processes are applied in a number of fields, including economics, physics, and mathematical biology. The book fills the gap between a calculus based probability course, normally taken as an upper level undergraduate course, and a course in stochastic processes, which is typically a graduate course.

Zielgruppe

Undergraduate students in probability, mathematics, statistics, engineering, and physics.

Autoren/Hrsg.

Kirkwood, James R.

Fachgebiete

• Wirtschaftswissenschaften Betriebswirtschaft Unternehmensforschung

Inhaltsverzeichnis

Review of Probability
Short History
Review of Basic Probability Definitions
Some Common Probability Distributions
Properties of a Probability Distribution
Properties of the Expected Value
Expected Value of a Random Variable with Common Distributions
Generating Functions
Moment Generating Functions
Exercises
Discrete-Time, Finite-State Markov Chains
Introduction
Notation
Transition Matrices
Directed Graphs: Examples of Markov Chains
Random Walk with Reflecting Boundaries
Gambler’s Ruin
Ehrenfest Model
Central Problem of Markov Chains
Condition to Ensure a Unique Equilibrium State
Finding the Equilibrium State
Transient and Recurrent States
Indicator Functions
Perron-Frobenius Theorem
Absorbing Markov Chains
Mean First Passage Time
Mean Recurrence Time and the Equilibrium State
Fundamental Matrix for Regular Markov Chains
Dividing a Markov Chain into Equivalence Classes
Periodic Markov Chains
Reducible Markov Chains
Summary
Exercises
Discrete-Time, Infinite-State Markov Chains
Renewal Processes
Delayed Renewal Processes
Equilibrium State for Countable Markov Chains
Physical Interpretation of the Equilibrium State
Null Recurrent versus Positive Recurrent States
Difference Equations
Branching Processes
Random Walk in
Exercises
Exponential Distribution and Poisson Process
Continuous Random Variables
Cumulative Distribution Function (Continuous Case)
Exponential Distribution
o(h) Functions
Exponential Distribution as a Model for Arrivals
Memoryless Random Variables
Poisson Process
Poisson Processes with Occurrences of Two Types
Exercises
Continuous-Time Markov Chains
Introduction
Generators of Continuous Markov Chains: The Kolmogorov Forward and Backward Equations
Connection Between the Steady State of a Continuous Markov Chain and the Steady State of the Embedded Matrix
Explosions
Birth and Birth-Death Processes
Birth and Death Processes
Queuing Models
Detailed Balance Equations
Exercises
Reversible Markov Chains
Random Walks on Weighted Graphs
Discrete-Time Birth-Death Process as a Reversible Markov Chain
Continuous-Time Reversible Markov Chains
Exercises
Bibliography