First Look at Stochastic Processes

This textbook introduces the theory of stochastic processes, that is, randomness which proceeds in time. Using concrete examples like repeated gambling and jumping frogs, it presents fundamental mathematical results through simple, clear, logical theorems and examples. It covers in detail such essential material as Markov chain recurrence criteria, the Markov chain convergence theorem, and optional stopping theorems for martingales. The final chapter provides a brief introduction to Brownian motion, Markov processes in continuous time and space, Poisson processes, and renewal theory.
Interspersed throughout are applications to such topics as gambler's ruin probabilities, random walks on graphs, sequence waiting times, branching processes, stock option pricing, and Markov Chain Monte Carlo (MCMC) algorithms.