An Introduction to Stochastic Processes

Markov chains and continuous-time Markov processes are treated on both finite and countable state spaces, with elementary proofs of central results such as limiting distributions and ergodic theorems. Differences between finite and countable settings are highlighted to enhance understanding, and martingales are introduced as a powerful framework for analyzing stochastic processes. The presentation remains accessible to students with a background in basic probability and linear algebra, without requiring measure theory.

Poisson processes are developed beyond the traditional one-dimensional case to include multi-dimensional processes, expanding applications while maintaining clarity. The book also provides a simple algorithm for generating non-homogeneous Poisson processes in any dimension. Packed with examples and exercises of varying difficulty, this text bridges theory and practice, making it an essential resource for students, instructors, and anyone seeking a solid foundation in stochastic processes.