Markov Processes

This page includes problems and solutions from my Directed Study on "Markov Processes" during the Spring 2016 semester.

The problems have been taken from the classic book "Elements of the Theory of Markov Processes with Applications" by A. T. Bharucha-Reid.

Following is the Problem Set:

Following are the solutions to the individual problems. The topics or concepts the problem touches are written with a "#":

# Markov Chain
# Conditional Independence

# Communicating States

# Transient State
# Absorption State
# Recurrent State

# Branching Process
# Generating Functions
# Moments
# Cumulants

# Time-Homogeneous Stochastic Process
# Independent Increments
# Differential Equations

# Generating Functions
# Reciprocals

# Birth Process
# Partial Differential Equations

# Periodic Chains
# Limit Matrix

# Random Walk
# Functional Independence
# Recurrence Relation

# Branching Processes (Fundamental Theorem)
# Generating Functions
# Absorption Probabilities

# Birth-and-Death Process
# Conditional Expected Time

# Coefficient of Variation
# Poisson Process
# Simple Birth Process
# Birth-and-Death Process