Abstract: This chapter focuses on the first stochastic process, Markov process { X t }, given the values of X t . The chapter discusses the discrete time Markov chain which is a Markov process whose state space is a finite or countable set, and whose (time) index set is T = (0, 1, 2, …). The transition probability matrices of a Markov chain are reviewed, and some Markov chain models like an inventory model, and the Ehrenfest urn model are discussed. The chapter discusses Markov Chains in genetics and a discrete queuing Markov chain, first step analysis, and reviews some special Markov Chains like the two state Markov chain, Markov chains associated with iid random variables, one-dimensional random walks and success runs, functionals of random walks and success runs like —the general random walk, cash management and the success runs markov chain. Another look at first step analysis is presented in the chapter. The Markov chain models and some special Markov chains explains the stochastic model more precisely.
Publication Year: 2011
Publication Date: 2011-01-01
Language: en
Type: book-chapter
Indexed In: ['crossref']
Access and Citation
Cited By Count: 5
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot