Jim is currently living in Scranton. Each year that he lives in Scranton, he has a
probability of 1/2 of staying in Scranton the next year. Otherwise, he has an equally
likely chance of moving to Chicago, Philadelphia, New York, or Seattle, the next year.
On any given year that he lives in Philadelphia, he has a 1/4 probability of moving to
Seattle, a probability of 3/8 of moving to Scranton and a probability of 3/8 of moving to
Chicago the next year. On any year that he lives in Chicago, he has a probability of 1/2
of moving to New York, a probability of 3/8 of moving to Scranton and a probability of
1/8 of moving to Philadelphia the next year.
In answering the questions below, assume Jim will be living in one of the 5 cities forever.
Also assume, for parts (a)-(e) that if Jim moves to Seattle or New York, he will stay
there and will not relocate again.
(a) Is this a valid Markov chain? Create the transition graph and matrix.
(b) What is the probability that Jim eventually will leave the non-Coastal cities (i.e.,
Scranton, Chicago, Philadelphia) permanently?
(c) What is the probability that Jim will eventually relocate permanently to New
(d) What is the expected number of years until Jim leaves Scranton permanently?
(e) Jim’s friend Karen also started out like Jim but in Chicago. She also eventually
relocated to New York or Seattle. What is the expected number of years she lived
Jim decides that New York is not in his future. Accordingly, when he is Scranton, he
remains there for another year with probability 1/2, and otherwise he has an equally
likely chance of moving to any of the other cities. When he is in Chicago, his probability
of moving to Scranton or Philadelphia are in the same proportion as before (does not
necessary imply same probability).
(f) Create the revised transition graph and matrix.
(g) What is the expected number of years until Jim is in Seattle?
For parts h and i, Jim changes his mind about New York and it is a potential relocation
city again (original chain in part a). However, now if Jim moves to Seattle, he is equally
likely to stay there or move to New York the following year. If he is in New York, he has
a 2/3 probability of staying there and 1/3 probability of moving to Seattle the following
(h) Calculate the probability of Jim being in each city (Seattle and New York) on any
given year far into the future.
(i) Given that after a very period of time, Jim is in Seattle. What is the approximate
probability that Jim was living in New York the previous year?