Part 1

Jim is currently living in Scranton. Each year that he lives in Scranton, he has a

probability of 1/2 of staying in Scranton the next year. Otherwise, he has an equally

likely chance of moving to Chicago, Philadelphia, New York, or Seattle, the next year.

On any given year that he lives in Philadelphia, he has a 1/4 probability of moving to

Seattle, a probability of 3/8 of moving to Scranton and a probability of 3/8 of moving to

Chicago the next year. On any year that he lives in Chicago, he has a probability of 1/2

of moving to New York, a probability of 3/8 of moving to Scranton and a probability of

1/8 of moving to Philadelphia the next year.

In answering the questions below, assume Jim will be living in one of the 5 cities forever.

Also assume, for parts (a)-(e) that if Jim moves to Seattle or New York, he will stay

there and will not relocate again.

(a) Is this a valid Markov chain? Create the transition graph and matrix.

(b) What is the probability that Jim eventually will leave the non-Coastal cities (i.e.,

Scranton, Chicago, Philadelphia) permanently?

(c) What is the probability that Jim will eventually relocate permanently to New

York?

(d) What is the expected number of years until Jim leaves Scranton permanently?

(e) Jim’s friend Karen also started out like Jim but in Chicago. She also eventually

relocated to New York or Seattle. What is the expected number of years she lived

in Scranton?

Jim decides that New York is not in his future. Accordingly, when he is Scranton, he

remains there for another year with probability 1/2, and otherwise he has an equally

Jim is currently living in Scranton. Each year that he lives in Scranton, he has a

probability of 1/2 of staying in Scranton the next year. Otherwise, he has an equally

likely chance of moving to Chicago, Philadelphia, New York, or Seattle, the next year.

On any given year that he lives in Philadelphia, he has a 1/4 probability of moving to

Seattle, a probability of 3/8 of moving to Scranton and a probability of 3/8 of moving to

Chicago the next year. On any year that he lives in Chicago, he has a probability of 1/2

of moving to New York, a probability of 3/8 of moving to Scranton and a probability of

1/8 of moving to Philadelphia the next year.

In answering the questions below, assume Jim will be living in one of the 5 cities forever.

Also assume, for parts (a)-(e) that if Jim moves to Seattle or New York, he will stay

there and will not relocate again.

(a) Is this a valid Markov chain? Create the transition graph and matrix.

(b) What is the probability that Jim eventually will leave the non-Coastal cities (i.e.,

Scranton, Chicago, Philadelphia) permanently?

(c) What is the probability that Jim will eventually relocate permanently to New

York?

(d) What is the expected number of years until Jim leaves Scranton permanently?

(e) Jim’s friend Karen also started out like Jim but in Chicago. She also eventually

relocated to New York or Seattle. What is the expected number of years she lived

in Scranton?

Jim decides that New York is not in his future. Accordingly, when he is Scranton, he

remains there for another year with probability 1/2, and otherwise he has an equally

likely chance of moving to any of the other cities. When he is in Chicago, his probability

of moving to Scranton or Philadelphia are in the same proportion as before (does not

necessary imply same probability).

(f) Create the revised transition graph and matrix.

(g) What is the expected number of years until Jim is in Seattle?

For parts h and i, Jim changes his mind about New York and it is a potential relocation

city again (original chain in part a). However, now if Jim moves to Seattle, he is equally

likely to stay there or move to New York the following year. If he is in New York, he has

a 2/3 probability of staying there and 1/3 probability of moving to Seattle the following

year.

(h) Calculate the probability of Jim being in each city (Seattle and New York) on any

given year far into the future.

(i) Given that after a very period of time, Jim is in Seattle. What is the approximate

probability that Jim was living in New York the previous year?

of moving to Scranton or Philadelphia are in the same proportion as before (does not

necessary imply same probability).

(f) Create the revised transition graph and matrix.

(g) What is the expected number of years until Jim is in Seattle?

For parts h and i, Jim changes his mind about New York and it is a potential relocation

city again (original chain in part a). However, now if Jim moves to Seattle, he is equally

likely to stay there or move to New York the following year. If he is in New York, he has

a 2/3 probability of staying there and 1/3 probability of moving to Seattle the following

year.

(h) Calculate the probability of Jim being in each city (Seattle and New York) on any

given year far into the future.

(i) Given that after a very period of time, Jim is in Seattle. What is the approximate

probability that Jim was living in New York the previous year?