Transition Reward

Concept

Theoretically, Transition Reward is the numerical value assigned to moving from one health state to another within a state-transition model. Unlike a state reward, which is accumulated while occupying a health state, a transition reward is incurred only when a specific transition occurs. In health economics, transition rewards are used to represent one-off costs or health effects associated with clinical events such as disease progression, hospital admission, surgery or death.

Mathematically, transition rewards are represented by a reward matrix in which each element specifies the reward associated with a transition between two health states. The expected transition reward for a model cycle is calculated by combining transition probabilities with the corresponding transition rewards. Total rewards are accumulated over successive cycles to estimate lifetime costs and health outcomes.

In practice, transition rewards are estimated from clinical studies, healthcare costing data, utility studies or published literature. Separate transition reward matrices are commonly specified for costs and health outcomes. During model execution, transition rewards are applied whenever a transition occurs, while state rewards continue to account for outcomes accumulated during state occupancy.

---

Purpose

Used to represent one-off costs or health effects associated with movement between health states, enabling accurate estimation of event-specific economic and clinical consequences within state-transition models.

---

Mathematical Formulae

Primary Formula

Expected transition reward for cycle t:

R? = ???(?? ? ??)??

where:

• ??? = state occupancy vector at cycle t
• ?? = transition probability matrix
• ?? = transition reward matrix
• ? = element-wise (Hadamard) product
• ?? = column vector of ones

Supporting Formulae

Transition reward matrix:

?? = [g??]

where:

• g?? = reward associated with transition from state i to state j

Cumulative transition reward:

R = ????? R?

Related Mathematical Methods

• Markov modelling
• State-transition modelling
• Matrix algebra
• Reward processes
• Cohort simulation
• Discounting
• Probabilistic sensitivity analysis

---

Example

A Markov model contains the transitions Stable Disease ? Hospitalisation and Hospitalisation ? Death.

The transition cost matrix includes:

• Stable ? Hospitalisation = �5,000
• Hospitalisation ? Death = �2,000
• All remaining transitions = �0

If 6% of the cohort transitions from Stable Disease to Hospitalisation during one annual cycle, the expected transition cost is:

0.06 ? �5,000 = �300

per member of the cohort for that cycle. This cost is added to any state-specific costs accumulated while occupying the health states.

---

Excel Implementation

Function	Example Formula	Health Economics Application
SUMPRODUCT	=SUMPRODUCT(TransitionProbabilities,TransitionRewards)	Calculate expected transition rewards for a model cycle
MMULT	=MMULT(StateVector,TransitionMatrix)	Estimate transition frequencies before applying rewards
SUM	=SUM(CycleTransitionRewards)	Accumulate transition rewards across the model horizon
IF	=IF(TransitionOccurred,Reward,0)	Apply one-off rewards when a specific transition occurs

---

VBA (Optional)

Automate application of transition reward matrices during each simulation cycle and accumulate discounted event-specific costs and health outcomes.

---

Sources

• Puterman ML. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley; 1994.
• Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322?338.
• Siebert U, Alagoz O, Bayoumi AM, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Medical Decision Making. 2012;32(5):690?700.
• Briggs A, Claxton K, Sculpher M. Decision Modelling for Health Economic Evaluation. Oxford University Press; 2006.
• NICE. Health Technology Evaluation Manual.
• Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. Oxford University Press.