Action Research

Action Research in Health Services and Priority Setting

Action research joins the study of a problem in health care to the work of changing it, so the people who run a service help design the research, act on its findings and judge what happened. Health economists meet it where economic evidence has to be put to work inside an organisation, for example when a health authority builds a programme budgeting and marginal analysis process with its own managers and clinicians. The approach trades the control of a trial for local relevance, which shapes how its findings should be read in commissioning and health technology assessment.

Origins in Lewin and three defining elements

Meyer, writing in the BMJ in 2000, described action research as a style of research rather than a specific method. Meyer traced the term to its first use in 1946 by Kurt Lewin, a social scientist concerned with intergroup relations and minority problems in the United States, and noted that it is now identified with research in which researchers work explicitly with and for people rather than on them.

Meyer identified three elements in most definitions. The first is participation, since participants must perceive the need to change and be willing to play an active part in both the research and the change. The second is a democratic impulse, in which participants are treated as equals and the researcher acts as a facilitator of change who consults them on the action and on how it will be evaluated. The third is a simultaneous contribution to social science and to social change.

Action research is not the same as qualitative research, but it typically draws on qualitative methods such as interviews and observation, and some projects add other methods, such as the focus groups with service users planned in the Holmes protocol described below.

The cycle of planning, acting, observing and reflecting

Action research proceeds in repeated cycles instead of a single fixed protocol. Meyer described collaborative spirals of planning, acting, observing, reflecting and replanning, with findings fed back to participants throughout the study for validation and to inform the next stage. Baum, MacDougall and Smith described the same iterative reflective cycle in participatory action research as a corkscrew of data collection, reflection and action.

A protocol for an NHS dentistry study by Holmes and colleagues gives a health economics example. The principal investigator was to act as a facilitator at advisory panel meetings and work alongside participants within multiple action research cycles of plan, act, observe and reflect, while the panel was to use programme budgeting and marginal analysis to rank proposals for local dental services. One stated reason for the design was to involve NHS staff as co-researchers instead of doing research on them.

Four types and the question of who controls the change

Meyer reproduced a typology adapted from Hart and Bond with four types, experimental, organisational, professionalising and empowering, running from a consensus model of society to a conflict model. In the organisational type the problem is defined by the most powerful group, with some negotiation with users, while in the empowering type it emerges from negotiation by less powerful groups. Meyer suggested using the typology to critique studies. A project run by senior managers to release resources is organisational in character.

Action research and economic evidence in local priority setting

Formal economic methods for allocating a fixed budget, such as marginal analysis, only change spending if managers adopt them, and action research has been one route for building such processes inside organisations. Mitton and colleagues reported a participatory action research project in the Calgary Health Region in Alberta that instituted a framework they called macro-marginal analysis. It identified areas for service growth and for resource release and then judged shifts against locally generated criteria. For the 2002/03 fiscal year the region identified over 40 million dollars in resource releases, about 3% of its total budget.

Patten, Mitton and Donaldson reported participants' reflections on a participatory action research project that built a macro-level priority setting framework in a large urban health authority in Alberta. They described the use of qualitative research in health economics to refine a priority setting model as a relatively new advance, found participatory action research an appropriate and helpful approach, and warned that support for the change must be sustained long enough to embed the new practices. Peacock and colleagues later proposed combining programme budgeting and marginal analysis with multi-criteria decision analysis, participatory action research and accountability for reasonableness.

Cornelissen and colleagues used action research from 2006 to 2009 to study the impact of programme budgeting and marginal analysis in a Canadian health authority. Participants prioritised 760,000 dollars of investment proposals and 38,000 dollars of disinvestment proposals, but there was no clear evidence that resources were reallocated as a result, and the authors argued for counting changes in priority setting knowledge, attitudes and practice as impacts too. Byskov and colleagues in the REACT project formed action research teams of district health managers and supporting researchers in one district each in Kenya, Tanzania and Zambia to apply accountability for reasonableness, using a Describe, Evaluate, Improve cycle. Effects ranged from greater awareness of fairness to broader engagement of staff and stakeholders in priority setting, and the authors noted the project was too short to show effects on ultimate outcomes such as quality, equity, trust and health.

Worked example of a reallocation cycle with an implementation shortfall

All figures here are illustrative. A commissioning group runs an action research project in which a panel of managers and clinicians uses marginal analysis. In the first plan stage the panel agrees to withdraw GBP 200,000 from a low-value service, losing an estimated 4 QALYs, and to reinvest it in a service expected to gain 14 QALYs. Facilitation and data collection cost GBP 50,000 per cycle from the same budget, and the cost-effectiveness threshold used to value that spending is GBP 20,000 per QALY. The net health effect of a cycle is given below.

$$NH = Q_{in} - Q_{out} - \frac{C_r}{\lambda}$$

where $NH$ is the net health effect in QALYs, $Q_{in}$ is the QALYs gained from reinvestment, $Q_{out}$ is the QALYs lost from disinvestment, $C_r$ is the cost of running the cycle in GBP and $\lambda$ is the threshold in GBP per QALY.

The planned result of the first cycle follows.

$$NH = 14 - 4 - \frac{50000}{20000} = 14 - 4 - 2.5 = 7.5$$

where 14 and 4 are the planned QALYs gained and lost, 50,000 is the cycle cost and 20,000 is the threshold.

In the observe stage the group finds that only 75% of the planned release happened, because staff in the withdrawn service were not redeployed. Gains and losses scale with the money moved.

$$NH = 10.5 - 3 - 2.5 = 5.0$$

where 10.5 is 14 times 0.75, 3 is 4 times 0.75 and 2.5 is the cycle cost expressed in QALYs.

At the reflect stage the panel redesigns the staffing plan, and a second cycle costing GBP 30,000 releases the remaining 25%.

$$NH = 3.5 - 1 - \frac{30000}{20000} = 3.5 - 1 - 1.5 = 1.0$$

where 3.5 is 14 times 0.25, 1 is 4 times 0.25, 30,000 is the second cycle cost and 20,000 is the threshold.

The project ends 1.5 QALYs below its first plan, because the second cycle cost 1.5 QALYs of opportunity cost. The observe and reflect stages found a shortfall that a one-off report of the planned reallocation would have hidden.

Judging quality, evidence and generalisability

Meyer noted that generalisations from action research differ from those of conventional studies, so reports need rich contextual detail, participants' responses to the findings and the researcher's own values made explicit. She also argued that success should not be judged solely by the size of change achieved, since what was learnt can matter as much.

For health economists this creates a gap between the evidence action research produces and the evidence an appraisal committee usually wants. A randomised controlled trial fixes the intervention in advance, whereas an action research project changes the intervention between cycles, so its outcomes describe a moving target in one setting. Waterman and colleagues argued from the PRIDE project that action research is best suited to studying the diffusion of innovations where each new setting needs a high level of adaptation, which fits local implementation better than estimating a transferable treatment effect. Where change has to be attributed, an interrupted time series design gives a clearer basis than a simple before-and-after study.

Action research beside participatory research, PDSA cycles and audit

Participatory research is the wider family of approaches that involve the people studied as partners, and participatory action research is the strand that ties that partnership to repeated cycles of action. Baum and colleagues described participatory action research as paying careful attention to power, advocating that power be shared between researcher and researched until the researched become researchers.

Plan-do-study-act cycles, used in quality improvement, share the cyclical form. Taylor and colleagues traced the method to Shewhart and Deming and described it as iterative, prediction-based testing of change on a small scale using data over time. The emphasis differs, since a plan-do-study-act cycle is a tool for testing a specific change, while action research is a research approach with a participatory and democratic character that aims to add to knowledge as well as practice. Clinical audit and continuous quality improvement also rely on measurement and re-measurement.

Limitations and common misreadings

The first misreading is that any project with a feedback loop is action research. In a scoping review of 124 health care studies, Cordeiro and Soares found participation ranging from no real input into the research to democratic knowledge sharing, so the label alone does not show how far participants shaped the research.

The second is that the method guarantees change. Taylor and colleagues found that fewer than 20% (14 of 73) of the plan-do-study-act studies they reviewed fully documented a sequence of iterative cycles, a warning that cyclical methods are often reported loosely. Cornelissen and colleagues could not show that resources were actually reallocated, and Byskov and colleagues could not show effects on health.

Meyer also warned that participation can be threatening and that democratic practice is not always a feature of health care settings. Baum and colleagues added that participatory action research is time consuming and unpredictable and less likely to attract competitive funding. In economic terms facilitation time is a real cost, and leaving it out overstates the net benefit of the change.

Sources

• Meyer J. Qualitative research in health care: using qualitative methods in health related action research. BMJ 2000;320(7228):178-181. https://doi.org/10.1136/bmj.320.7228.178
• Baum F, MacDougall C, Smith D. Participatory action research. Journal of Epidemiology and Community Health 2006;60(10):854-857. https://doi.org/10.1136/jech.2004.028662
• Holmes RD, Steele J, Exley CE, Donaldson C. Managing resources in NHS dentistry: using health economics to inform commissioning decisions. BMC Health Services Research 2011;11:138. https://doi.org/10.1186/1472-6963-11-138
• Mitton C, Patten S, Waldner H, Donaldson C. Priority setting in health authorities: a novel approach to a historical activity. Social Science and Medicine 2003;57(9):1653-1663. https://doi.org/10.1016/s0277-9536(02)00549-x
• Patten S, Mitton C, Donaldson C. Using participatory action research to build a priority setting process in a Canadian Regional Health Authority. Social Science and Medicine 2006;63(5):1121-1134. https://doi.org/10.1016/j.socscimed.2006.01.033
• Peacock S, Mitton C, Bate A, McCoy B, Donaldson C. Overcoming barriers to priority setting using interdisciplinary methods. Health Policy 2009;92(2-3):124-132. https://doi.org/10.1016/j.healthpol.2009.02.006
• Cornelissen E, Mitton C, Davidson A, Reid C, Hole R, Visockas AM, Smith N. Determining and broadening the definition of impact from implementing a rational priority setting approach in a healthcare organization. Social Science and Medicine 2014;114:1-9. https://doi.org/10.1016/j.socscimed.2014.05.027
• Byskov J, Marchal B, Maluka S, et al. The accountability for reasonableness approach to guide priority setting in health systems within limited resources: findings from action research at district level in Kenya, Tanzania, and Zambia. Health Research Policy and Systems 2014;12:49. https://doi.org/10.1186/1478-4505-12-49
• Waterman H, Marshall M, Noble J, et al. The role of action research in the investigation and diffusion of innovations in health care: the PRIDE project. Qualitative Health Research 2007;17(3):373-381. https://doi.org/10.1177/1049732306298976
• Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Quality and Safety 2014;23(4):290-298. https://doi.org/10.1136/bmjqs-2013-001862
• Cordeiro L, Soares CB. Action research in the healthcare field: a scoping review. JBI Database of Systematic Reviews and Implementation Reports 2018;16(4):1003-1047. https://doi.org/10.11124/jbisrir-2016-003200