Measuring Health System Performance
Course Summary:

This course aims to present a framework to discuss the opportunities and challenges with performance measurement in health care, examine the various dimensions and levels of health system performance, identify the measurement instruments and analytic tools needed, and examine the implications of these issues for policy makers and regulators. Lectures generally focus on measuring health system performance in high-income countries but draw on the experience of other countries where relevant.

The course also covers a number of related analytical tools such as cost benefit analysis, decision analysis, and sensitivity analysis. The main goal is for participants to understand the strengths and limitations of these methods and how to apply them to a broad range of health issues.

Course Objectives:

  • Understand the principles of performance measurement
  • Appreciate the challenges, approaches, and opportunities in performance measurement in four dimensions: population health, patient outcomes, equity, quality and appropriateness of care, and productivity
  • Understand the methodological issues facing performance measurement relating to risk adjustment, developing composite measures, and measuring attribution and causality
  • Identify key issues relevant to policy makers relating to: developing targets and reporting on progress to the public, and developing incentives to improve performance
  • Understand the strengths and limitations of cost effectiveness analysis and other decisionmaking tools in a variety of contexts—in other words, to understand what these tools can and cannot answer.

Course Outline

Introductions; What is a Health System?

This session aims to orientate participants into the course, and provides an introduction to thinking about health systems, how they are constructed and function in particular contexts, their value bases and their place in history and society.

Topics and activities

  • Introductions and course overview: Participatory activity
  • What is a health system and why is it important?:
  • The Political economy of African health systems: Lecture; Podcast on a Nigerian example; ‘Gap-minder’ review and task

Frameworks for describing and analysing health systems and policy

This session fleshes out the discussion of what a health system is and why it is important in a society. Concepts, measures and frameworks for thinking about systems are introduced and the integrative nature of health systems is emphasised. Students then apply these to their own country’s health system. The session also introduces reflection on teamwork skills. Topics and activities : 1. Students’ discuss own countries’ health systems histories and make comparisons between them: Group discussion and feedback; 2. Understanding systems: Flashmob Game 3. Ideas and frameworks for thinking about health systems and policy: Video and lecture; group discussion. 4. Application of frameworks to specific country scenarios: Thinking pairs to prepare for homework 5. Readings and discussion: what makes for effective group work?

Understanding the Thai experience of health system development

This session deepens the ideas about what health systems are: that they are complex and integrative, and that health system development is a long-term task, influenced by broader political, economic and social forces, and requiring persistence, vision and adaptability. It introduces the notion of system effectiveness, and relevant indicators. It uses a case study of Thailand to develop ideas about understanding and strengthening health systems, including the interactions between system hardware and software. It encourages personal reflection on teamwork skills. Topics and activities : 1. Review reading of Thai case study: Individual reading 2. Thai case study as an example of health systems strengthening: Discussion in group, class discussion 3. Lecture: wrap-up of the Thai experience

Whole system change – Primary health Care (PHC) and Universal Health Coverage (UHC)

This session develops further and consolidates ideas about what systems are and how to strengthen them. It focuses on primary health care and universal health coverage as changes affecting all aspects of the health system. It then asks students to apply lessons from the Thai case study as well as UHC and PHC to think about and articulate concise ideas about strengthening key elements of their own country’s health system by writing a policy briefing note as an exercise in communication. Topics and activities: 1. PHC and UHC as ‘whole systems’ change’: lecture and discussion. 2. Lessons from Thai experience and UHC/PHC debate for own country health systems: individual work, thinking pairs, plenary discussion 3. What is required for developing a policy briefing?: Lecture, Input/handout 4. Develop a policy brief on an aspect of Thai experience for own health system: preparatory brainstorming exercise

Recognising agents in health systems

This session focuses on the central role of people and their roles in health systems. It focuses on the importance of people’s values and mind sets, agency and power, as drivers of their behaviour and that all these impact on how health systems function. Topics and activities: 1. Readings about people in systems: Group discussion of range of papers; plenary discussion 2. Systems dynamics - getting to people in the system: Squaring the circle exercise 3. What drives health system actors?: Lecture and plenary discussion 4. Overview of stakeholder analysis: Brief input

Exploring power, agency and mind sets

The focus of this session is on applying the ideas from session 5 and thinking about the roles of health system actors in relation to the process of system change. It looks at actors’ values, beliefs, mind sets, and power relations, and considers how behaviour and attitudes influence system performance. It then considers the significance of the different views actors have of health systems and the world, for health systems development, reflecting also on the interaction between system software and hardware. Topics and activities: 1. Stakeholder analysis in practice: Group exercise using actor maps based on budgeting scenario; de-briefing 2. Different paradigms of knowledge: Read and re-discuss papers on people in systems, hardware-software etc. 3. Re-thinking about actors in own contexts, their location, power and mind sets: Thinking pairs.

Managing change in health systems

The main aim of this session is to introduce ideas for understanding and managing actors in health system change. Using a range of analytic frameworks it will focus on complexity and agency, but also, the understanding that health systems are knowable and changeable. Students will begin to develop strategies which support intervention and change in systems. Topics and activities: 1. Managing agents and change: Discussion of readings; lecture (commitment planning; small wins, sense-making) 2. Managing agents and commitment planning: Group task developing strategies for managing change using SHA scenario; feedback discussion

Intervening in health systems

This session continues the in-depth look at case studies and what they reveal about health policy and systems in practice, calling for the ability to demonstrate an understanding of health systems’ contribution to public value and societal development. The session will also require presentation and team work skills, and allow reflection on both. The group presentations will be marked by students in class (as a peer assessment), and the marks will become part of the portfolio and the formative assessment marks. Topics and activities: 1. Presentation of case studies: Group presentations and feedback 2. Cross case review of experiences; implications for action in and across cases: Plenary discussion 3. Reflection on teamwork and leadership in relation to the group task: Plenary discussion

Health system complexity and change

The final session aims to wrap up the course by deepening the understanding of complex adaptive systems and linking this understanding to the different concepts, frameworks and ideas introduced in the course. The session will place particular emphasis on the analytic skills of students in being able to apply new concepts in different contexts, and on communicating ideas and understandings to an audience. Topics and activities: 1. Discussion of papers applying complex adaptive systems (CAS) thinking and concepts to health systems: Student-led journal club

2. Application of CAS concepts to case studies discussed in sessions 8 and 9: Group discussion. 3. Linking CAS thinking to core concepts discussed in the course: Wrap-up lecture and discussion 4. Course evaluation: Group discussion; individual written task