Performance Anxiety: thoughts from the 2014 CHSPR health policy conference
On February 25th the healthcare funding team attended the 26th annual Centre for Health Services and Policy Research (CHSPR) health policy conference held in Vancouver, BC. This year’s conference, titled “Performance Anxiety,” focused on the measurement of healthcare system performance; what is being done, how, and the impact of performance measurement on healthcare system change. Speakers represented researchers, clinicians, policy-makers and patient voices from across Canada, the UK and Australia. Among the speakers, there was a general agreement that performance measurement in Canada is a work in progress due to uncoordinated and unstructured reporting, limitations in access to timely data, and challenges on deciding what performance indicators or benchmarks to focus on to make a change in the quality of the healthcare system. Some other key themes that emerged during the conference were the audience for public reporting (for example, patients in Canada seem to be ambivalent, while patients in Australia are more engaged in performance results), the ethics surrounding performance reporting, the effect of data burden on both patients and policy-makers (how much is too much?), and how to improve performance indicators to better capture performance over time and space. In addition, panels discussed ways to effectively target public performance reports to evoke change in the healthcare system. Despite the uncertainties surrounding performance measurement in Canada, it’s clear from the dialogue during the conference that it plays a key role in measuring the potential improvement of the Canadian health system. There are many groups working to increase the effectiveness of performance measurement in Canadian healthcare, and it will definitely be a field to watch for new developments.
To access speaker slides from the 2014 CHSPR health policy conference, please click here.
New analysis on long-term care funding in Alberta
In September 2013 we released a report called “The Alberta Health Services Patient/Care-Based Funding Model for Long Term Care: A Review and Analysis.” This report evaluates long-term care funding in Alberta. Creating an environment that provides cost-efficient, accessible and high quality LTC is a key objective for Alberta Health Services (AHS). Analyses of weighted cost per day in 2009 revealed previous LTC funding mechanisms in Alberta were disconnected from resident’s needs and reforms were deemed necessary. The development and implementation of the Patient/Care-Based Funding (PCBF) represents a monumental multi-year undertaking by a large number of individuals across the healthcare system. In function, the PCBF model is a stand-alone outputbased allocation method which ‘splits the pie’ of total LTC funding among providers based on aligning the funding per resident with the intensity of caring for each resident, including the scope of expected services. This report was commissioned by AHS and identifies strengths and weaknesses of the LTC funding model and provides specific feedback to AHS regarding possible actions to mitigate actual or perceived weaknesses of the LTC funding model.
Reflections on the 2013 Canadian Association for Health Services and Policy Research conference
The Canadian Association for Health Services and Policy Research held their annual conference in Vancouver from May 28 to 31, 2013. The conference was well attended by the Healthcare Funding team, and we gave a presentation about our evaluation of the effects of introducing activity-based funding in BC and a poster about linking costs of care across different parts of the health system.
- Presentation: Activity-based Funding in BC: Changes in Activity? Liu G, Repin N, Crump T, Sutherland J.
- Poster: Linking Administrative Data Sources to Analyze Cost of Care over the Continuum. Chu S, Liu G, Crump T, Sutherland J.
We appreciated the chance to present our work and to receive feedback, and we were impressed by the breadth and quality of the conference plenary and breakout sessions. Our team of researchers, data analysts, and students liked the theme of the conference, Making Good on the Triple Aim. The following are some of our comments:
“Improving the patient experience of care (including quality and satisfaction), improving the health of populations and reducing the per capita cost of health care should be major goals of Canadian health care policy. Utilizing this framework helps me, as an analyst in healthcare research, to understand broadly the issues challenging the Canadian healthcare system. I am glad to be able to contribute to improving the Canadian healthcare system through my own involvement in research. Participants from all over the world were all keen in searching for the best fit policy for Canada. Particularly, I was inspired by the in high quality presentations from graduate students – the next generation of health policy experts in Canada.”
~Researcher/analyst
“I enjoyed the sub-plenary session, “Pursuing the Triple Aim: Measuring Progress, Performance and Outcomes.” This session provided a great general overview of population health, patient reported outcome measures and healthcare funding. Besides learning about different research topics and findings from concurrent and poster sessions, I gained poster presentation skills and obtained insightful feedback from participants by presenting a student poster at CAHSPR. Overall, CAHSPR has broadened my understanding of healthcare services and policy research.” ~Graduate student
“As my first conference in the field everything was new and exciting. I like the mixture of formats: plenary talks, panels and shorter talks with recent results.”
~Postdoctoral fellow
“The conference brought together a good mix of policy and evaluative research across health sectors and across Canadian jurisdictions. The enthusiasm of Canada’s health services researchers was evident throughout the conference. It was clear from the abstracts and presentations that research related to performance measurement, disparities in health and access to care and the integration of care is well developed in Canada. Research related to funding model design and evaluation seemed under-represented in the agenda compared to its relative importance. Given the fiscal situation of most provinces, funding is not likely to keep pace with the rapidly increasing demands and costs of care over at least the next five years. Research in the policy, economic, and service delivery domains that would help providers cope with this problem seemed absent from the conference.”
~Postdoctoral fellow
Our research team also presented work on patient-reported outcome measures.
Hope to see you at CAHSPR next year in Toronto!
How do we measure change if our data is wrong?
This site is about funding, it’s about how we fund healthcare, and about how we fund hospitals. It’s about how we measure and analyze the effect that different funding methods have on the healthcare so that, in the end, we can design better systems.
To do this, we analyze vast amounts of data. We often make use of hospital clinical administrative databases. These databases contain rich clinical and demographic data for every hospital discharge. We rely heavily on the diagnostic data captured in administrative data to adjust our comparisons for differences in the complexity of patients both over time and across hospitals. A crucial question for us and other health services researchers is, “how reliable are the data we use?”
In an article that looks at how diagnostic coding errors affect estimates of adverse event costs, Gavin Wardle and colleagues analyzed the results of a large reabstraction study performed in Ontario. Wardle reports that diagnostic coding disagreements are extensive. For example, the original and reabstracting coders agreed on only about half of important comorbidities and adverse events. Of importance to researchers interested in comparing hospitals, the rate of coding disagreement varied extensively by hospital.
Wardle’s study was of Ontario data, but BC’s data are subject to the same risks for coding error. Variation in the reliability of diagnostic data elements can be caused by:
- Differences in the rate of chart completion and intensity of documentation by clinicians.
- Different interpretation of coding standards established by the Canadian Institute for Health Information (CIHI).
- Willful violation of CIHI’s coding standards. The type and extent of error can depend on how the data are used for funding and performance measurement.
Ontario’s experience, and that of other jurisdictions, has been that some hospitals change their coding practice to improve their funding under models that rely on administrative data. As BC increases its use of administrative data for funding, researchers and policymakers will need to closely monitor the response of hospital coding practice.
What options are available to monitor the quality of hospital data in BC? Some jurisdictions rely on random chart auditing programs to ensure that hospitals do not miscode data to optimize funding. Since these programs must be routine and large scale to ensure the reliability of data, they are resource intensive. Another way to eliminate hospital level variation is to centralize the coding function and randomly assign charts to coders. This would remove hospital-level effects on coding and randomize coder-level effects across hospitals.
Since the accuracy of hospital performance measurement and some reimbursement depends on the quality of data, BC will have to increase its focus on data quality measurement and surveillance.
Hospitals and quality: Some initial thoughts on measurement and funding
We pay people to do things. It’s an underpinning of western civilization, which has evolved to a point where we pay people to do more of things we want.
This is the idea behind merit-based pay, fundamental to bureaucracies and a large part of business practices. For example, Mark Carney, the soon to be former Governor of the Bank of Canada, is going to be paid over $1.3 million to fix England’s financial system. This concept holds remarkably true right up until we begin talking about healthcare. In healthcare, we either do not pay people to do things better (often in public systems), or paying people to do things better doesn’t seem to work.
Paying hospitals for what they do is common elsewhere. Specifically, volume-based payment systems, like activity-based funding (ABF), are relatively common in the rest of the OECD nations. Currently, only BC and Ontario pay a portion of hospital funding based on the type and volume of activity the hospitals engage in. Ironically, in an article on reforming Medicare, Rick Mayes identifies that in Canada the problem is managing volume of care, in the US it is managing expenditures (without volume control). Neither ABF nor global budgets appears to be ideal.
A new potential payment solution to healthcare woes (discussed in a previous post) are bundled payments (along with global or capitated payments). Aside from controlling the volume of care (and thus expenditures), both Canada and the US have policy goals that include increasing the quality of care provided.
Mayes indicates that moving from paying for healthcare based on amount to paying for healthcare based on how good it is (i.e. quality) will require a substantial investment in research to produce valid measures of quality. After all, how do you determine which hospital is doing better? What does better mean? Is it the same for all hospitals? These are things we remain unclear on.
A fair bit of research has been carried out in the US to try and tackle this question. How do we measure hospital quality? A new section of our website summarizes much of this research. It describes the three different types of quality measures that are commonly used: process measures, outcome measures and structural measures. It also lays out the strengths and weaknesses of each measure. We provide examples of some of the organizations that are collecting indicators, both in the US and internationally. Finally we being to provide examples of the ways that we can link between measuring hospital quality and funding hospitals based on those measures. We also explore some of the potential negatives in linking quality to funding, which, as a policy option, is one that remains controversial at times.
Mayes R. Moving (realistically) from volume-based to value-based health care payment in the USA: starting with medicare payment policy. Journal of health services research & policy. 2011:16(4):249–51.
Evidence and policy: How do we make a difference?
One of the most fulfilling aspects of our research is its ability to impact policy. Yet getting on the radar of decision-makers can often be a challenge. According to Les Levin, with Health Quality Ontario, the track record for traction between evidence and policy is weak. We can also see that the relationship between evidence and policy is fluid, and subject to change based on politics, governments, public opinion and a host of other factors. How do we as researchers and subject experts, get our research and our voices heard by the right people?
The National Institute for Health and Clinical Excellence (NICE) in the U.K. was established to provide evidence for the UK National Health Service (NHS). A recent article discusses how NICE recognizes that decisions are informed by many factors and how NICE tries to influence decisions beyond a straight-forward presentation of the evidence. Evidence about effectiveness and, to a lesser extent cost-effectiveness, are ones that we are most interested in as researchers, but often the values of policy-makers play an important (if not the most important) role in decision-making. Some of the ways that NICE sees values influencing decision-making is in judgements about suitability of outcome measures, weighting of aspects of health, public v. private expenditures (and in Canada 30% of health expenditures are private so we shouldn’t ignore private spending), the acceptability of processes of care delivery and the affordability of different treatment options (both for individuals and for the state).
One of the ways that NICE attempts to resolve issues that arise from differences in values and value judgments is through a deliberative decision-making process which includes a Citizen’s Council as a form of direct democracy. This is not a foreign concept to BCers, many of which will remember the Citizen’s Assembly on electoral reform.
Deliberative dialogues are also used by the McMaster Health Forum, which convenes a broad cross-section of policy-makers, stakeholders and researchers that work through a specific health issue. According to Michael Wilson, assistant director at McMaster Health Forum, these deliberative dialogues can be uniquely supportive of evidence-informed decision-making by bringing together the data and research evidence with the knowledge, views and experiences of those who will be affected by the specific health issue.
While a full blown Citizen’s Council or Assembly is clearly beyond the scope of most researchers (including ourselves), the concept of a process of deliberative dialogue between researchers and academics and policy- and decision-makers is a useful one to consider for knowledge translation. How do we get decision-makers engaged in our research? Not only can we tell them about the evidence uncovered by our research, but we can also engage in discussions around values and value judgements. Learning what values are important to decision-makers will allow us to tailor our message and presentation of our evidence in a way that is more likely to resonate with them.
This post is based on ideas inspired by the Healthcare Quarterly volume 15 special issue “Toward Performance and Quality,” December 2012
Reflections on bundled payments in Canada
What can we learn from US evidence about bundled payments?
Bundled payments are one of the newest funding mechanisms being piloted in countries like the US. Bundled payments are when a single payment is provided for an acute care episode that covers all costs, including acute and post-acute care, for a patient’s episode of care – following their trajectory between setting, different providers and over time. For more information see Bundled Payments.
The feasibility of studying the trajectory of care for patients is also something that is being explored in Canada, particularly in Ontario, where linking healthcare utilization over time opens up the opportunity to a wider set of longitudinally-based quality and appropriateness measures, and fits within the context of Ontario’s Quality-based Pricing (QBP) initiative. QBPs are identified clusters of patients with clinically related diagnoses or treatments where the evidence shows that there are opportunities for improved outcomes and cost savings. Healthcare providers will be reimbursed for the types and quantities of patients they treat, using prices based on the quality of care delivered.
Sood et al (2011) looked at the design of bundled payments in the US National Pilot Program on Payment Bundling. This article examines design considerations for the bundled payment pilot project and determines that, according to Medicare data, hip fractures and joint replacements are good conditions to include, due to their potential to drive cost savings for Medicare. The authors conclude that longer episodes of care are more suitable for bundled payments since longer episodes cover most of the costs and readmissions – but add little in the way of financial risk to US providers.
This article makes us think about the future applicability of bundled payments for funding episodes of care in Canada, and what some of the design differences would be. While costs of care and the overall sustainability of the healthcare system is a consideration in Canada, the more pressing reason to adopt bundled payments is related to quality, as bundles can address gaps in care between the acute care and post-acute care settings. By providing incentives to coordinate care across settings, it is hoped that better outcomes are achieved for patients.
The potential for bundled payments to improve efficiency and effectiveness in the US Medicare system is fairly clear. However, before provinces can even consider developing longitudinal measures of quality and effectiveness for patients’ episodes of care, the underlying data has to exist and be linkable. In some provinces, emergency department data are not available, in others, long-term care, home care or assisted living data are not available in a form that describes the nature of services provided, patient functional status or the cost of services. Before complex issues, such as who should receive a bundled payment, are tackled, data issues must be acted upon.
One of the most important considerations for implementing bundled payments in Canada would be choosing who is in charge of receiving and doling out the payment. In the US, market mechanisms provide a relatively straightforward method to sort this out. However, in Canada, such structures do not exist or are unfamiliar to healthcare providers. Should the payment go to the hospital? What are the potential system effects of giving hospitals in Canada control of bundled payments? Will this help or hinder the overall agenda of reform to move away from expensive hospital treatments and towards more primary care? These are all important considerations with few easy answers raised by the intriguing possibility of enacting bundled payments in Canada.
Sood N, Huckfeldt PJ, Escarce JJ, Grabowski DC, Newhouse JP. Medicare’s bundled payment pilot for acute and postacute care: analysis and recommendations on where to begin. Health affairs (Project Hope). 2011 Sep 7;30(9):1708–17.
