Evidence in Public Health and Health Impact Assessment (The Role of Panayotov Matrix) more

FOR SUSTAINABLE AND PROSPEROUS SOCIETY HEALTH ECONOMICS Evidence in Public Health and Health Impact Assessment (The Role of Panayotov Matrix) Jordan Panayotov Jordan Panayotov Jordan Panayotov is Founder and Director of the Independent Centre for Analysis and Research of Economies in Melbourne, Australia. He was born in December 1961 in Sofia, Bulgaria. In April 1984 he got a degree in Economics, Labour Economics, from The University of National & World Economy Sofia. In his thesis Panayotov pays special attention to efficiency of labour in so called non-material sphere: education, health, arts, science. During 1984 – 1991 he had different positions at Medical Academy, Sofia. From May 1991 till January 2002 Panayotov was Managing Director of a company, exclusive distributor for Bulgaria of several world-leading producers of medical equipment and consumables. He gained extensive inside experience for both: the demand and the supply side in health care sector. In 1992 Panayotov initiated patients’ participation in decision-making process, which resulted in maximizing health outcomes from the available resources in renal treatment in Bulgaria. In 2004 Panayotov received degree of Master of Public Health, Health Economics, from The University of Melbourne and founded the Independent Centre for Analysis and Research of Economies. Panayotov works for improving efficiency in relation to health. His interests are in decision-making process and priority setting, with particular attention to power asymmetry influencing the outcomes of this process. Panayotov has developed model for evaluation of public health policies, programs and interventions. The model analyses the correlation between average health status and health inequalities in its dynamic, being constantly affected by the implementation of different policies. The model provides proper theoretical framework for health impact assessment of policies, programs and interventions in other spheres of the economy, as well as for addressing social determinants of health. It has been presented successfully on several international forums. Evidence in Public Health and Health Impact Assessment (The Role of Panayotov Matrix) Jordan Panayotov Health Economics Department Independent Centre for Analysis and Research of Economies Melbourne, Australia Melbourne, 01.02.2009 This work is protected by copyright law. Except as specifically permitted in writing, no portion of this work may be distributed or reproduced by any means, or in any form, without prior written permission. Copyright © 2004 – 2009 Jordan Panayotov. All rights reserved. Suggested Citation: Panayotov J., Evidence in Public Health and Health Impact Assessment, ICARE 2009 For further information, or to obtain appropriate permission, please visit www.icare.biz or contact permissions@icare.biz Content Foreword for the open access release of the paper, 01.02.09 ……………………………..i Abstract ………………………………………………………………………...ii Introduction ……………………………………………………………………1 Implicit and Explicit Decision Making………...…………………………...2 Decision Making in Relation to Health…………………………………….4 Formal Evaluation……………………………………………………………..7 Evaluation of personal services applied to individuals…………………………….7 Evaluation of complex interventions applied to populations…………………..…9 Evidence in Public Health………………………………………………….13 Replicability of the Evidence………………………………………………16 Usefulness of Average Data for Informing Decision Makers on Choices for Resource Allocation in Public Health ………………..17 Implications for Researchers, Decision Makers and Practitioners…19 Conclusion …………………………………………………………………...20 Appendix ……………………………………………………..……………….21 Notes …………………………………………………………………………..22 References ……………………………………………………………………23 Foreword to the open access release of the paper, 01.02.2009 Evidence appears to be one of the biggest problems for making informed decisions in public health and health impact assessment. Researchers, decision makers and practitioners are puzzled: Why identical interventions implemented to populations achieve different results to different groups of this population? Why replicating a successful intervention from somewhere delivers poor results in many other cases, or does not “work” at all, or is even counterproductive in some cases? Why randomised control trials, which provide reliable evidence for interventions applied on individuals, stumble when interventions are implemented to populations? What is the solution? This paper offers a solution based on Panayotov Matrix – a model with universal explanations and predictions, which is useful for appraisal of any interventions implemented to populations, no matter whether improving the health of the population is primary objective ( in public health ) or not ( in health impact assessment ). Most of the content of this paper has been presented on various international forums during 2004 – 2008. For more details please visit www.icare.biz/resources.html For five years now I’m saying that evidence in public health is relative and depends on the distribution of the benefit from an intervention at local level. In order to comprehend properly the content of this paper it is essential for the readers to be familiar with Panayotov Matrix, which is available at www.icare.biz/health.html I’m saying this, because everyone was asking me the same question: “What to do in (my) specific case?”, expecting to get detailed, concrete answer, which he/she will use as a “silver bullet” to solve the problem. However, as they were not familiar with the concept, they were not getting it, when I gave them the answer: “Avoid double losers.” So, if you want a solution for evidence in public health and health impact assessment, take your time to read thoroughly the paper “Public Health and Average Health Status: Do Health Inequalities Matter?” and then continue with this one here. No funds from any governmental, NGO, public or private institution or entity, or person were ever received, as well as no any other form of support has ever been received neither during the process of creating the model, nor for presenting it on any occasion. Melbourne, 1st of February 2009 Jordan Panayotov i Evidence in Public Health and Health Impact Assessment (The Role of Panayotov Matrix) Abstract When making decisions for allocating limited resources people use evidence in order to have certainty about achieving intended result in relation to predetermined goal. Different interventions to choose between are nothing more than means for getting the desired outcome. It should be clear to everyone that, although that a population is sum of individuals, achieving highest attainable health for an individual and for whole population is not the same thing. While the evidence that an intervention achieves the desired outcome for an individual can be unambiguous and usually can be replicated at any local context, the evidence for interventions implemented to populations is often weak, equivocal, inconclusive and even controversial. Why identical interventions implemented to populations achieve different results to different groups of this population? Why replicating successful intervention delivers poor result in other cases? Since public health is about populations, it is about distribution of the benefit within these populations, therefore ultimately it’s about who-gets-what from an intervention. Distribution of the benefit should not be confused with distribution of the population, which is normal distribution with bell-shape. The former impacts the shape of the later. Average data alone has very limited informative value for decision makers in order to make the right choices for interventions implemented to whole populations. For any intervention implemented to populations the distribution of the benefit within the population is the most important factor, which affects both average health status and health inequalities. There are eight different possible combinations of distribution of the benefit from an intervention defined in Panayotov Matrix, which lead to very different outcomes for average health status and health inequalities. Therefore, no matter whether improving the health is primary objective (in public health) or not (in health impact assessment), the evidence that an intervention “works” becomes relative, depending on replicating the same combination of distribution of the benefit. Key w ords: Evidence-Based Policy, Health Impact Assessment, Health Inequa lities, Social Determinants of Health, Average Health Status ii Evidence in Public Health and Health Impact Assessment (The Role of Panayotov Matrix) Introduction Resources are and always will be limited and less the needs, therefore choices are inevitable. When making decisions, people have different options – alternatives. Even when there is only one option, there is alternative – doing nothing. Then, based on their experience, knowledge, resources, etc., people make their choice, the outcome of which will be best – they believe – for achieving their objectives. Some decisions are rather spontaneous – people do not think about it. They just like/want the result, which other people might, or might not like/want. If asked on what basis such decision was made, people often can not give logical explanation. Furthermore, the same people might have different decision next time, when they face similar situation. Other decisions come after “weighting” of pros and cons according to predetermined goal, i.e. some form of evaluation is required. In such case people are, or at least try to be rational – they want to achieve maximum outcome with minimal resources required. People do not want to make mistakes when making decisions for allocating their limited resources, i.e. they don’t want to make wrong choices. Therefore they use some kind of evidence in order to justify the correctness of their choice. Usually the same evidence can be used from other people in similar situation. If people want to get the best outcome in relation to their predetermined goal, they rely on the best practice, or the best available evidence. During the last decade there is growing number of papers about the evidence in public health. While with few exceptions (1) there is consistent improvement for all major health indicators after World War II, health inequalities persist not only in relatively poorer economies, but even “all developed countries are faced substantial inequalities in health within their populations”, according to Erasmus MC (2007). Addressing health inequalities becomes a major concern for many decision makers, who look for appropriate evidence in order to make proper choices of interventions. 1 However, as Nutbeam (2004), Kelly et al., (2006) and others note most of the available papers describe empirical observations, while there are very little explaining what should be done in practice in order to reduce health inequalities. Kelly et al., (2004) note that in public health “the definition and type of evidence is controversial” and “change based on evidence is not straightforward”. In absence of proper theoretical framework based on critical realism (if A than always B), which identifies and explains the generative mechanisms for health inequalities, Connelly (2001, 2005), Kelly et al. (2006), it is difficult for decision makers to determine unequivocally “what works” in order to improve health and quality of life of whole populations. Choosing the best alternative – value for money – is even more difficult. Implicit and Explicit Decision Making When making personal choices, decisions are made usually implicitly. For example, I’m not expected to explain to anyone why I have chosen to buy smoked salmon from the local supermarket for dinner. This is so, because the outcome from this decision concerns me only, and furthermore I have spent my own money for the fish, so it is nobody else’s business what shall I buy for my own dinner. Implicit decision making has the following advantages for the decision maker: firstly he/she should not explain any reasons for the decisions made, and secondly the whole process is much quicker, as there is no formal evaluation of the available alternatives. It will be different story, if I have to make decision for spending someone else’s money and/or the outcome from this decision concerns other people. I have to evaluate pros and cons for different alternatives, and I have to explain explicitly that the proposed decision is the best (from the available alternatives) in relation to predetermined goal from certain perspective, which is someone else’s, not mine. However, if I have had the necessary power, I could have made an implicit decision concerning others (even when involving their own money) and impose it onto them. When consequences from the decisions made concern other people, and/or involve their money or other resources, implicit decision making is about exercising power. 2 For Max Weber power is the ability to impose someone’s will onto other individuals’ behaviour. Persons or organizations, which have power, are able to impose their will onto others. The greater the power one has, the more individuals onto which his/her will can be imposed, and/or the greater the impact onto these other individuals (2). For example, in the army, the higher one officer’s rank, the greater the power he has and the more individuals he commands. Decision making in the army is mainly implicit, as there is a fundamental principle that the decisions of the superiors are not to be discussed or commented, but to be followed. The apogee of implicit decision making concerning other people is represented by the absolute monarchy. French king Luis XIV, known as “Le Roi Soleil” – Sun King, is remembered with the phrase: “L’Etat, c’est moi”, “The state – that’s me“. Whatever he decides becomes imperative for everyone and nobody dares to question his decisions, or to ask for reasons, or for evaluation of available alternative options in order to make sure that the right choices have been made. What is best for him is considered to be best for the state. By contrast, in democratic societies, like Athens with Pericles, it is expected that any decisions concerning other members of the society will be explicit. People discuss pros and cons for available alternatives and then agree on what is best for them. This is more valid for all decisions regarding publicly financed policies, programs, projects and interventions, like for example health spending in modern democracies. In all OECD countries (3) public spending on health dominates (Table 1). Decisions should be explicit not only because they concern practically everyone in the society, but also because the funding involved is mainly public – tax payers’. Since economic efficiency requires that limited available resources should be allocated in line with the value people place on outputs, Hurley (2000), decision makers should always take into account preferences of tax payers, who finance health collectively, when making choices for publicly financed policies, programs, projects and interventions. In fact taking into account preferences of tax payers, who finance health collectively, is mandatory for getting right five of Donabedian’s “seven pillars of quality” (4). (See Appendix for Table 1). 3 Decision Making in Relation to Health For decision making in relation to health there are three types of decisions, depending on who is concerned by the outcome and who is financing the choice: I) Decisions of individuals for their personal choices, including decisions of parents for their children. The outcomes from these decisions concern mainly the individual and whatever has been chosen is mainly paid by the individual (or his/her family). People make their own decisions for allocating their own limited resources in order to maximize their own utility. By the free choices they make, these people reveal their preferences. Theoretical foundation applied and valid is the Classical welfarism. These decisions will not be considered in this paper. II) Decisions for personal services applied to individuals. These are related mainly in the provision of health care. The outcomes concern mainly the individual and the services provided are paid collectively: either by taxpayers and/or insurer, or mixed with different percent paid by the individual receiving the service. While preferences of the recipients of these services are often considered, and preferences of the insurer are taken into account with clauses such as: “yes, but” and “not, if”, it is not clear who takes into account preferences of taxpayers, who finance collectively the provision of these services. Neither is clear on what basis the decisions are made, i.e. what is it to be maximized: individual’s utility of the recipient, income of the providers, or utility of those who finance collectively the provision of these services. III) Decisions for complex interventions applied to whole populations. The outcome concerns populations and these are financed collectively, mainly by the taxpayers. There are two subgroups, depending on the primary objective for those policies, programs, projects and interventions (term interventions will be used for all of them): III.1) Interventions with primary objective improving health of whole populations. III.2) Interventions in other spheres of the economy (transport, education, etc.) with primary objective different than health, however with impact on health of populations. 4 Decision making for personal services applied to individuals has been traditionally implicit for centuries. Physicians did what they believed was in the best interest for their patients, without any explanations to anyone for the decisions they have made, or without any evaluation of alternatives, may be because in most cases the only alternative was doing nothing. The patient, or his family did pay 100 % of the fee for the personal services provided, and other members of the society had nothing to do with it, since one’s health was considered only his own business and responsibility. Implicit decision making regarding personal services applied to individuals remained unchanged, when during beginning of 20th century some people voluntarily participated in “friendly societies”, which were paying for the personal services provided by physicians to their members. Even after World War II, when many countries have introduced publicly financed National Health Insurance schemes, most decisions in relation to health, including these for public health interventions, i.e. interventions applied to whole populations, were still mainly implicit. It was only after a rapid increase of health care costs during late 1970s and early 1980s, when public calls for more explicit decision making in provision of health care (predominantly publicly financed personal services applied to individuals) have brought into life Diagnostic Related Groups (DRG) and Casemix (5). As health care costs consistently were growing faster than inflation, by late 1980s and beginning of 1990s the share of health care in GDP did double in many developed countries. One major reason for this was an increased demand for health care services, i.e. personal services applied to individuals. There are many factors for this increased demand. Aging of the population, being an increase of absolute number and the percentage of older people in the society, is associated with an increased demand for health care services. New technologies available now also are associated with an increased demand for health care services. Increased income of the people, together with increased information they have today, result in an increase of their expectations for getting better health care and being treated with the latest technologies available. Other factors are terrorism 5 and new infections (SARS, Bird Flu) as well as reemerging of some “old” infections, due to drug resistant bacteria. Climate change with corresponding greater frequency and magnitude of natural disasters affect adversely human health, leading to a disproportionate (by geographical area and socio-economic status) surge in the demand for health care services. Asymmetric information together with asymmetric power in health care are premise for a phenomenon called supplier induced demand (SID). SID exists when suppliers, in order to increase their income, create demand for provision of unnecessary goods and services for the consumer, or excessive quantity of necessary goods and services, i.e. SID is consumers’ over servicing (6). It is difficult to determine to what extend the demand for health care services has increased as a result of the above mentioned factors and their interaction. This paper does not pretend to make analysis of the drivers for the increased demand for health care services, neither for constantly growing related costs. However, we should be able to distinguish between objective factors (like natural disasters) and subjective factors (like SID) for the growth of the demand for health care services. What is important here is that this increased demand for health care services is met by budget constraints. Therefore the need for rationing of scarce resources is inevitable. Choices have to be made in order to achieve best outcomes with the limited resources available. While rationing in relation to individual utilities derived from goods and services without public financing or subsidy is based on ability to pay and the decisions are from type I, rationing in relation to utilities derived from collectively financed goods and services – decisions from type II and type III – depends on legitimate processes. The collective financing by taxpayers imply that rationing in relation to society’s (public) utilities is everybody’s business, thus it should be clear and explicit. Hindle & Kerridge (1993) note that “Rationing in secret is less likely to find the best answer and is ethically indefensible”. Cromwell & Halsall (1995) point out that “Unless rationing is undertaken explicitly, it is not known whether it is being administered in an equitable or efficient manner”. 6 Formal Evaluation Any explicit decision making requires formal evaluation of the available alternatives. This requires identification, measurement and valuation of benefits (outcomes) and costs, and comparing the benefits (outcomes) with resources spent for them (costs). Both benefits and costs can be direct, or indirect. There are different levels at which benefits and costs can accrue: individual, family, employer, community, and society. In most cases benefits and costs accrue in different points in time with variable lag. Proper evaluation explicitly states which benefits and which costs will be compared, when these accrue, to whom, and what are implications to different parties involved. Evaluation of personal services applied to individuals With the development of the medicine it became possible different interventions to lead to the same (or very similar) outcome, thus people, financing collectively the provision of health services, want to be sure that the right decisions (choices) are made. For example, gallbladder stones can be treated with drugs, can be removed by conventional open surgery, or by laparoscopic surgery, or can be treated with lithotripter. All four interventions are quite different and have their pros and cons, therefore the decision made in a specific case (individual) is based on the best available evidence about “what works” best in similar (to this individual) cases. This way “evidence-based medicine” (EBM) was borne. It has been defined quite recently – in 1996 – by Sackett et al. (1996) as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”. With accumulation of evidence about “what works” in different cases, systematic reviews provide reliable answers, which facilitate making the right choices for interventions applied on individuals. Systematic reviews, being synthesis of primary research, use selection criteria in order to include or not specific evidence (study). This has led to creating a hierarchy of evidence, putting randomized control trials (RCT) on the top. RCT are considered to be the golden standard for EBM, because it is the best tool for detecting and evaluating a change, resulting from interventions 7 applied on individuals, i.e. RCT answers the questions: does the intervention “work”, to whom, and to what extend. Consequently, systematic reviews based on evidence from RCT are useful and helpful to find out “what works” regarding interventions applied on individuals. Those systematic reviews are also able to explain “how it works” and “why it works” (or doesn’t work) on individuals, thus provide guidelines for choosing the best intervention for individuals in a specific case. In most cases this evidence can be used at any local context on “one-on-one” basis. In other words, the intervention can be replicated anywhere without any modification or adjustment, as the local context has little or no impact on the outcome. However, RCT can not answer questions about cost-effectiveness of interventions. Based on EBM, which provides answers to questions such as: what works, to whom and to what extend, there are well developed techniques for economic evaluation of personal services applied to individuals. Health Technology Assessment (HTA) is a process of systematic review of existing evidence about: 1) medical technologies, including: a) single-use products (medicines, consumables and other disposable goods), i.e. consumer goods (the whole its value is finally consumed) ; b) multiple-use products (equipment, devices and instruments), i.e. capital goods (part of its value – depreciation – is finally consumed); c) single-use procedures (examination, consultation, operation), i.e. services (these are intangible consumer goods); as well as 2) the way the provision of these medical technologies is organized in the society. HTA compares the outcomes of different interventions in order to determine which one is better than the alternatives in achieving predetermined goal. However, the answers are not unequivocal when the local context (including: organization of the provision of different alternatives; costs incurred to individuals, family, employers, and society; and not least cultural specificities) is considered in the comparison. While the evidence about clinical efficacy of an intervention can be unambiguous and in most cases is not affected by the local context, evidence about effectiveness and especially about cost-effectiveness can vary, depending on the local context. 8 Since the opportunity cost – benefit forgone from alternative allocation of limited resources – is relative and depends on local context and the perspective, the best intervention in one local context may well not be the best in different local context. If, from a number of interventions achieving the same outcome, intervention A is considered to be cost-effective for United Kingdom, it does not mean at all that the same intervention will be also cost-effective for Bulgaria, or Malawi, or United States. Therefore, when comparing effectiveness and especially cost-effectiveness, HTA should explicitly state not only achieved outcomes, but also what is the local context in terms of: organization of services provided, associated costs considered, and last but not least – cultural specificities. Evaluation of complex interventions applied to populations Constantly growing costs for health care met by budget constraints have led to analyzing factors which influence both outcomes and costs. In order to achieve best outcomes with limited available resources people have started to go upstream. With acknowledging that prevention is better than treatment, attention is paid to health promotion, as a major pillar of the new public health, defined by Nutbeam (1998) as social and political concept aiming at improving health, prolonging life and improving the quality of life among whole populations through health promotion, disease prevention and other forms of health intervention. However, when making choices for interventions applied to populations, decision makers should be aware of the fact that achieving highest attainable health for an individual and for whole population is not the same thing, Panayotov (2004 – 2008). While personal services are designed to maximize the benefit for the individual receiving these personal services, public health interventions aim to maximize the benefit for whole populations. Since any allocation of limited resources leads to declining some claims of the recipients (i.e. either all claims of one of the recipients, or some claims of both recipients are declined (7)), concrete decisions, regarding which claims of which recipient will be declined in a specific case in order to maximize the outcome/benefit for whole populations, depend on local context and unique conditions, where the policy, program or intervention will be implemented, Panayotov (2008). 9 HTA – with its arsenal of well developed techniques for evaluating personal services applied to individuals – is not up to the task for proper evaluation of interventions implemented to whole populations. For variety of reasons interventions implemented to populations achieve different benefit to different groups of that population. Therefore a proper evaluation should consider the distribution of the benefit among the population where these interventions are implemented. Currently HTA examines benefits and costs, as well as time horizons when these accrue, from interventions applied to individuals, i.e. HTA does not analyze the distribution of the benefit among population. In other words, HTA examines the benefit on vector OA, or respectively OB alone, while proper evaluation of interventions applied to whole populations should consider the combined benefit on both vectors, i.e. the sum of OAi and OBi. Consequently, by looking for maximum benefit for individuals from an intervention (i.e. OA or OB alone), RCTs in their current form are not useful for proper evaluation of a change occurring from interventions applied to populations (i.e. OAi + OBi). Graph 1. Conflict of Interests When Allocating Resources B S1 B1 B2 S2 O A1 A2 A AB – Possible efficient allocations of given resources OAi – Benefit for person A (max = OA, when allocation is in point A) OBi – Benefit for person B (max = OB, when allocation is in point B) Si – Allocation of resources providing benefit OAi + OBi S 1 – Allocation of resources providing benefit OA 1 + OB 1 S 2 – Allocation of resources providing benefit OA 2 + OB 2 10 In relation to health, as mentioned earlier, interventions applied to populations have two subgroups depending on the primary objective for these interventions: III.1) Interventions with primary objective improving health of whole populations. Health Program Evaluation (HPE) is used in order to determine: a) whether an intervention “works”, i.e. examines its efficacy; and (not always) b) the extend to which objectives are achieved, i.e. examines its effectiveness. As already mentioned, evidence about effectiveness for interventions applied to individuals is not always unequivocal, depending on the local context. When there are interventions applied to populations, the local context has one important extra dimension – distribution of the benefit within the population (Graph 1). However, HPE does not address this aspect properly, if at all. If it is not clear who-gets-what from an intervention applied to populations, even conclusions about its efficacy, i.e. does it “work”, can be shallow or misleading. Different combinations of previous and new winners and losers in specific case cause variations in the outcomes, therefore evidence about efficacy of interventions applied to populations is not unequivocal, and evidence about effectiveness is even more ambiguous. Many HPEs fail in first steps of any evaluation – identification and measurement of the benefit. Then, being unable to explain why and how identical interventions achieve very different outcome to different groups of that population and/or when applied to different populations, HPE struggles to provide reliable evidence which can be replicated. Often the term generalisability is used instead with rather broad directions for achieving intended outcome with no guarantee for success, still without providing any general principles with universal validity. Beside having problems with the evidence (even for efficacy), HPE can not judge for cost-effectiveness, neither for allocative efficiency – value for money – in relation to health, since HPE usually does not consider costs involved and respectively does not compare them with outcome/benefit achieved. Even if HPE evolves to consider the distribution of the benefit within the population and associated costs, the evidence about effectiveness and cost-effectiveness for interventions applied to populations will be relative, depending on the local context. 11 III.2) Interventions in other spheres of the economy (transport, education, etc.) with primary objective different than health, however with impact on health of populations. Health Impact Assessment (HIA) is used in order to determine: a) what are the potential and/or unintended, effects of an intervention on the health of a population; and b) the distribution of those effects within the population. The ultimate goal for HIA is to provide recommendations in order to maximize the positive effects on the health of a population and to minimize any negative effects. While HTA and HPE examine interventions with primary objective improving the health of individuals (HTA) or populations (HPE) and distribution of the outcome (benefit) is not properly analyzed, HIA is concerned with the distribution of the effects on health which, however, is not the primary objective for the intervention. Since outcomes from the primary objective can have various influences on different determinants of health, the evidence about effectiveness in HIA can be even more ambiguous, depending on the local context, than apparently it is for HTA and HPE. Any analysis of the distribution of the outcome – being either direct (HTA and HPE) or indirect (HIA) – means analysis of the winners and the losers from an intervention. Proper analysis requires considering of previous and new winners and losers, or applying who-gets-what approach in time – past, present, future, Panayotov (2008). While HTA is able to provide decision makers with reliable recommendations about maximizing individual’s benefit – as HTA examines rigorously individual’s benefit from different interventions and explains how it works and why it works – HIA currently does not examine rigorously who-gets-what from interventions applied to populations, can’t explain how it works and why it works, thus recommendations for maximizing the benefit for populations are often inconclusive for decision makers. Furthermore HIA hasn’t methodologies and tools to explain observed variations in the outcomes, therefore has similar problems with the evidence. Even if there were such tools, the evidence about effectiveness would be relative, since it depends on the local context. Often HIA does not consider costs, which in turn are not compared 12 with positive or negative effects on health, consequently no judgments about costeffectiveness or allocative efficiency – value for money – of proposed actions can be made, neither all those identified effects on the health of the population can be compared by decision makers with outcomes from primary objective of interventions. These deficiencies of HPE and HIA require revision of currently available and applied methodologies and techniques for assessment of the outcome/benefit from interventions applied to whole population. Recent variations of HIA, like health equity impact assessment (HEIA) aiming reduction of health inequalities, or mental wellbeing impact assessment (MWIA) focusing on one aspect of the health, still make little progress for proper appraisal of the interventions. Any proper evaluation of interventions applied to whole populations should examine rigorously the distribution of the outcome/benefit within the population, should explicitly state what it is in any specific case and should compare it with resources spent for achieving this outcome/benefit. Having in mind that interventions applied to populations are predominantly, if not completely, funded by the public, these methodologies and techniques should be able to take into account not only preferences of the recipients, but also preferences of the tax payers. This is necessary for achieving both: economic efficiency and five of Donabedian’s “seven pillars of quality” (4). Evidence in Public Health When people want to get the best outcome in relation to their predetermined goal, they rely on the best practice, or the best available evidence. For decision making in relation to health – decisions of type II and type III – evidence is not mere registering of a fact, or just seeing a pattern in database. For decision makers something becomes reliable evidence when it is explained how it works and why it works in certain cases, thus making it possible to replicate the outcome in new cases. It is important to underline that public health is about maximizing health and quality of life of whole populations, and not individuals, therefore theoretical frameworks and models, which are used for appraisal of interventions, should properly reflect this. 13 Only then the answers about “what works”, to whom, and to what extend will be reliable, as these are in EBM. However, as mentioned earlier, the evidence about effectiveness and especially cost-effectiveness of interventions can vary a lot depending on the local context, both: regarding interventions applied to individuals, and, even to greater degree, regarding interventions applied to whole populations. There are four aspects of the local context with impact on effectiveness and especially on cost-effectiveness of interventions applied to populations: 1) Organization of services provided. For example, do providers visit recipients, or recipients visit providers. 2) Costs incurred to individuals, families, employers, communities, society. For example, is there cost-shifting between central and local authorities, or between authorities and employers and/or families. 3) Cultural specificities. For example, which of the available alternative interventions is considered to be more appropriate and acceptable for specific group of the population. 4) Distribution of the benefit from an intervention within the population. For example, there are eight different combinations of distribution of the outcome/benefit from an intervention within population, Panayotov (2008). Identical interventions applied to populations achieve very different results due to differences in one or more aspects of the local context. Usually something “works” in varying degree when applied on different groups of the population. What “works” in one case may not “work” to the same extend in many other cases, or may not “work” at all in some cases, or can be even counterproductive in certain cases. Probably because of this, everyone who ever wrote about evidence in public health has noted that evidence is weak, ambiguous, equivocal, and even controversial. Pointing out the importance of proper theoretical framework for correct interpretation and use of the evidence, Green (2000) notes that “the accumulation of empirical evidence about 14 effectiveness is of limited value to the practitioner unless accompanied by general principles which might inform wider application”. She concludes that “Of more relevance to the practitioner are general principles together with an understanding of context-specific factors, which will allow adaptation to suit different situations”. Until now different models and frameworks used in public health do notice the evidence phenomenon, but were unable to produce satisfactory explanation of it. Panayotov Matrix (8) is a model based on critical realism (if A than always B), thus provides universal explanations and predictions, i.e. it is valid for any local context. The model examines the distribution of the benefit from interventions applied to populations and identifies eight possible combinations of distribution of the benefit (plus one, when there is no change). The matrix shows what happens with average health status and health inequalities in each specific combination of distribution of the benefit. These are the general principles with universal validity. By differentiating recipients of the benefit from an intervention: previous and new winners, relative losers, absolute losers, double losers, which are unique in different cases, Panayotov Matrix allows taking into account of local context. This way, by providing universally valid general principles and considering unique local context at the same time, Panayotov Matrix allows overcoming the major hurdle for HPE and HIA – relativeness of the evidence. Panayotov Matrix allows HTA, HPE and HIA to analyze properly the distribution of the benefit among the population in any specific case, thus is able to provide reliable recommendations to decision makers for maximizing the outcome/benefit for whole populations, i.e. the sum OAi + OBi. Combined with existing techniques for economic evaluation, for example cost-benefit analysis, choosing the best alternative – value for money – is achievable in any specific case. Any intervention implemented to populations (i.e. decisions of type III), fits in Panayotov Matrix, no matter whether improving the health of whole population is primary objective or not. The matrix shows what will happen with the health of that population in terms of changes in average health status and health inequalities. For decision making there are two important things to look for, when analyzing the distribution of the benefit at local level: firstly – are there any absolute losers, i.e. 15 groups from the population who will be worse-off compared to their previous situation, and secondly – are there any double losers, i.e. groups from the population who will be relative losers from the new intervention while already being relative losers (or even worse – absolute losers) before the change or from another intervention applied at the same time to the same population, Panayotov (2008). If from any specific intervention there are absolute losers and/or double losers at local level, then adjustments regarding distribution of the benefit from the intervention should be made until the intended result is achieved – no absolute losers and no double losers at local level. In all cases which can demonstrate evidence that an intervention “works”, i.e. achieves the goal for improving health and quality of life of whole populations, it is because there are no absolute losers and no double losers from the distribution of the benefit from this intervention at local level. Replicability of the evidence The concept of evidence is meaningful when, based on previous successful experience, it is possible to achieve intended result. When making choices people decide to replicate intervention, because they want certainty for replicating the result. Interventions are nothing more than means for achieving a predetermined objective. As already mentioned, achieving highest attainable health for an individual and for whole population is not the same thing. Therefore successful approach for achieving the later objective is different from successful approach for achieving the former. While for interventions applied to individuals replicability is copying of an intervention in order to achieve the same outcome for an individual, for interventions implemented to whole populations replicability is not the mere replication of the same intervention, but adjusting, modifying, or even changing it in order to achieve the intended outcome in terms of distribution of the benefit according to Panayotov Matrix. Usually, in order to make the right choice for specific local context, it doesn’t matter what is the evidence in terms of concrete interventions from other cases with different local context. Of more relevance for decision makers are interventions, which can achieve at local level the desired distribution of the benefit according to 16 Panayotov Matrix, thus leading to the desired predetermined objective – maximizing health and quality of life of populations, and not individuals. Analysis of interventions from other cases can be of help, provided that there is enough information regarding previous and new winners and losers in specific case, i.e. what is the distribution of the benefit, or who-gets-what, before and after the intervention. Replicability of the evidence in relation to interventions applied to populations means replication of certain combination of distribution of the benefit, which in turn will lead to replication of the result in terms of impact on average health status and health inequalities. In all cases – no matter other differences in the local context – where previous winners benefit more from the new intervention, the result will be always the same – growing health inequalities, even if nobody is worse-off compared to their previous situation. In all cases – no matter other differences in the local context – where there are no absolute losers and no double losers, the result will be always the same – growing average health status together with decreasing of health inequalities, i.e. maximizing health and quality of life of whole populations. Usefulness of Average Data for Informing Decision Makers on Choices for Resource Allocation in Public Health Any intervention aims achieving certain outcome, which presumably improves the status quo in relation to predetermined goal. In order to make whatever decision, either implicit or explicit, anyone needs information. Using average data alone has enough informative value for decisions regarding interventions applied to individuals. Since the goal is maximizing the outcome for specific individual, decision maker would choose the intervention with higher average success rate in similar cases. How does the same intervention perform to individuals with different characteristics is irrelevant both for the decision maker and for the recipient. However, if the goal is improving health and wellbeing of whole populations, then average data alone has very limited informative value for decision makers. Since the population consists of different groups – some are better-off than the average, others are worse-off – 17 decision makers should have information not only about these characteristics, but also what change an intervention would achieve to the different groups within this population, i.e. what is the distribution of the benefit from this intervention. Only then decision makers would be able to make the right choices in order to maximize the outcome for whole populations. If it is not clear who-gets-what and only average data for the population is available, then the decisions for allocating resources are made in blind, or are based on intuition at best, and when those groups who already are better-off benefit more from the interventions than rest of the population, the result can be only one – growing health inequalities, Panayotov (2008). One of the main roles of the research is to provide information, so that decision makers can choose wisely between alternative allocations of limited resources. Therefore the type and quality of the information provided are paramount for the quality of the decisions made. To blame decision makers for failure, i.e. making wrong choices of interventions, when addressing specific problem would be unfair, if they have had unsuitable, inappropriate, irrelevant and /or insufficient information. Since public health is about populations, it is about distribution of the benefit within these populations, therefore ultimately it’s about who-gets-what from an intervention. Quite reasonably decision makers want concrete and clear answers about the impact of specific intervention on different groups of the population. It is a duty of researchers to provide this type of detailed information, even if they are not asked to. Failure to provide information about who-gets-what from different interventions is a major shortcoming of the research in public health. Without this type of stratified information decision makers can not identify appropriate interventions for relevant populations, i.e. they can not make the right choices for resource allocation in order to maximize the outcome – improving health and quality of life for whole populations. Average data, although having its importance for aggregate results, can only blur the real picture at local level. Masking the distribution of the benefit by using only average data can be of interest only for those who get greater benefit from a certain intervention than rest of the population, while already being better-off. Beside that this way public health goals can’t be achieved, one should ask: “Is it fair, after all?” 18 Implications for Researchers, Decision Makers and Practitioners Distribution of the benefit is the most important aspect for interventions applied to populations. This should not be confused with distribution of populations. The former impacts the shape of the later. The later is normal distribution, which is defined by two parameters: the mean (average) μ and standard deviation σ. The mean identifies the location on the horizontal axis and standard deviation describes the spread. Normal distribution has bell-shape and the larger is the standatrd deviation, the more spread out is the distribution, i.e. the more flattened is the bell. Data for average health status alone gives no idea about the shape of the bell. One should be aware that average health status (AHS) can increase together with growing health inequalities (HInEq). Paradoxically AHS can increase even when some people are absolute losers, i.e. they are worse-off compared to their previous situation, provided that improvement among the winners from the intervention can exceed the deterioration among the losers, Panayotov (2008). As result, although that AHS improves, the bell will get more and more flattened. Health inequalities – being differences in health which are considered avoidable and unfair – represent cases when standard deviation is larger then it could and should be. Goal to reduce HInEq means implementing interventions with such distribution of the benefit among the population, which will lead to reducing the standard deviation. In other words, the bell will get narrower, more gathered around the new, improved mean – AHS (8). When choosing between different interventions (resource allocations with different combination of winners and losers) Panayotov Matrix predicts what will happen with both AHS and HInEq as result from any intervention in any specific case. This way decision makers will know what will be the change in the population bell after the intervention, i.e. will it get more flattened, or remain the same, or will it get narrower. Interventions with distribution of the benefit aiming maximizing the outcome with limited resources (i.e. the sum OAi + OBi) actually is achieving economic efficiency and will result in narrowing of population bell. Reducing standard deviation, while improving the mean, is maximizing health and quality of life for whole populations. 19 Conclusion Evidence facilitates making the right choices in order to achieve the best outcomes. In relation to health decision makers should clearly distinguish between personal services applied to individuals and complex interventions applied to populations. It should be clear for everyone that, although that the population is sum of individuals, achieving highest attainable health for individual and for population is not the same thing. Any intervention implemented to populations – no matter whether improving health is primary objective or not – is characterized by the distribution of the benefit from this intervention among the population. This is the most important factor affecting health and quality of life of whole populations. In any case there are eight different possible combinations of winners and losers from any intervention, which cause the corresponding changes in AHS and HInEq. Identical interventions can have different combination of winners and losers at local level, which will lead to different outcomes in AHS and HInEq. Therefore evidence regarding interventions applied to populations is not and can not be universal, but becomes relative, depending on distribution of the benefit from these interventions among population. Panayotov Matrix defines the general principles with universal validity for achieving identical outcomes – in terms of directions – for AHS and HInEq at any local context. Specific combination of winners and losers at local level will achieve nothing else, but the predicted outcome. If this is not the intended by decision makers outcome, then changes in distribution of the benefit among population should be made, so that the desired combination of winners and losers is achieved. Therefore – no matter what the differences in local context can be – Panayotov Matrix facilitates making the right choices in order to achieve best outcomes in relation to predetermined goal. _______________________________ 20 Appendix Table 1. Health expenditure by Source of Financing, OECD Countries, 2004 21 Notes (1) Countries of former USSR and Sub-Saharan countries during 1990s (and beyond). Vagero (2007) notes that 24 countries had decline in average life expectancy during 1990s: 6 out of 14 former USSR countries and 16 out of 41 Sub-Saharan countries. In 2004 both for men and women in Russia life expectancy was below the levels from 1965. Men (2003) estimates 2.5 up to 3 million excessive deaths in Russia during 1992-2001. (2) John Kenneth Galbraith describes the sources of power as well as the instruments for exercising power in “The Anatomy of Power”, Hamish Hamilton, New York, 1984 (3) Woolhandler and Himmelstein (2002) show that in USA tax-funded share of health spending is higher than the figure in OECD report and actually is almost 60 per cent. (4) Donabedian (1990) in “The seven pillars of quality”. The five I refer to are: 3) efficiency: the ability to obtain the greatest health improvement at the lowest cost; 4) optimality: the most advantageous balancing of costs and benefits; 5) acceptability: conformity to patient preferences regarding accessibility, the patientpractitioner relation, the amenities, the effects of care, and the cost of care; 6) legitimacy: conformity to social preferences concerning all of the above; and 7) equity: fairness in the distribution of care and its effects on health. (5) The history of attempts for taking into account preferences of the society (tax payers) is a topic for separate article and will not be analyzed here. These include: Diagnostic Related Groups, Casemix (1980), Pharmaceutical Benefits Scheme, Life Course Approach (1997), Hospitals’ Performance Monitoring, addressing Social Determinants of Health (2005) (6) There is a phenomenon called “supplier induced demand”, when providers abuse the agency relationship with their patients (consumers) in order to generate demand for personal gain, Folland et al. (2001). Rice (1999) has noted that “the waste is thought to be generated through provision of unnecessary services far more than through excess demand by patients”. (7) See chapter “Opposing interests” in “Public health and average health status: Do health inequalities matter”, Panayotov J., ICARE, Health Economics, 2008 (8) See chapter “Panayotov Matrix” in “Public health and average health status: Do health inequalities matter”, Panayotov J., ICARE, Health Economics, 2008 Panayotov Matrix and distribution-of-population bell. Red line represents cases when mean improves & standard deviation grows, i.e. the bell gets more and more flattened, while moving to right. Black line represents cases when mean decreases & standard deviation grows, i.e. the bell gets more and more flattened, while moving to left. Dashed black line represents cases when mean & standard deviation decrease, i.e. the bell gets narrower, while moving to left. Green line represents cases when mean grows & standard deviation decreases, i.e. the bell gets narrower, while moving to right. 22 References: Connelly J., Critical realism and health promotion: effective practice needs an effective theory, Editorial, Health Edu Research, Vol.16, No2, 115-120, Oxford University Press 2001 Connelly J., More public health theory please – but make it adequate, Editorial, Journal of Public Health, Vol.27, No.4, p. 315, 2005 Cromwell & Halsall 1995, recite from Hancock L. & Mackey P., Health care funding and rationing health care, in Health Policy in the Market State, Allen& Unwin 1999 Donabedian A., The seven pillars of quality, Arch Path Lab Med, 1990; 114(11):115-8 Green J., The role of theory in evidence-based health promotion practice, Editorial, Health Education Research, Vol. 15, No. 2, 125-129, Oxford University Press 2000 Erasmus MC, Tackling Health Inequalities in Europe: An Integrated Approach, Final Report, Department of Public Health, University Medical Centre Rotterdam, The Netherlands, 2007 Folland et al., The Economics of Health and Health Care, 3rd edition, Prentice Hall 2001 Hindle & Kerridge 1993, recite from Hancock L. & Mackey P., Health care funding and rationing health care, in Health Policy in the Market State, Allen& Unwin 1999 Hurley J., An overview of the normative economics of the health sector, Ch.2 in Handbook of health economics, Vol.1A, Elsevier 2000 Kelly M., et al., Getting evidence into practice in public health, Health Development Agency, London, 2004 Kelly M., et al., The development of the evidence base about the social determinants of health, Measurement and evidence knowledge network, CSDH, WHO, May 2006 Men, T. et al. “Russian mortality trends for 1991-2001: Analysis by cause and region”, Br Med J, 2003; 327: 964-970 Nutbeam D., Health promotion glossary, Health Promotion International 1998,13(4):349-364 Nutbeam D., Getting evidence into policy and practice to address health inequalities, Health Promotion International, Vol.19, No.2, Oxford University Press 2004 Panayotov J., Public health and average health status: Do health inequalities matter?, ICARE, Health Economics, 2008, Available at: www.icare.biz/health.html Rice T., The Economics of Health Reconsidered, Health Administration Press 1999 Sackett D. et al., Evidence based medicine, what it is and what it isn’t. BMJ 132:71-2, 1996 Vagero D., Health inequalities across the globe demand new global policies, Editorial, Scandinavian Journal of Public Health, 2007; 35: 113–115 23