The American Board of Quality Assurance and Utilization Review Physicians, Inc., (ABQAURP) and the Population Health Impact (PHI) Institute joined forces on a new kind of quality improvement initiative. A collaborative arrangement was announced to help health professionals acquire knowledge and tools in the area of quality evaluation processes to better assess the impact of population health programs. Experts from the PHI Institute will provide training and technical content, and ABQAURP will assure the educational elements meet standards for CME/CE accreditation.
Our organizations will jointly offer a Certificate in Quality Evaluation Processes (CQEPSM) to interested health care professionals including: nurses, pharmacists, physicians, administrators, and researchers that are responsible for disease management or other services related to population health and program evaluation.
The PHI Institute's mission is to "promote credible, transparent, standardized, and replicable evaluations of defined population health program ...through education and research." ABQAURP's mission is to "improve the overall quality of health care that is provided to the consuming public [and to] develop and provide educational opportunities for medical and health care professionals." The interrelated missions of our non-profit organizations will foster the appropriate environment for this program.
When a claim is made regarding the effectiveness of a health-related intervention, it is essential to understand the validity of that claim. The individual health care professional that is providing the intervention should be able to understand and respond to this issue. To understand how to create a credible program evaluation, it is necessary to distinguish standard clinical-based effectiveness from population-based effectiveness.
The most common way to assess validity of clinical effectiveness is to rely on expert panels. These panels are typically charged with reviewing the clinical literature and drafting "evidence-based" consensus treatment guidelines. While this evidence is often founded on results derived from defined population randomized control trials, observational studies may provide evidence as well. These guidelines are to help direct the practitioner in his or her treatment of individual patients. The big challenge for policy makers has been to "translate" that research into clinical "practice". However, this is not as successful as one would like, according to the oft-cited RAND study that reported that adults receive about half of recommended services (McGlynn et al NEJM, 2003; 348: 2635-46). Importantly, if a guideline-based treatment does not achieve the desired effect on one individual, one rarely, if ever, questions the effectiveness of the practitioner who has followed evidence-based medicine guidelines.
There is now a different kind of "medicine" taking hold playing by somewhat different rules. This is population-based medicine, also known as disease management, care management, health promotion, value-based reporting, payment for performance, etc. The goal of these programs is to encourage health care professionals, and their patients, to follow consensus guidelines. But if the defined population does NOT experience an increase in guideline-based metrics, or if the important sequelae (e.g. financial savings) are not measurably improved, the effectiveness of the population-based program is often questioned. However, in the case when the guideline-based metric DOES increase from baseline, or their sequelae are measurably improved, we rarely question the effectiveness of the population-based program. But we should begin to do so. Why? Because other initiatives, organized or not organized, could have been responsible. An example of the former is where two or more of the aforementioned population health management programs are operating concurrently. An example of the latter is where many different practitioners are independently following guidelines. Both, or neither, could have been responsible for the change.
It is essential that clinicians, who are often not well trained in advanced research techniques, or the "science of uncertainty", or the "art of probability", achieve some basic level of knowledge in understanding the elements necessary for valid inferences and common errors made along the way that lead to invalid inferences. Having the knowledge of a quality evaluation process, one can begin to ask simple questions to purveyors of population-based programs.
One basic question is, "How do we know the program we are reviewing was responsible for the improvement in guideline-based metrics?" For that matter, was the program responsible for any and all outcome metrics targeted by the program, whether clinical, economic, and/or satisfaction? The only way to answer this question is to have it compared to something comparable. This could be from the past, or, ideally, something that is occurring at the same time. We need to look at the intervention population and compare it to an equivalent population ' historical or concurrent ' that did not receive the intervention. In an ideal world, we would like a double-blind randomized process to be followed, but this is often not the case, and sometimes it is not even possible.
Randomized or not, one then needs to ask additional questions. Were the populations and metrics apples-to-apples comparisons? Were there arithmetic errors, random errors, systematic errors, and inferential errors? Even if done perfectly, what about "external validity"? Were there "so-what" errors? That is, is the "valid" finding even useful? The causality must be determined. Did the disease management or other population health program really "cause" the health improvement? We have entered an era of transparency in health care where the processes and outcomes of what individual practitioners do are being questioned on a daily basis. Similarly, those individuals who analyze, coordinate, intervene, and provide these disease management programs and other population health activities need to be similarly questioned, which is the purpose of offering the CQEPSM, beginning in 2007.
Over the coming months, the two organizations will bring their expertise to advancing this effort towards a standardized process where one can assess methods and results of population health programs for validity.
To opt-in to the QEP information list, please fill out the short inquiry form below. As information becomes available, you will be notified.
