 |
     |
  |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Confronting the Evidence in Evidence-Based Prevention: Current Scientific and Political ChallengesSummer 2008
Recent years have witnessed the increasing adoption of an evidence-based public health approach in HIV prevention science, yet this approach is limited by lack of consensus about what counts as "evidence." In partnership with the Caucus for Evidence-Based Prevention -- an alliance of more than 50 non-governmental HIV/AIDS organizations -- SFAF hosted sessions at the 2007 United States Conference on AIDS in Palm Springs, the 2007 National HIV Prevention Conference in Atlanta, and the Microbicides 2008 Conference in New Delhi, India. Key questions addressed at these sessions include:
This article addresses the thorny issues underlying these questions. Defining and Deriving Evidence: What Counts?The evidence-based approach to HIV prevention emphasizes the application of rigorous research methodologies in public health science that parallel those used in clinical research -- chiefly, the adoption of experimental study design, and specifically, the randomized controlled trial (RCT) -- for assessing intervention efficacy. The RCT is considered the most rigorous experimental method because of its high level of control, and thus its ability to minimize bias and to avoid false conclusions. Generally, in experimental studies, the investigator exercises control over some factor that, when varied, can produce different outcomes, as well as control over the recruitment of and inclusion criteria for study participants, the assessment of baseline measures, the content and delivery of the intervention, and how intervention outcomes are measured. Random assignment of "like" individuals to experimental or control groups balances any potential effects of background or contextual factors (such as sex, race, or geographic location) that might influence the outcome of an intervention, thereby enhancing the internal validity of experimental interventions.
Although the RCT remains the "gold standard" method of obtaining evidence in clinical and public health intervention research, it does have some limitations for HIV prevention work. As many people have noted, the RCT -- and experimental studies more generally -- may be inappropriate and even unethical in many situations. For example, a researcher interested in HIV prevention for injection drug users could not randomize some individuals to receive sterile injection equipment and others not to, since we know from observational studies that sterile syringes prevent HIV transmission and needle sharing promotes transmission. Withholding a known effective intervention from study participants is unethical. But even where appropriate, experimental studies may be limited in their replicability and generalizability. Some researchers have argued that the very characteristics that make an intervention successful in an efficacy trial, such as standardization in population group and intervention protocol, are fundamentally different from -- and perhaps at odds with -- those that help effective interventions succeed at a population level, such as having broad appeal and adaptability. This is sometimes discussed as the tension between "internal validity" and "external validity."
Community-based organizations (CBOs) in the U.S. face increasing pressure from governmental and non-governmental funding sources to adopt RCT-based prevention interventions. For example, the U.S. Centers for Disease Control and Prevention (CDC), the primary funding agent of HIV prevention programs in the country, promotes a "tiers of evidence" paradigm that places RCTs at the highest, most fundable level. CDC encourages, if not requires, eligible CBOs to adopt prevention models from a list of RCT-derived behavioral interventions that CDC deems efficacious. This has posed problems for many CBOs that do not have the research capacity to implement and evaluate these particular interventions. Others have found it necessary to make significant adaptations to the given interventions so that they are situationally and culturally appropriate to the CBOs' particular populations and settings, which then raises questions about "fidelity" to the original intervention and how to interpret evidence about the modified versions' success. Moreover, many CBOs, as well as many social science researchers, believe in a definition of evidence that encompasses not only rigorous findings from biological, behavioral, and social science research, but also knowledge gained from the lived experience of individuals and communities affected by HIV/AIDS. This would suggest a broader range of HIV prevention interventions and programs with a wider scope of outcomes than those officially deemed "evidence-based" and derived only from experimental methods. However, there is no consensus among all relevant sectors that this comprehensive definition of evidence warrants scientific pursuit or funding, or is an acceptable basis for policy decisions. Methods for Evaluating Social and Policy Interventions: What Is Appropriate?As the HIV prevention science field has increasingly come to recognize the importance of social factors, institutions, and structures in fueling and/or stemming the spread of HIV/AIDS, serious questions have been raised about the appropriateness of RCTs for social and policy interventions, and what other methods might be a better fit. Medical interventions (e.g., drugs or procedures) are generally more transferable to different populations than are behavioral and social interventions that inherently interact with participants' characteristics and social contexts, and in which the causal chain between agent and outcome is more complex. The problem is that complex social phenomena -- such as gender relations, poverty and economic inequality, and violence -- cannot be reduced to a handful of variables that can easily be modified or controlled for testing in experimental designs. One could argue that it is precisely the confounding of people's individual, community, and social characteristics that explains the problem of HIV in the first place. Moreover, many social interventions address outcomes that are often perceived as "intermediate" (behavior change) or "distal" (e.g., poverty alleviation, gender equity, educational attainment), rather than the "proximate" outcomes of ultimate interest (HIV incidence or prevalence). And, should social interventions be implemented, the time lag inherent in evaluating their impact on proximate, intermediate, or distal HIV-related outcomes at the population level can be significant.
Because of these complicating factors, additional methods, such as observational studies, mathematical modeling, ethnography, and policy analysis, are necessary to evaluate the effects of social and policy interventions for HIV prevention. The bottom line is that, in all cases, the question should drive the methodology, not the other way around. We should not begin the discussion with the question: "What kind of randomized trial with an HIV-incidence outcome should we design in order to get at 'X' social factor?" Rather, we should begin by asking: "What do we still need to better understand about 'X' social factor; and once we understand it, what is the best intervention design to address it, and how can we best evaluate the outcomes of that intervention with respect to HIV prevention?" Methodological Limitations of Prevention TrialsEven where RCTs are the most appropriate method to test certain interventions, null and negative findings (that is, where no difference is observed between experimental and control groups, or where the intervention appears to cause harm) from important prevention technology trials raise questions about our ability to generate the evidence we need to answer fundamental questions about efficacy and, ultimately, effectiveness. A recently completed trial testing the efficacy of the latex diaphragm for HIV prevention among women provides a good example. Known as the MIRA trial, this study involved more than 5,000 sexually active HIV-negative women in Zimbabwe and South Africa who were randomized to receive either a lubricant gel or both the gel and a latex diaphragm. All women in both arms were also provided the current "standard of prevention," which includes information and access to proven prevention strategies -- particularly male condoms. This ethical requirement, which applies to all HIV prevention trials, posed a scientific dilemma in the MIRA study. If participants followed the advice to use condoms, it would be very difficult to separate out an independent effect of the diaphragm, since any effect observed might be a result of either diaphragm use or condom use. This is precisely what occurred: all participants increased their condom use, so it became statistically impossible to detect an independent effect of the diaphragm and to make statements about its efficacy for HIV prevention over and above male condoms. The outcomes of the MIRA trial were further complicated by lower than expected rates of adherence (using the diaphragm as directed): 73% of participants in the diaphragm arm reported using the device during their last sex act and only 58% reported always using the diaphragm in the previous three months, despite researchers' attempts to improve adherence rates. Other prevention trials have had similar problems with adherence. For example, in the Carraguard microbicide trial, completed in 2007, 50% of sex acts were covered by gel (microbicide candidate or placebo) and condoms, and 27% were covered by gel alone; this trial, too, found no difference in HIV infection rates between the experimental and control groups. If trial participants do not use the study product as directed 100% of the time, even when they do not use condoms, we cannot know if outcomes reflect the lack of efficacy of the product or its under-use. An additional related issue is that what we know about product use and adherence is usually derived from self-reports -- that is, study participants telling researchers whether they had sex, the frequency and timing of sex acts, whether or not they used the product and/or condoms, how many sex partners they had, whether they washed their genitals before or after sex, etc. Although there is a rich science of self-report and its validity, there also is a great deal of skepticism about it. Doubts arise particularly when there are biomarker data available that appear to contradict self-report data. In the Carraguard microbicide trial, for example, of the women who reported not having had sex in the past 48 hours, 19% tested positive for semen in vaginal screenings. These issues of condom use, adherence to products under study, and the validity of self-report data will be faced by all current and future HIV prevention technology trials, and will continue to make it difficult to answer the ultimate question of interest: Does this new technology reduce HIV infection rates by itself, and, if so, at what level of efficacy? Even where the efficacy of prevention interventions has been established in controlled studies, often those findings have not translated to effectiveness in real-world settings. Over the past 25 years, hundreds of behavioral interventions have been developed, tested, and replicated in a range of population groups and settings. But the 20%-30% risk reduction typically seen in the most efficacious behavioral interventions so far has not translated into population-wide effects on HIV incidence. This is likely a result of lack of widespread scale-up of behavioral interventions and difficulties in sustaining behavior change, such as consistent condom use, over time. Key questions remain about the relationship between what is observed in very controlled, small-scale studies among individuals and small groups and what is observed at the population level in whole communities and societies. Ideological Obstacles to Evidence-Based PreventionEven where scientific consensus exists about what constitutes good evidence for making policy decisions, it often is ignored or challenged by people with ideological agendas that run counter to the evidence. Such challenges have intensified over the past eight years. The promotion of abstinence-only-until-marriage sex education and the opposition to syringe exchange programs are perhaps the keenest examples. Abstinence-Only-Until-Marriage ProgrammingOver the past quarter century, nearly 1.5 billion federal tax dollars have been allocated to the abstinence-only-until-marriage approach to the prevention of HIV, other sexually transmitted diseases, and unintended pregnancy. Since 1998, the U.S. government has offered every state the opportunity to apply for grants (totaling $50 million per year) that have as their "exclusive purpose teaching the social, psychological, and health gains to be realized by abstaining from sexual activity." These programs are prohibited from discussing condoms or contraception, except to emphasize their failure rates; the "exclusive purpose" clause prevents communication of any other risk-reduction messages -- including discussion of condoms, a known effective HIV prevention tool. In 2001, social conservatives secured additional federal funds to create the Community-Based Abstinence Education program, which provides direct competitive grants for abstinence-only-until-marriage programs. What began as a $20 million per year program has grown by 450%, reaching a current total of $113 million. President Bush has proposed increasing this funding stream to $141 million for Fiscal Year 2008. These programs are conducted in diverse settings, from schools to county health departments to right-wing-operated "crisis pregnancy centers." Few venues that are considered essential to conducting sound, evidence-based health education have escaped the explosion in abstinence-only-until-marriage funding. Furthermore, the U.S. government has exported these programs to resource-limited countries most affected by HIV/AIDS; U.S. funding for international programs that promote the so-called "ABC" approach to HIV prevention -- "abstain, be faithful, use condoms" -- favors the "A" and the "B" over the "C" in both policy and dollars (see sidebar below). Despite the exponential growth in abstinence-only-until-marriage funding and programming at the state, federal, and international levels, evidence from program evaluations simply does not support the approach. Most recently, a large federally funded study found no evidence that abstinence-only-until-marriage programs have achieved their goal of increasing rates of sexual abstinence -- the explicit purpose of the programs. The study, commissioned by the U.S. Department of Health and Human Services, enrolled more than 2,000 children and teenagers in four states, and found that youth in the abstinence-only-until-marriage program group were no more likely than those in the control group to have abstained from sex: 49% in each group remained abstinent. In addition, those who reported being sexually active had a similar average age at sexual debut (14.9 years) and similar numbers of sexual partners in the program and control groups. These findings have sparked a welcome debate. On April 23, 2008, concerned policy makers and medical experts testified during the first-ever Congressional hearing on the effectiveness of federally funded abstinence-only-until-marriage programs, held by the U.S. House of Representatives Committee on Oversight and Government Reform and led by Chairman Henry Waxman (D-CA). Witnesses spoke of medically inaccurate information provided by program materials, pressure to sign "virginity pledges" in front of peers, and the alarmingly high numbers of teenage girls with sexually transmitted infections -- one in four, according to CDC. "Continuing funding for ineffective abstinence-only programs makes no sense," said Congresswoman Barbara Lee (D-CA), author of the Responsible Education about Life (REAL) Act, which would initiate federal funding for comprehensive sex education. The flaws in abstinence-only-until-marriage programs have not escaped states' attention. Seventeen states currently do not participate in the grant program, and to date, 13 states have conducted evaluations of their federally funded abstinence-only-until-marriage programs; not a single one has found the programs to be a good investment. By focusing on the lack of evidence to support abstinence-only-until-marriage programming as a viable HIV prevention tool, advocates have raised the standard of evidence for more comprehensive behavioral interventions. Syringe Exchange for HIV Risk ReductionSyringe exchange provides another compelling example of policy makers and funders challenging evidence and suppressing a proven HIV prevention tool on ideological grounds. Among industrialized countries, only Sweden and the United States fail to federally fund syringe exchange programs; the rest of the industrialized world supports these programs because the evidence is clear that they are an overwhelming success. For example, no fewer than eight comprehensive reports prepared by U.S. government agencies have concluded that syringe exchange could be a vital resource in curbing new HIV infections. It is estimated that nearly 200 needle exchange programs currently operate (legally or otherwise) in 38 states, Puerto Rico, Washington, D.C., and tribal lands. Yet most of these programs constantly struggle for resources to remain in operation. Without federal support, scaling up this evidence-based HIV prevention strategy to the necessary level is impossible. So why, despite evidence that these programs lower HIV incidence among injection drug users, their sex partners, and their children -- and that needle exchange does not promote drug use -- does the U.S. government continue to withhold funding? Chiefly because social conservatives have convinced many federal and state policy makers that supporting such programs is tantamount to political suicide: it is interpreted as a sign of being "soft on drugs." This argument has been extremely effective over the years. In early 2008, however, reflecting a first glimmer that things might be changing in favor of public health evidence over politics and ideology, Congress passed and President Bush signed a law allowing the District of Columbia to use its own city funds to support syringe exchange programs. With 17% of the entire country's new infections associated with injection drug use, broadening coverage of this intervention would go far in reducing new HIV infections. Whereas the standard of prevention applied to clinical trials posits that withholding a proven intervention is unethical, ideology-based policy has turned this imperative on its head, withholding information about or hindering access to condoms and clean needles -- established HIV prevention tools -- based on the belief that providing such information or access encourages sexual promiscuity and drug use. Both the promotion of abstinence-only-until-marriage programs and the suppression of syringe exchange continue, despite clear evidence showing the folly and the wisdom, respectively, of pursuing each approach. ConclusionThe methodological and ideological challenges to HIV prevention have prompted many in the science and advocacy community to focus on developing clear consensus about evidence -- what constitutes it and how best to obtain and interpret it. To this end, and as a culmination of the past year's initiative, SFAF, in partnership with the Caucus for Evidence-Based Prevention, will host a panel discussion of these evidence issues in an official satellite session at the 17th International AIDS Conference in Mexico City this August. Our aim is to provide better guidance to funders, policy makers, and program implementers who wish to have an effective and lasting impact on the HIV/AIDS pandemic through interventions that are based on sound scientific principles and the lived experience of people with HIV/AIDS and their communities, and that place public health over politics and ideology. Judith D. Auerbach, PhD, is Deputy Executive Director for Science and Public Policy at the San Francisco AIDS Foundation. William Smith is Vice President for Public Policy at the Sexuality Information and Education Council of the United States. The authors are founders of the Caucus for Evidence-Based Prevention.
Selected Sources
 This article was provided by San Francisco AIDS Foundation. It is a part of the publication Bulletin of Experimental Treatments for AIDS. Visit San Francisco AIDS Foundation's Web site to find out more about their activities, publications and services. |
The Body: About The Body | Contact The Body | Site Policies | Content Providers | Advertise With Us | Site Map
Remedy Health Media: About Remedy Health Media | Contact Remedy Health Media | Terms of Use | Privacy Policy
The Body is a service of Remedy Health Media, LLC, 750 3rd Avenue, 6th Floor, New York, NY 10017. The Body and its logos are trademarks of Remedy Health Media, LLC, and its subsidiaries, which owns the copyright of The Body's homepage, topic pages, page designs and HTML code. General Disclaimer: The Body is designed for educational purposes only and is not engaged in rendering medical advice or professional services. The information provided through The Body should not be used for diagnosing or treating a health problem or a disease. It is not a substitute for professional care. If you have or suspect you may have a health problem, consult your health care provider.
Email
Print-Friendly
Glossary
PDF
