Independent Review of Research Data
There are certain minimum protections designed to help assure the integrity of research data that one should look for when evaluating reports of clinical trial results. There should always be several layers of review evident that not only evaluate the final result, but monitor the trial from initial design all the way through publication. In each of these, independent review is perhaps the most important common thread.
Institutional Review Boards
One of the first levels of review evaluates a research plan before it is implemented. Institutional Review Boards (or IRBs) exist at the actual sites in the communities where trials are undertaken. A typical IRB would include physicians, ethicists, members of the clergy, and patient representatives who assure that a trial will be appropriate and ethical within the culture of that particular community, and that limited resources are expended on questions that are relevant to that community.
Community Advisory Boards (CAB)
CABs are patient advisory groups drawn from the communities in which the trials will be conducted. They give the patient's perspective on whether a trial offers ethical, reasonable approaches to the issues that are relevant to that community. The federally-funded clinical trial networks such as the CPCRA or the AACTG are mandated to establish CABs. Each local CAB also elects one or several members to serve on a network-wide Community Constituency Group (CCG). This system of representation tries to insure that the medical and administrative leadership of the national networks will hear the concerns of community members that their research is supposed to benefit.
It was not always this way. Up until the late eighties, meetings of federally-funded AIDS research groups were closed affairs -- patient representatives were not even welcome as observers. Courageous and brash activists crashed the group meetings of the AACTG (then called simply the ACTG) and demanded not only to observe the meetings, but also to participate in decision-making. Fortunately, some investigators were wise enough to see that community input could only help to create more relevant and attractive trials. It is now mandated in grants and cooperative agreements that federally-funded AIDS research groups not only have functional CCGs, but that patient representatives and other advocates be full members of key committees and protocol teams. Frequently the CCG representative will be listed among the group's research paper authors, thus helping assure responsible oversight from a patient perspective.
Data and Safety Monitoring Boards
Every trial in the U.S. is overseen by what is called a Data and Safety Monitoring Board or DSMB. These are physicians, researchers and ethicists who have ongoing access to the trial data and are empowered to make important suggestions to modify, prolong, or even terminate a study, depending on what occurs in the course of that trial. If it becomes absolutely clear that one treatment is superior to another, the DSMB can choose to end the trial, so as to reduce the amount of time patients remain on the "loser" therapy. Alternatively, if they decide that a trial has no possibility of ever achieving a meaningful answer, they may decide to halt the study and stop wasting resources and participants' time. Or they may modify a trial to improve its scientific integrity or assure patient safety. A good example of this was an early AIDS trial of a drug called pyremethamine that depleted B vitamins. A DSMB decided that a special form of B vitamin called leucovorin needed to be added to the protocol to protect patients.
In general, while a trial is ongoing, very few people have access to the data. Usually only the statistician(s) and DSMB get full summaries of key endpoint data. Even the principal investigator is unable to review the data in midstream. This is to ensure that no one jumps to conclusions about trends seen in a trial that are not yet statistically significant (like trying to figure out the average number of coin tosses after only three flips). Otherwise there is a risk that the investigators or others could consciously or subconsciously alter their behavior, and affect the trial's outcome. The investigators are said to be "blinded" to the endpoint data. The DSMB also tries to strike a balance between closely watching the trial, and not looking too frequently. The reason for this is actually pretty simple: every time anyone takes a look at the data, they increase the risk of prematurely declaring a winner, when in fact, not enough data (or "coin flips") have accumulated to really be certain. So when the sample size (the number of patients) is calculated for a trial, it is adjusted upward for each time the DSMB is expected to take a peek.
Because the DSMB is privy to such sensitive information, it is absolutely imperative they be as independent as possible. If premature "hints" about how a trial is going leak into the community it could cause people to make up their minds even though the data are not yet reliable. So an independent DSMB is an absolute necessity for credible research.
Disclosing Financial Ties
In all of these cases -- IRBs, CABs, CCGs and DSMBs -- the essential characteristic is that they be independent of the company making the drug and the organization conducting the trial. Independent reviewers must have no personal financial, ideological, or ego investment in the outcome of the trials they oversee.
These independent review mechanisms also protect investigators who often do have financial and other ties to corporations keenly interested in the outcome of the research they conduct. For better or worse, these financial relationships are now so common as to seem unavoidable. Therefore it is crucial that investigators disclose these relationships whenever writing or speaking about their research or offering their expert opinion on matters that could affect the business of their patrons.
Marcia Resnick, the former editor of the New England Journal of Medicine, perhaps the most prestigious medical journal in the world, was among the first to speak out strongly about the risks of conflict of interest and requiring the routine disclosure of the investigators' financial ties along with publication. This was an important first step. After all, if you want to critically evaluate a research article, it's reasonable to want to know if the investigator had a financial stake in the outcome. Sadly, while everybody seems to agree with Dr. Resnick in principle, not all journals have taken such a firm stand on the right of the reader to consider possible conflicts-of-interest just as carefully as a study's methodology. Happily, this seems to be an area the National Institutes of Health is showing leadership in at this time, and some of the federally-funded research groups are already drafting, or putting in place, much stronger conflict of interest disclosure policies.
Back to the GMHC Treatment Issues February 2002 contents page.