Evaluating the Validity of a Harm Study

Are the results of this article valid?

FOR COHORT STUDIES:   Aside from the exposure of interest, did the exposed and control groups start and finish with the same risk for the outcome?

1. Were patients similar for prognostic factors that are known to be associated with the outcome (or did statistical adjustment level the playing field)?
The two groups, those exposed to the harm and those not exposed, must begin with the same prognosis. The characteristics of the exposed and non-exposed patients need to be carefully documented and their similarity (except for the exposure) needs to be demonstrated. The choice of comparison groups has a significant influence on the credibility of the study results. The researchers should identify an appropriate control population before making a strong inference about a harmful agent. The two groups should have the same baseline characteristics.  If there are differences investigators should use statistical techniques to adjust or correct for differences.

2. Were the circumstances and methods for detecting the outcome similar? 
In cohort studies determination of the outcome is critical.  It is important to define the outcome and use objective measures to avoid possible bias.  Detection bias may be an issue for these studies, as unblinded researchers may look deeper to detect disease or an outcome.

3. Was follow-up sufficiently complete? 
Patients unavailable for complete follow-up may compromise the validity of the research because often these patients have very different outcomes than those that stayed with the study. This information must be factored into the study results.

FOR CASE CONTROL STUDIES:  Did  the cases and control group have the same risk (chance) of being exposed in the past?

1. Were cases and controls similar with respect to the indication or circumstances that would lead to exposure?
The characteristics of the cases and controls need to be carefully documented and their similarity needs to be demonstrated. The choice of comparison groups has a significant influence on the credibility of the study results. The researchers should identify an appropriate control population that would be eligible or likely to have the same exposure as the cases.

2. Were the circumstances and methods for determining exposure similar for cases and controls? 
In a case control study determination of the exposure is critical.  The exposure in the two groups should be identified by the same method. The identification should avoid any kind of bias, such as recall bias. Sometimes using objective data, such as medical records, or blinding the interviewer can help eliminate bias.