Run-in Periods: Research Staple or Simply Running into Trouble?

By: Peter Lais

Peer Reviewed

A run-in period (“run-in”) is a pre-randomization phase in a randomized controlled trial (RCT) in which participants receive a placebo, investigational intervention, or other therapies to assess adherence, tolerability, and placebo responsiveness. Those who demonstrate nonadherence, intolerance of side effects, or a marked placebo response during the run-in are excluded, and the remaining participants proceed to randomization. Investigators conducting run-in studies contend that these phases enhance RCT efficiency by enriching the study sample with participants more likely to complete the protocol and demonstrate a response to the intervention under study (a “treatment effect”).^1–3 In essence, run-ins aim to improve signal-to-noise by filtering out variability, allowing for the discovery of treatment effects that would otherwise remain obscured.

Yet the world is a noisy place: treatment nonadherence, intolerable side effects, and placebo effects abound, and patients may not always follow their treatment regimens strictly. These unintended outcomes attenuate observed treatment effects by decreasing patient exposure to the intervention under study:¹ if one patient takes an effective antihypertensive medication and the other forgets about it entirely, for example, the average medication effect will appear diminished. This raises a critical question about external validity: do RCT samples “filtered” by run-ins accurately reflect real-world populations, or do they instead create idealistic settings that deviate from typical clinical environments^1,2,4? Put differently, should RCTs analyze a treatment’s “best-case” efficacy or “average-case” effectiveness?

An argument in favor of run-ins posits that best-case conditions mitigate confounding that may otherwise obscure treatment effects; consequently, clinicians may replicate these effects by targeting therapy to comparable patients.^4,5 If a run-in excludes nonadherent patients from a certain RCT, for instance, then clinicians may simply administer the corresponding treatment to individuals they believe will be adherent. Although intuitive, this rationale falters in practice since clinicians are poor judges of treatment adherence a priori.⁴ As a result, interventions are likely to be administered to patients who are not comparable to those originally studied, where the magnitude of effect is uncertain. Including average individuals in study samples, on the other hand, biases the study towards the null hypothesis (i.e., no treatment effect) by decreasing average exposure in the treatment arm.^1,4 If a treatment effect remains statistically significant under average-case conditions, the effect is more likely to be robust and clinically meaningful. This perspective functions as a hedge: by ensuring an investigational drug performs well in the average case, it will likely perform similarly—or better—in most real-world cases that are difficult to anticipate experimentally.

Run-ins also introduce additional work needed for experimental setup, data collection, analysis, and reporting, increasing the chance of errors. Indeed, meta-analyses of run-in studies reveal that up to 90% of these studies fail to report certain elements of the run-in, such as the demographics of the excluded patients or the exclusion criteria.⁶ Even when adequately reported, run-in protocols may still diverge from current clinical practice. One frequently cited example is PARADIGM-HF,⁷ known for introducing the heart failure medication sacubitril-valsartan, in which the run-in involved patients receiving trial doses of 10 mg oral enalapril twice daily and 200 mg oral sacubitril-valsartan twice daily. Such dosing may exceed what is commonly achieved in routine care, and many patients may not tolerate these higher targets, highlighting potential discrepancies between trial populations and typical practice.⁸ Similar concerns have prompted investigators to question whether run-ins can compromise external validity (generalizability).^1,2,4,9

Judicious use of run-ins can nonetheless aid interpretation, particularly in the evaluation of potential harms.⁴ The CAST trial,¹⁰ which unexpectedly demonstrated increased sudden death associated with strict suppression of ventricular ectopy following myocardial infarction, exemplifies this point. CAST employed a run-in that excluded participants who did not achieve predefined thresholds of ectopy suppression on antiarrhythmic therapy (e.g., at least 80% suppression of ventricular premature depolarizations and at least 90% suppression of runs of ventricular tachycardia). In this instance, the best-case scenario—effective suppression—was associated with harm. Intuitively, an average-case scenario in which suppression is less consistently achieved could produce a more moderate risk that might require larger samples to detect. Regardless of whether one attributes harm to effective suppression or to the absence of clinical benefit despite surrogate improvement, the trial provided little impetus to pursue strict ectopy suppression. In this context, the run-in strengthened the argument against an ineffective and potentially harmful practice.⁴ More broadly, when assessing safety signals, a run-in can serve as a conservative test: if best-case use of an intervention yields harm, there is little justification for its real-world implementation.

These considerations have motivated meta-analytic work, such as that from Murphy and colleagues, intended to assess the effects of run-in periods on treatment effect magnitude.² By matching run-in studies to comparable non-run-in studies based on populations, interventions, controls, and outcomes, the authors found no significant aggregate difference in effect estimates attributable to run-ins. However, individual matched comparisons varied, with run-ins sometimes amplifying and sometimes attenuating observed effects unpredictably. Caution is therefore warranted when assessing the generalizability of outcomes from a limited number of landmark run-in studies; conversely, consistent results across multiple trials should inspire confidence regardless of whether run-ins are utilized. Run-ins must therefore be analyzed through multiple lenses: although a single run-in may unpredictably affect the outcome of its parent study, a series of studies with judiciously utilized run-ins may frame the same outcomes in a manner that more clearly demonstrates their clinical importance.

Run-in periods are among the many methodologies in evidence-based medicine that merit scrutiny. Readers should thoughtfully assess whether run-in study samples represent the populations they serve and evaluate how run-ins influence the strength and generalizability of authors’ claims. This includes me, even as a clerkship student—and when I come across another run-in study, I’ll be spending an extra moment determining if the reported findings meaningfully apply to my patients.

Peter Lais is a Class of 2028 medical student at NYU Grossman School of Medicine

Peer Reviewed by Kevin Zhang MD, Clinical Assistant Professor, Department of Medicine, NYU Grossman School of Medicine

Image Courtesy of Wikimedia Commons See page for author: https://commons.wikimedia.org/wiki/File:Pills_in_blister_pack.jpg 

References

1. Huo X, Armitage J. Use of Run-in Periods in Randomized Trials. JAMA. 2020;324(2):188-189. doi:10.1001/jama.2020.6463
2. Murphy RP, O’Donnell MJ, Nolan A, et al. Effect of a Run?In Period on Estimated Treatment Effects in Cardiovascular Randomized Clinical Trials: A Meta?Analytic Review. J Am Heart Assoc. 2022;11(20):e023061. doi:10.1161/JAHA.121.023061
3. Schechtman KB. Run-in Periods in Randomized Clinical Trials. J Card Fail. 2017;23(9):700-701. doi:10.1016/j.cardfail.2017.07.402
4. Pablos-Méndez A, Barr RG, Shea S. Run-in Periods in Randomized Trials: Implications for the Application of Results in Clinical Practice. JAMA. 1998;279(3):222-225. doi:10.1001/jama.279.3.222.
5. Guyatt GH. Users’ Guides to the Medical Literature: II. How to Use an Article About Therapy or Prevention B. What Were the Results and Will They Help Me in Caring for My Patients? JAMA. 1994;271(1):59. doi:10.1001/jama.1994.03510250075039
6. Scott AJ, Sharpe L, Quinn V, Colagiuri B. Association of Single-blind Placebo Run-in Periods With the Placebo Response in Randomized Clinical Trials of Antidepressants: A Systematic Review and Meta-analysis. JAMA Psychiatry. 2022;79(1):42. doi:10.1001/jamapsychiatry.2021.3204
7. McMurray JJ, Packer M, Desai AS, et al. Angiotensin–neprilysin inhibition versus enalapril in heart failure. N Engl J Med. 2014;371(11):993-1004. doi:10.1056/NEJMoa1409077
8. Jessup M. Neprilysin inhibition — a novel therapy for heart failure. N Engl J Med. 2014;371(11):1062-1064. doi:10.1056/NEJMe1409898
9. Collister D, Rodrigues JC, Mbuagbaw L, et al. Prerandomization run-in periods in randomized controlled trials of chronic diseases: a methodological study. J Clin Epidemiol. 2020;128:148-156. doi:10.1016/j.jclinepi.2020.09.035
10. Echt DS, Liebson PR, Mitchell LB, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo: The Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991;324(12):781-788. doi:10.1056/NEJM199103213241201
11. Laursen DRT, Paludan-Müller AS, Hróbjartsson A. Randomized clinical trials with run-in periods: frequency, characteristics and reporting. Clin Epidemiol. 2019;11:169-184. doi:10.2147/CLEP.S188752
12. Packer M. Why Has a Run-In Period Been a Design Element in Most Landmark Clinical Trials? Analysis of the Critical Role of Run-In Periods in Drug Development. J Card Fail. 2017;23(9):697-699. doi:10.1016/j.cardfail.2017.07.401

 

Share: Twitter | Facebook | Email