Objectives
Randomized controlled trials are the gold standard for estimating treatment effects. However, in rare diseases with high unmet need, single-arm trials are used when randomization of patients to placebo or standard of care is infeasible or unethical. We evaluated various methods to control for confounding in estimating treatment effects in a small single-arm trial with a historical comparator group.
Methods
We used simulation to evaluate different techniques to estimate the “true” treatment effect on overall survival (OS) and objective response rate (ORR) in a specific target population using an external comparator design. We varied effect size, sample size, confounders, and correlation between confounders. We assessed two broad categories of methods: i) requiring specification for treatment allocation [propensity score (PS)-based inverse probability of treatment weighting (IPTW), standardized mortality/morbidity ratio (SMR), stabilized IPTW (SIPTW), overlap weighting (OW), stratification, and matching], and ii) adding outcome information (g-computation). Their precisions and accuracies were evaluated by a combination of 95% confidence interval (CI) coverage, power, bias, and mean square error (MSE).
Results
G-computation resulted in the most accurate and precise estimator of OS (95% CI coverage: 93.5%, power: 69.3%, bias: -0.001, MSE: 0.055) in a small sample size scenario of 30 treated subjects compared with 120 comparator subjects. Similar results were observed for ORR. In comparison, results for OS were: 95% CI coverage: 72.8%, 65.6%; power: 75.9%, 62.6%; bias: -0.026, 0.072; MSE: 0.114, 0.167 for SMR and IPTW, respectively.
Conclusions
In our simulated example, the g-computation estimator performed best to control confounding in a small single-arm trial with an external comparator group. PS based methods (e.g., SMR & IPTW) may be suitable as an initial step in the creation of the comparator arm when researchers are blind to the outcome, while g-computation can be subsequently used to estimate the efficacy of treatment.
Article info
Identification
Copyright
© 2021 Published by Elsevier Inc.
User license
Elsevier user license | How you can reuse 
Elsevier's open access license policy
Elsevier user license
Permitted
For non-commercial purposes:
- Read, print & download
- Text & data mine
- Translate the article
Not Permitted
- Reuse portions or extracts from the article in other works
- Redistribute or republish the final article
- Sell or re-use for commercial purposes
Elsevier's open access license policy
