MathSciDoc: An Archive for Mathematician ∫

Double-blind randomized controlled trials are traditionally seen as the gold standard for causal inferences as the difference-in-means estimator is an unbiased estimator of the average treatment effect in the experiment. The fact that this estimator is unbiased over all possible randomizations does not, however, mean that any given estimate is close to the true treatment effect. Similarly, while predetermined covariates will be balanced between treatment and control groups on average, large imbalances may be observed in a given experiment and the researcher may therefore want to condition on such covariates using linear regression. This article studies the theoretical properties of both the difference-in-means and OLS estimators conditional on observed differences in covariates. By deriving the statistical properties of the conditional estimators, we can establish guidance for how to deal with covariate imbalances.

Introduction. This paper presents a study protocol for a comparative effectiveness evaluation of abiraterone acetate against enzalutamide in clinical practice, two cancer drugs given to patients suffering from advanced prostate cancer. Method and analysis. The protocol designs a comparative-effectiveness analysis of abiraterone acetate against enzalutamide. With the substantial number of covariates a two-step procedure is suggested in choosing relevant covariates in the matching design. In the first step, an exploratory factor analysis reduces the dimension of a large set of continuous covariates to nine factors. In the second step, we reduce the dimension of the covariates, interactions and second order terms for the continuous covariates using propensity score estimation. The final design makes use of a genetic matching algorithm. The study protocol provides a detailed statistical analysis plan of the analysis sample derived from the matching design. The analysis will make use of linear regression and robust inference adjusted for multisignificance testing. Discussion. As in a randomised experiment the focus is on the design of the assignment to treatment. This allows the publication of this preanalysis plan before having access to outcome data. This means that the p values will be correct if the maintained assumption of uncounfoundedness is valid. Given that is p-hacking is substantial problem in empirical research, this is a substantial strength of this study. However, while design yields, balance on the observed covariates one cannot discard the possibility that unobserved confounders are not balanced. For that reason, sensitivity tests for the maintained assumption of uncounfoundedness are presented. Ethics and dissemination. The study was approved by the Regional Ethical Review Board in Uppsala, Sweden (Dnr 2017/482). Results will be published in a peer-reviewed journal and distributed to relevant stakeholders in healthcare.

Rerandomization is a strategy for improving balance on observed covariates in randomized controlled trials. It has been both advocated and advised against by renowned scholars of experimental design. However, the relationship and differences between stratification, rerandomization, and the combination of the two have not been previously investigated. In this paper, we show that stratified designs can be recreated by rerandomization and explain why, in most cases, stratification on binary covariates followed by rerandomization on continuous covariates is more efficient than rerandomization on all covariates at the same time.

Blocking is commonly used in randomized experiments to increase efficiency of estimation. A generalization of blocking removes allocations with imbalance in covariate distributions between treated and control units, and then randomizes within the remaining set of allocations with balance. This idea of rerandomization was formalized by Morgan and Rubin (Annals of Statistics, 2012, 40, 1263–1282), who suggested using Mahalanobis distance between treated and control covariate means as the criterion for removing unbalanced allocations. Kallus (Journal of the Royal Statistical Society, Series B: Statistical Methodology, 2018, 80, 85–112) proposed reducing the set of balanced allocations to the minimum. Here we discuss the implication of such an ‘optimal’ rerandomization design for inferences to the units in the sample and to the population from which the units in the sample were randomly drawn. We argue that, in general, it is a bad idea to seek the optimal design for an inference because that inference typically only reflects uncertainty from the random sampling of units, which is usually hypothetical, and not the randomization of units to treatment versus control.

Recently a computational-based experimental design strategy called rerandomization has been proposed as an alternative or complement to traditional blocked designs. The idea of rerandomization is to remove, from consideration, those allocations with large imbalances in observed covariates according to a balance criterion, and then randomize within the set of acceptable allocations. Based on the Mahalanobis distance criterion for balancing the covariates, we show that asymptotic inference to the population, from which the units in the sample are randomly drawn, is possible using only the set of best, or ‘optimal’, allocations. Finally, we show that for the optimal and near optimal designs, the quite complex asymptotic sampling distribution derived by Li et al. (2018), is well approximated by a normal distribution.

This article analyses the effect of cognitive behavioral therapy (CBT) for individuals with mild or moderate mental illness. We study the effects on sick leave, health care consumption, and drug prescriptions. We find that CBT improved health and prevented sick leave for individuals who were not on sick leave when treatment was initiated but had no effect for individuals who were on sick leave when the treatment was initiated.

This article considers experimental design based on the strategy of rerandomization to increase the efficiency in experiments. Two aspects of rerandomization are addressed. First, we propose a two-stage allocation sample scheme for randomization inference to the units in experiments that guarantees that the difference-in-mean estimator is an unbiased estimator of the sample average treatment effect for any experiment, conserves the exactness of randomization inference, and halves the time consumption of the rerandomization design. Second, we propose a rank-based covariate-balance measure which can take into account the estimated relative weight of each covariate. Several strategies for estimating these weights using pre-experimental data are proposed. Using Monte Carlo simulations, the proposed strategies are compared to complete randomization and Mahalanobis-based rerandomization. An empirical example is given where the power of a mean difference test of electricity consumption of 54 households is increased by 99%, in comparison to complete randomization, using one of the proposed designs based on high frequency longitudinal electricity consumption data. Supplementary materials for this article are available online.