Abstract

In observational studies, subjects are not randomly assigned to treatment or control, so they may differ in their chances of receiving the treatment. A simple method is developed and demonstrated for displaying the sensitivity of conventional two-group permutation inferences to departures from random assignment of treatments. The unmatched case, discussed here, differs in certain technical and computational details from the matched case, discussed previously; however, the underlying model and the method for quantifying departures from randomization are the same. The method may be applied to Wilcoxon's rank sum test, the Gehan and log-rank tests for censored outcomes, Mantel's test for scored categories, and Fisher's exact test for binary responses. The method embeds the usual randomization reference distribution in a one-parameter family of departures involving an unobserved covariate that would have been controlled by adjustments had it been observed. As this parameter is varied, the sensitivity of permutational significance levels and confidence intervals is displayed. Brief discussion of an example shows that comparisons vary considerably in their degree of sensitivity to unobserved biases.