Distribution-Free Analysis for Stratified Oneway Layouts
The novel stats::free1way() function implements semiparametrically efficient population- and permutation-based inference in distribution-free stratified K-sample oneway layouts. Some classical stats::*.test() procedures gain effect estimates and corresponding confidence intervals as well as tools for power analysis and model diagnostics.
R Core
stats
Author
Torsten Hothorn, Kurt Hornik
Published
November 7, 2026
Introduction
Since more than two decades, the stats package serves as a home for well-established statistical analysis tools, including linear and generalized linear models or a versatile collection of statistical test procedures featuring *.test() user interfaces. In some cases, such as t.test(), var.test(), or binom.test(), the null and alternative hypotheses are formulated in terms of a parametric model. Model diagnostics and power analysis is relatively simple in this situation and corresponding functionality is provided by stats, for example as qqnorm(), power.t.test() etc.
Other procedures are fully nonparametric. The Smirnov two-sample test, for example, allows testing the null of equal distributions against arbitrary alternatives in the two-sample oneway layout via ks.test().
For yet another group of test procedures it is harder to understand the underlying model. This makes model diagnostics and also power analysis much more challenging, which explains the absence of, for example, power.wilcox.test() or power.friedman.test() from the stats package.
Beginning with R version 4.6.0, the novel free1way() function provides a unified interface to semiparametrically efficient population and permutation inference in distribution-free stratified K-sample oneway layouts. This post explains the rationale of this model class and discusses how existing test procedures, for example kruskal.test(), are enhanced by this new tool.
Distribution-free oneway layouts
When comparing the distributions of some outcome in certain experimental groups, we might refrain from making assumptions regarding the distribution function in one of these groups (typically a control group corresponding to the first factor level, say \(F_0\)) while, at the same time, assuming that the distribution functions in the remaining groups are simple transformations of this control distribution characterized by a single parameter. Under this model, we assume that the distribution function in the second group \(F_1\) is a function of distribution function in the control group \(F_0\) and a parameter \(\tau\). For example \(F_1 = F_0^{\exp(-\tau)}\), with \(\tau\) being our unknown treatment effect parameter to be estimated.
This idea, going back to Erich Lehmann’s 1953 paper on The Power of Rank Tests, allows the formulation of many statistical tests for the composite null \(\tau = 0\) or one-sided alternatives. As an example, consider the Wilcoxon-Mann-Whitney rank sum test. It turns out that this classical test is the score test under the proportional-odds model \[
\frac{F_1}{1 - F_1} = \exp(-\tau) \frac{F_0}{1 - F_0}
\] whose parameters \(F_0\) and \(\tau\) can be estimated by maximising the empirical log-likelihood. If more than two samples are to be compared, the model corresponds to the Kruskal-Wallis test.
As an illustration, consider the example presented in the manual page of kruskal.test():
## Hollander & Wolfe (1973), 116.## Mucociliary efficiency from the rate of removal of dust in normal## subjects, subjects with obstructive airway disease, and subjects## with asbestosis.example(kruskal.test, echo =FALSE)
Kruskal-Wallis rank sum test
data: x by g
Kruskal-Wallis chi-squared = 0.77143, df = 2, p-value = 0.68
(ft <-free1way(x ~ g))
3-sample Kruskal-Wallis test against proportional odds alternatives
data: x by g (Normal, COPD, Asbestosis)
Perm chi-squared = 0.77143, df = 2, p-value = 0.68
alternative hypothesis: true log-odds ratio is not equal to 0
The test statistics and p-values are identical, but the print method of the latter procedure informs about the alternative this test is powerful (in fact, the locally most powerful test) against.
In contrast to the classical implementation in kruskal.test() (where coef(kt) comes back empty handed), the novel free1way() is aware of the semiparametric nature of the model and allows to access the estimated log-odds ratios along with a confidence interval (a Rao score interval is presented here):
The data analysis could also question the sample size needed to demonstrate an overall effect when the true log-odds ratios are close to the one estimated from this dataset
power.free1way.test(delta =coef(ft), power = .8)
3-sample Kruskal-Wallis test against proportional odds alternatives
n = 30
Total sample size = 30 (Control) + 30 (gCOPD) + 30 (gAsbestosis) = 90
power = 0.8002641
sig.level = 0.05
log-odds ratio = 1.0350087, -0.3706949
NOTE: 'n' is sample size in control group
Features
Binary, ordered categorical, numeric and possibly right-censored outcomes can be analyzed using free1way(). In stratified designs, strata can be specified as blocks. Wald, likelihood-ratio and Rao score tests and confidence intervals are available. In addition, permutation tests (asymptotic and resampling-based, exact for two-sample proportional odds models) can be performed to test hypotheses. Multiple comparisons can be conducted via an interface to the multcomp package. Similar to Firth’s method in binary GLMs, penalisation via Jeffreys’ prior is implemented.
Resources
In addition to the documentation, examples, and tests provided in the stats package, a dedicated free1way.docreg package features two vignettes explaining the implementation and statistical framework in depths, along additional regression tests comparing the implementation against benchmarks such as rms::orm() (rms package), MASS::polr() (MASS package), or tram::Polr() (tram package).