Test Wizard
What do you want to evaluate?
Differences
Test for differences between groups/variablesDifferences
Differences / Categorical outcomes
Do you want to compare 2 or more groups/variables or compare the mean/median of 1 variable to a population value/0?
2 or more groups1 group
Differences / Categorical outcomes / 2 or more groups
For independent variables/outcomes (default) use:
Pearson's chi-squared test.
Use the Fisher's exact test instead if any outcome has less then 10 counts.
For dependent variables/outcomes (clustered dat) use:
McNemar's test
Differences / Categorical outcomes / 1 group
You can use:
Binomial or Chi-squared goodness of fit test.
Differences / Continuous outcomes
Do you want to compare 2 or more groups/Variables or compare the mean/median of 1 variable to a population value/0?
Differences / Continuous outcomes / 2 groups
Are the groups unrelated (independent/non-clustered data)?
Differences / Continuous outcomes / 2 groups / Independent
For normally distributeed outcome (default) use:
2-sample Student's t-test.
For non-normally distributeed outcome use:
Wilcoxon rank-sum test (a.k.a Mann-Whitney U test).
Differences / Continuous outcomes / 2 groups / Dependent
For normally distributeed outcome (default) use:
Paired Student's t-test.
For non-normally distributeed outcome use:
Wilcoxon signed-rank test.
Differences / Continuous outcomes / >2 groups
For normally distributeed outcome (default) use:
One-way ANOVA.
For non-normally distributeed outcome use:
Kruskal-Wallis test.
NB: performing an analysis on more than 2 groups of dependent continuous data usually entails a repeated-measurement analysis on so-called longitudinal data. This either requires separate comparisons of each 2-time points or a complex type of analysis such as linear mixed effects regression modelling (learn more in the GCR academy).
Differences / Continuous outcomes / 1 group
For normally distributeed outcome (default) use:
1-sample Student's t-test.
For non-normally distributeed outcome use:
Sign test.
Association-Correlation
Are both variables dichotomous/categorical or continuous?
Association-Correlation / Categorical
You can use:
Pearson's chi-square test
Use Fisher's exact test if any of the outcomes has less then 10 counts.
You can also use:
Logistic regression analysis
Association-Correlation / Continuous
You can use:
Pearson's correlation coefficient
You can also use:
Simple linear regression analysis
NB: use this approach to evaluate whether two different variables are correlated. Evaluating the agreement/reliability between two similar variables requires a different approach.
Prediction
What is the nature of the outcome that you want to predict?
Prediction / Categorical
You can use:
Logistic regression analysis
Prediction / Continuous
You can use:
Linear regression analysis
Prediction / Survival
For a univariable or descriptive analysis use:
Kaplan-Meier analysis +/- Log-rank test
For a multivariable prediction model use:
Cox Proportional-Hazards model
Agreement-reliability
What is the nature of the outcome that you want to predict?
Nominal variables are unranked categorical, ordinal variables are ranked categorical.
Agreement-reliability / Nominal
For agreement use:
Proportions of (specific) agreement
For reliability use:
Cohen's kappa
Agreement-reliability / Nominal
For agreement use:
Proportions of (specific) agreement
For reliability use:
Cohen's kappa
NB: Reproducibility is the umbrella term for agreement and reliability.
Agreement is an evaluation of the measurement error (number of cases with agreement/number of total cases).
It answers the question: How close are scores for repeated measurements?
Reliability is an evaluation of discriminatory ability (measure of agreement corrected for agreement by chance).
It answers the question: How well can patients be distinguished from each other despite measurement error?
See the reproducibility research page in the study design aid for more info.
Agreement-reliability / Ordinal
For agreement use:
Proportions of (specific) agreement
For reliability use:
Ranked intraclass correlation
Matrix of kappa coefficients
Weighted kappa
NB: Reproducibility is the umbrella term for agreement and reliability.
Agreement is an evaluation of the measurement error (number of cases with agreement/number of total cases).
It answers the question: How close are scores for repeated measurements?
Reliability is an evaluation of discriminatory ability (measure of agreement corrected for agreement by chance).
It answers the question: How well can patients be distinguished from each other despite measurement error?
See the reproducibility research page in the study design aid for more info.
Agreement-reliability / Continuous
For agreement use:
Bland-Altman plot with bias and 95% limits of agreement
Standard error of measurement
Coefficients of variation
For reliability use:
Intraclass correlation coefficients
NB: Reproducibility is the umbrella term for agreement and reliability.
Agreement is an evaluation of the measurement error (number of cases with agreement/number of total cases).
It answers the question: How close are scores for repeated measurements?
Reliability is an evaluation of discriminatory ability (measure of agreement corrected for agreement by chance).
It answers the question: How well can patients be distinguished from each other despite measurement error?
See the reproducibility research page in the study design aid for more info.