Nemenyi Critical-Difference & Friedman Rank Plotter

Nemenyi Critical-Difference & Friedman Rank Plotter | Online Free Tool | Elysia Tools

Tool usage guide

Learn when to use this tool, what it supports, and how real users apply it.

Key facts

Category: Data Analysis
Input types: textarea, select, text
Output type: html
Sample coverage: 4
API ready: Yes

Overview

The Nemenyi Critical-Difference & Friedman Rank Plotter evaluates machine learning benchmark results across multiple datasets using rigorous non-parametric statistical testing. It computes Friedman and Iman-Davenport test statistics with exact p-values, calculates average ranks, and generates classic Demšar Critical Difference (CD) diagrams with non-significant cliques or Bonferroni-Dunn control comparisons.

When to use

Comparing multiple machine learning classifiers or algorithms across multiple benchmark datasets.
Assessing whether performance gains over a baseline model are statistically significant under a control comparison.
Generating Demšar Critical Difference (CD) diagrams and rank tables for academic research papers.

How it works

1Paste a CSV-formatted score matrix where columns represent models, rows represent datasets, and cells contain evaluation metrics.
2Select the metric direction (higher is better for accuracy/AUC or lower is better for RMSE/loss) and the significance level α (0.05 or 0.10).
3Choose a post-hoc procedure (Nemenyi for all-pairs or Bonferroni-Dunn against a named control model).
4Review the resulting Friedman/Iman-Davenport test statistics, average rank tables, pairwise significance verdicts, and interactive CD diagram with connecting clique bars.

Use cases

Algorithm benchmarking in academic papers following Demšar's standard statistical comparison methodology.
Model selection in industrial ML pipelines to confirm candidate architecture improvements over production baselines.
Hyperparameter tuning analysis across cross-validation folds or disparate evaluation test beds.

Examples

1. Multi-Classifier Benchmark with Nemenyi All-Pairs

Machine Learning Researcher

Background

Evaluating four classification algorithms across twelve benchmark datasets using classification accuracy.

Problem

Determine if AdaBoost performs significantly better than SVM, kNN, and C4.5 across all datasets.

How to use

Input the 12-row dataset score matrix, set metric direction to 'higher', significance level to 0.05, and select Nemenyi post-hoc testing.

matrix: 'dataset, AdaBoost, SVM, kNN, C4.5\n...' | direction: 'higher' | alpha: '0.05' | posthoc: 'nemenyi'

Outcome

Friedman test rejects the null hypothesis (χ²_F = 32.4, p < 0.001); CD diagram isolates AdaBoost as significantly superior and groups SVM and kNN into a single non-significant clique.

2. Regression Baseline Comparison with Bonferroni-Dunn

Data Scientist

Background

Testing two newly tuned regression models against a production baseline across eight distinct operational datasets using RMSE.

Problem

Verify whether either challenger provides a statistically significant error reduction compared to the existing baseline.

FAQ

What is the difference between Nemenyi and Bonferroni-Dunn post-hoc tests?

Nemenyi compares all pairs of algorithms against each other, whereas Bonferroni-Dunn compares all candidate models against a single specified control baseline.

How does the metric direction affect model ranking?

Selecting 'Higher is better' assigns rank 1 to the highest value, while 'Lower is better' assigns rank 1 to the lowest numerical score per dataset.

What do the horizontal bars (cliques) mean on the CD diagram?

Horizontal clique bars connect groups of algorithms whose difference in average rank is smaller than the calculated Critical Difference (CD), meaning they are not statistically significantly different.

Why are both Friedman and Iman-Davenport tests computed?

The Iman-Davenport correction provides an F-distribution approximation that is less conservative and often more reliable than the standard Friedman chi-square statistic.

What format should the score matrix be in?

matrix: 'dataset, Baseline, ChallengerA, ChallengerB\n...' | direction: 'lower' | alpha: '0.05' | posthoc: 'bonferroni-dunn' | control: 'Baseline'

Run this tool

Examples that match this tool

Continue with connected tools and hubs

Learn when to use this tool, what it supports, and how real users apply it.

Key facts

Overview

When to use

How it works

Use cases

Examples

1. Multi-Classifier Benchmark with Nemenyi All-Pairs

2. Regression Baseline Comparison with Bonferroni-Dunn

FAQ

Web Image Processing Python Samples

Android Image Processing Java Samples

Android Image Processing Kotlin Samples

Web Image Processing Rust Samples

Kurtosis Analyzer

Correlation Analyzer

Pivot Table Generator

Regression Analyzer