# Nemenyi Critical-Difference & Friedman Rank Plotter

Friedman and Iman-Davenport tests with exact p-values, average ranks, the Nemenyi critical difference and the classic CD diagram with cliques — plus an optional Bonferroni-Dunn overlay against a control model.

> Canonical page: https://elysiatools.com/en/tools/nemenyi-critical-difference-friedman-rank-plotter

- **Category:** Data Analysis

- **Keywords:** nemenyi, friedman test, critical difference, cd diagram, demsar, benchmark comparison

## Overview

The Nemenyi Critical-Difference & Friedman Rank Plotter evaluates machine learning benchmark results across multiple datasets using rigorous non-parametric statistical testing. It computes Friedman and Iman-Davenport test statistics with exact p-values, calculates average ranks, and generates classic Demšar Critical Difference (CD) diagrams with non-significant cliques or Bonferroni-Dunn control comparisons.

## Inputs

- **Score matrix (header: dataset, model names; rows: scores)** (textarea): dataset, Random Forest, XGBoost, MLP, Logistic dataset-a, 0.91, 0.90, 0.88, 0.84 dataset-b, 0.82, 0.85, 0.80, 0.79
- **Metric direction** (select)
- **Significance level α** (select)
- **Post-hoc procedure** (select)
- **Control model (Bonferroni-Dunn)** (text): exact column name of the baseline model

## When to use

- Comparing multiple machine learning classifiers or algorithms across multiple benchmark datasets.
- Assessing whether performance gains over a baseline model are statistically significant under a control comparison.
- Generating Demšar Critical Difference (CD) diagrams and rank tables for academic research papers.

## How it works

- Paste a CSV-formatted score matrix where columns represent models, rows represent datasets, and cells contain evaluation metrics.
- Select the metric direction (higher is better for accuracy/AUC or lower is better for RMSE/loss) and the significance level α (0.05 or 0.10).
- Choose a post-hoc procedure (Nemenyi for all-pairs or Bonferroni-Dunn against a named control model).
- Review the resulting Friedman/Iman-Davenport test statistics, average rank tables, pairwise significance verdicts, and interactive CD diagram with connecting clique bars.

## Use cases

- Algorithm benchmarking in academic papers following Demšar's standard statistical comparison methodology.
- Model selection in industrial ML pipelines to confirm candidate architecture improvements over production baselines.
- Hyperparameter tuning analysis across cross-validation folds or disparate evaluation test beds.

## Frequently asked questions

### What is the difference between Nemenyi and Bonferroni-Dunn post-hoc tests?

Nemenyi compares all pairs of algorithms against each other, whereas Bonferroni-Dunn compares all candidate models against a single specified control baseline.

### How does the metric direction affect model ranking?

Selecting 'Higher is better' assigns rank 1 to the highest value, while 'Lower is better' assigns rank 1 to the lowest numerical score per dataset.

### What do the horizontal bars (cliques) mean on the CD diagram?

Horizontal clique bars connect groups of algorithms whose difference in average rank is smaller than the calculated Critical Difference (CD), meaning they are not statistically significantly different.

### Why are both Friedman and Iman-Davenport tests computed?

The Iman-Davenport correction provides an F-distribution approximation that is less conservative and often more reliable than the standard Friedman chi-square statistic.

### What format should the score matrix be in?

Provide comma-separated values (CSV) with a header row containing 'dataset' followed by model names, and rows containing dataset identifiers followed by numerical scores.

## Related tools

- [Kurtosis Analyzer](https://elysiatools.com/en/tools/kurtosis-analyzer): Analyze data kurtosis to measure the "tailedness" of distribution and detect heavy-tailed or light-tailed patterns
- [Correlation Analyzer](https://elysiatools.com/en/tools/correlation-analyzer): Advanced correlation analysis tool that calculates correlation coefficients between variables to measure the strength and direction of their linear relationships. Perfect for statistical analysis, financial modeling, scientific research, and data exploration. Features: - Multiple correlation methods (Pearson, Spearman, Kendall) - Correlation matrix generation - Statistical significance testing (p-values) - Confidence intervals calculation - Heatmap visualization - Scatter plot matrix generation - Missing value handling strategies - Outlier detection and handling - Group analysis capabilities - Detailed statistical reports Common Use Cases: - Financial market analysis and risk assessment - Scientific research and hypothesis testing - Customer behavior and marketing analysis - Healthcare and medical data analysis - Quality control and process optimization - Educational performance evaluation
- [Pivot Table Generator](https://elysiatools.com/en/tools/pivot-table-generator): Generate interactive pivot tables from CSV/JSON data with customizable row, column, and value field configurations
- [Regression Analyzer](https://elysiatools.com/en/tools/regression-analyzer): Advanced regression analysis tool for performing linear regression analysis, calculating regression statistics, and making predictions. Perfect for statistical modeling, trend analysis, forecasting, and understanding relationships between variables. Features: - Simple linear regression (y = mx + b) - Multiple linear regression support - Regression coefficients calculation - Statistical significance testing - R-squared and adjusted R-squared - Residual analysis and diagnostics - Prediction intervals and confidence intervals - Outlier detection in regression - Model validation metrics - Visual regression diagnostics - Data transformation support Common Use Cases: - Sales forecasting and trend analysis - Financial modeling and risk assessment - Scientific research and hypothesis testing - Quality control and process optimization - Marketing analytics and ROI analysis - Medical and biological research
- [Skewness Analyzer](https://elysiatools.com/en/tools/skewness-analyzer): Analyze data skewness to measure the asymmetry of probability distribution and identify data patterns
- [AI Domain Name Idea Generator](https://elysiatools.com/en/tools/ai-domain-name-idea-generator): Generate up to 10 creative domain names with AI and filter out already-registered ones via DNS + RDAP.
- [Buoyancy Calculator (Archimedes, F = ρ·V·g)](https://elysiatools.com/en/tools/buoyancy-calculator): Compute buoyant force, fluid density, or displaced volume from any two, with optional float/sink analysis
- [Cohort Retention & RFM Funnel Analyzer](https://elysiatools.com/en/tools/cohort-retention-rfm-funnel-analyzer): Paste an order or event CSV and get the growth stack in one page: a cohort retention triangle with a heatmap, RFM quintile scoring mapped to the ten standard lifecycle segments, and a strict-order funnel.

## Samples

- [Web Image Processing Python Samples](https://elysiatools.com/en/samples/web-image-processing-python): Web Python image processing examples using PIL/Pillow including reading, saving, resizing, and format conversion
- [Android Image Processing Java Samples](https://elysiatools.com/en/samples/android-image-processing-java): Android Java image processing examples including reading/saving images, scaling, and format conversion
- [Android Image Processing Kotlin Samples](https://elysiatools.com/en/samples/android-image-processing-kotlin): Android Kotlin image processing examples including reading/saving images, scaling, and format conversion
- [Web Image Processing Rust Samples](https://elysiatools.com/en/samples/web-image-processing-rust): Web Rust image processing examples including image read/save, scaling, and format conversion