Parquet Bloom Filter ZoneMap Dictionary and Column Statistics Pruner Evaluator

Parquet Bloom Filter ZoneMap Dictionary and Column Statistics Pruner Evaluator | Online Free Tool | Elysia Tools

Tool usage guide

Learn when to use this tool, what it supports, and how real users apply it.

Key facts

Category: Data Analysis
Input types: file, textarea
Output type: html
Sample coverage: 4
API ready: Yes

Overview

The Parquet Bloom Filter ZoneMap Dictionary and Column Statistics Pruner Evaluator parses Apache Parquet Thrift footers to analyze metadata and measure predicate pushdown efficiency. It decodes column statistics into zone maps, executes XXH64 and split-block Bloom filter lookups, inspects dictionary encodings, and calculates exact row group skip savings in bytes for custom WHERE clauses.

When to use

When evaluating whether split-block Bloom filters or column min/max statistics are effectively pruning row groups on targeted queries.
When debugging query performance and scan bottlenecks across Parquet files written by engines like Arrow, DuckDB, or Spark.
When verifying dictionary encoding ratios and page-level column statistics before deploying analytical datasets to object storage.

How it works

1Upload a Parquet file to decode the Thrift compact protocol footer and parse column metadata across all row groups and page headers.
2Enter optional WHERE clauses with operators like =, !=, <, <=, >, >=, BETWEEN, or IN to test predicate pushdown rules.
3The engine performs XXH64 hashing and Split-Block Bloom Filter arithmetic against decoded zone maps and dictionary tables.
4Inspect the generated HTML report displaying per-row-group scan decisions, pruning causes, dictionary ratios, and byte savings based on total compressed size.

Use cases

Optimizing data lake layouts by measuring the pruning efficacy of sorting keys against column statistics and zone maps.
Auditing newly generated Parquet datasets to ensure Split-Block Bloom filters are properly serialized and non-empty.
Estimating network and I/O byte reduction when applying selective WHERE clauses against partition files in cloud storage.

Examples

1. Evaluate Bloom filter and zone map pruning on multi-row-group files

Data Platform Engineer

Background

A data engineer wants to verify whether point queries on an ID column are pruned by zone maps or Bloom filters across multiple row groups.

Problem

Unclear whether query engines will prune row groups or scan entire partitions for point lookups.

How to use

Upload the Parquet file and supply target WHERE clauses in the predicates text area.

id = 75
id > 60
name = 'zeta'

Outcome

Receives an HTML report showing row group 0 pruned by zone maps, Bloom filter hit details, and the exact byte savings achieved.

2. Audit dictionary encodings and zone maps on Arrow-written Parquet

Database Optimizer

Background

An analyst is checking column compression and dictionary encoding efficiency on an Arrow table export.

Problem

Need to confirm whether an equality filter falls within column min/max bounds and if missing Bloom filters cause full scans.

How to use

FAQ

Which SQL predicate operators are supported?

The evaluator supports =, !=, <, <=, >, >=, BETWEEN a AND b, and IN (x, y) predicates formatted as one clause per line.

How does the tool evaluate Split-Block Bloom Filters?

It extracts raw SBBF byte blocks from the footer/pages, hashes predicate literals using XXH64, and tests bit set membership directly.

What determines whether a row group is pruned or scanned?

A row group is pruned if the predicate falls outside min/max zone maps or fails the Bloom filter test; otherwise, it is marked for scan.

What is the maximum supported file upload size?

The tool accepts Parquet files up to 100 MB (104,857,600 bytes).

How are byte savings calculated for skipped row groups?

Byte savings are calculated by summing the total_compressed_size metadata values of row groups that are fully pruned by filters or zone maps.

product = 'Keyboard'

What this tool helps you do

Run this tool

Examples that match this tool

Continue with connected tools and hubs

Learn when to use this tool, what it supports, and how real users apply it.

Key facts

Overview

When to use

How it works

Use cases

Examples

1. Evaluate Bloom filter and zone map pruning on multi-row-group files

2. Audit dictionary encodings and zone maps on Arrow-written Parquet

FAQ

Parquet Pruning Samples

Android Image Processing Java Samples

Android Image Processing Kotlin Samples

Web Image Processing Python Samples

Social Media Template Resize Grid Composer

HDR AVIF to SDR PNG Tone Mapper

Batch Image Compress

Data URI Generator