Module datafusion::physical_optimizer

This module contains a query optimizer that operates against a physical plan and applies rules to a physical plan, such as “Repartition”.

Re-exports

pub use optimizer::PhysicalOptimizerRule;

Modules

aggregate_statistics

Utilizing exact statistics from sources to avoid scanning data

coalesce_batches

CoalesceBatches optimizer that groups batches together rows in bigger batches to avoid overhead with small batches

enforcement

Enforcement optimizer rules are used to make sure the plan’s Distribution and Ordering requirements are met by inserting necessary [RepartitionExec] and [SortExec].

join_selection

Select the proper PartitionMode and build side based on the avaliable statistics for hash join.

optimizer

Physical optimizer traits

pruning

This module contains code to prune “containers” of row groups based on statistics prior to execution. This can lead to significant performance improvements by avoiding the need to evaluate a plan on entire containers (e.g. an entire file)

repartition

Repartition optimizer that introduces repartition nodes to increase the level of parallelism available