Struct PullUpCorrelatedExpr

pub struct PullUpCorrelatedExpr {
    pub join_filters: Vec<Expr>,
    pub correlated_subquery_cols_map: HashMap<LogicalPlan, BTreeSet<Column>>,
    pub in_predicate_opt: Option<Expr>,
    pub exists_sub_query: bool,
    pub can_pull_up: bool,
    pub need_handle_count_bug: bool,
    pub collected_count_expr_map: HashMap<LogicalPlan, HashMap<String, Expr>>,
    pub pull_up_having_expr: Option<Expr>,
    pub pulled_up_scalar_agg: bool,
    /* private fields */
}

This struct rewrite the sub query plan by pull up the correlated expressions(contains outer reference columns) from the inner subquery’s ‘Filter’. It adds the inner reference columns to the ‘Projection’ or ‘Aggregate’ of the subquery if they are missing, so that they can be evaluated by the parent operator as the join condition.

Fields

join_filters: Vec<Expr>

correlated_subquery_cols_map: HashMap<LogicalPlan, BTreeSet<Column>>

mapping from the plan to its holding correlated columns

in_predicate_opt: Option<Expr>

exists_sub_query: bool

Is this an Exists(Not Exists) SubQuery. Defaults to FALSE

can_pull_up: bool

Can the correlated expressions be pulled up. Defaults to TRUE

need_handle_count_bug: bool

Do we need to handle the count bug during the pull up process.

The “count bug” was described in Optimization of Nested SQL Queries Revisited. This bug is not specific to the COUNT function, and it can occur with any aggregate function, such as SUM, AVG, etc. The anomaly arises because aggregates fail to distinguish between an empty set and null values when optimizing a correlated query as a join. Here, we use “the count bug” to refer to all such cases.

collected_count_expr_map: HashMap<LogicalPlan, HashMap<String, Expr>>

mapping from the plan to its expressions’ evaluation result on empty batch

pull_up_having_expr: Option<Expr>

pull up having expr, which must be evaluated after the Join

pulled_up_scalar_agg: bool

whether we have converted a scalar aggregation into a group aggregation. When unnesting lateral joins, we need to produce a left outer join in such cases.

Implementations

impl PullUpCorrelatedExpr

pub fn new() -> PullUpCorrelatedExpr

pub fn with_need_handle_count_bug( self, need_handle_count_bug: bool, ) -> PullUpCorrelatedExpr

Set if we need to handle the count bug during the pull up process

pub fn with_in_predicate_opt( self, in_predicate_opt: Option<Expr>, ) -> PullUpCorrelatedExpr

Set the in_predicate_opt

pub fn with_exists_sub_query( self, exists_sub_query: bool, ) -> PullUpCorrelatedExpr

Set if this is an Exists(Not Exists) SubQuery

Trait Implementations

impl Debug for PullUpCorrelatedExpr

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for PullUpCorrelatedExpr

fn default() -> PullUpCorrelatedExpr

Returns the “default value” for a type.

impl TreeNodeRewriter for PullUpCorrelatedExpr

type Node = LogicalPlan

The node type which is rewritable.

fn f_down( &mut self, plan: LogicalPlan, ) -> Result<Transformed<LogicalPlan>, DataFusionError>

Invoked while traversing down the tree before any children are rewritten. Default implementation returns the node as is and continues recursion.

fn f_up( &mut self, plan: LogicalPlan, ) -> Result<Transformed<LogicalPlan>, DataFusionError>

Invoked while traversing up the tree after all children have been rewritten. Default implementation returns the node as is and continues recursion.