Struct datafusion::execution::options::ParquetReadOptions

pub struct ParquetReadOptions<'a> {
    pub file_extension: &'a str,
    pub table_partition_cols: Vec<String>,
    pub parquet_pruning: bool,
    pub skip_metadata: bool,
}

Options that control the reading of Parquet files.

Note this structure is supplied when a datasource is created and can not not vary from statement to statement. For settings that can vary statement to statement see ConfigOptions.

Fields

file_extension: &'a str

File extension; only files with this extension are selected for data input. Defaults to “.parquet”.

table_partition_cols: Vec<String>

Partition Columns

parquet_pruning: bool

Should DataFusion parquet reader use the predicate to prune data, overridden by value on execution::context::SessionConfig

skip_metadata: bool

Tell the parquet reader to skip any metadata that may be in the file Schema. This can help avoid schema conflicts due to metadata. Defaults to true.

Implementations

impl<'a> ParquetReadOptions<'a>

pub fn parquet_pruning(self, parquet_pruning: bool) -> Self

Specify parquet_pruning

pub fn skip_metadata(self, skip_metadata: bool) -> Self

Tell the parquet reader to skip any metadata that may be in the file Schema. This can help avoid schema conflicts due to metadata. Defaults to true.

pub fn table_partition_cols(self, table_partition_cols: Vec<String>) -> Self

Specify table_partition_cols for partition pruning

pub fn to_listing_options(&self, target_partitions: usize) -> ListingOptions

Helper to convert these user facing options to ListingTable options

Trait Implementations

impl<'a> Clone for ParquetReadOptions<'a>

fn clone(&self) -> ParquetReadOptions<'a>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Default for ParquetReadOptions<'a>

fn default() -> Self

Returns the “default value” for a type.