Module datafusion::datasource::physical_plan

Execution plans that read file formats

Re-exports

• pub use self::parquet::ParquetExec;
• pub use self::parquet::ParquetFileMetrics;
• pub use self::parquet::ParquetFileReaderFactory;

Modules

• parquet
  Execution plan for reading Parquet files

Structs

• ArrowExec
  Execution plan for scanning Arrow data source
• AvroExec
  Execution plan for scanning Avro data source
• CsvConfig
  A Config for CsvOpener
• CsvExec
  Execution plan for scanning a CSV file
• CsvOpener
  A FileOpener that opens a CSV file and yields a FileOpenFuture
• FileGroupPartitioner
  Repartition input files into target_partitions partitions, if total file size exceed repartition_file_min_size
• FileMeta
  A single file or part of a file that should be read, along with its schema, statistics
• FileScanConfig
  The base configurations to provide when creating a physical plan for any given file format.
• FileSinkConfig
  The base configurations to provide when creating a physical plan for writing to any given file format.
• FileStream
  A stream that iterates record batch by record batch, file over file.
• JsonOpener
  A FileOpener that opens a JSON file and yields a FileOpenFuture
• NdJsonExec
  Execution plan for scanning NdJson data source
• SchemaMapping
  The SchemaMapping struct holds a mapping from the file schema to the table schema and any necessary type conversions that need to be applied.

Enums

• OnError
  Describes the behavior of the FileStream if file opening or scanning fails

Traits

• FileOpener
  Generic API for opening a file using an ObjectStore and resolving to a stream of RecordBatch

Functions

• wrap_partition_type_in_dict
  Convert type to a type suitable for use as a ListingTable partition column. Returns Dictionary(UInt16, val_type), which is a reasonable trade off between a reasonable number of partition values and space efficiency.
• wrap_partition_value_in_dict
  Convert a ScalarValue of partition columns to a type, as decribed in the documentation of wrap_partition_type_in_dict, which can wrap the types.

Type Aliases

• FileOpenFuture
  A fallible future that resolves to a stream of RecordBatch