Module datafusion::datasource

DataFusion data sources

Re-exports

• pub use self::datasource::TableProvider;
• pub use self::default_table_source::provider_as_source;
• pub use self::default_table_source::source_as_provider;
• pub use self::default_table_source::DefaultTableSource;
• pub use self::memory::MemTable;
• pub use self::view::ViewTable;

Modules

• datasource
  Data source traits
• default_table_source
  Default TableSource implementation used in DataFusion physical plans
• empty
  An empty plan that is usefull for testing and generating plans without mapping them to actual data.
• file_format
  Module containing helper methods for the various file formats
• listing
  A table that uses the ObjectStore listing capability to get the list of files to process.
• listing_table_factory
  Factory for creating ListingTables with default options
• memory
  In-memory data source for presenting a Vec<RecordBatch> as a data source that can be queried by DataFusion. This allows data to be pre-loaded into memory and then repeatedly queried without incurring additional file I/O overhead.
• object_store
  ObjectStoreRegistry holds all the object stores at Runtime with a scheme for each store. This allows the user to extend DataFusion with different storage systems such as S3 or HDFS and query data inside these systems.
• streaming
  A simplified TableProvider for streaming partitioned datasets
• view
  View data source which uses a LogicalPlan as it’s input.

Enums

• TableType
  Indicates the type of this table for metadata/catalog purposes.

Functions

• get_statistics_with_limit
  Get all files as well as the file level summary statistics (no statistic for partition columns). If the optional limit is provided, includes only sufficient files. Needed to read up to limit number of rows.