Struct datafusion::datasource::file_format::csv::CsvFormat

pub struct CsvFormat { /* private fields */ }

Character Separated Value FileFormat implementation.

Implementations

impl CsvFormat

pub fn with_schema_infer_max_rec(self, max_rec: Option<usize>) -> Self

Set a limit in terms of records to scan to infer the schema

• default to DEFAULT_SCHEMA_INFER_MAX_RECORD

pub fn with_has_header(self, has_header: bool) -> Self

Set true to indicate that the first line is a header.

• default to true

pub fn has_header(&self) -> bool

True if the first line is a header.

pub fn with_delimiter(self, delimiter: u8) -> Self

The character separating values within a row.

• default to ‘,’

pub fn with_file_compression_type( self, file_compression_type: FileCompressionType ) -> Self

Set a FileCompressionType of CSV

• defaults to FileCompressionType::UNCOMPRESSED

pub fn delimiter(&self) -> u8

The delimiter character.

Trait Implementations

impl Debug for CsvFormat

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CsvFormat

fn default() -> Self

Returns the “default value” for a type.

impl FileFormat for CsvFormat

fn as_any(&self) -> &dyn Any

Returns the table provider as Any so that it can be downcast to a specific implementation.

fn infer_schema<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, _state: &'life1 SessionState, store: &'life2 Arc<dyn ObjectStore>, objects: &'life3 [ObjectMeta] ) -> Pin<Box<dyn Future<Output = Result<SchemaRef>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Infer the common schema of the provided objects. The objects will usually be analysed up to a given number of records or files (as specified in the format config) then give the estimated common schema. This might fail if the files have schemas that cannot be merged.

fn infer_stats<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, _state: &'life1 SessionState, _store: &'life2 Arc<dyn ObjectStore>, _table_schema: SchemaRef, _object: &'life3 ObjectMeta ) -> Pin<Box<dyn Future<Output = Result<Statistics>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Infer the statistics for the provided object. The cost and accuracy of the estimated statistics might vary greatly between file formats.

fn create_physical_plan<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _state: &'life1 SessionState, conf: FileScanConfig, _filters: Option<&'life2 Arc<dyn PhysicalExpr>> ) -> Pin<Box<dyn Future<Output = Result<Arc<dyn ExecutionPlan>>> + Send + 'async_trait>>where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Take a list of files and convert it to the appropriate executor according to this file format.