pub trait SchemaMapper:
Debug
+ Send
+ Sync {
// Required methods
fn map_batch(&self, batch: RecordBatch) -> Result<RecordBatch>;
fn map_partial_batch(&self, batch: RecordBatch) -> Result<RecordBatch>;
}
Expand description
Maps, columns from a specific file schema to the table schema.
See DefaultSchemaAdapterFactory
for more details and examples.
Required Methods§
Sourcefn map_batch(&self, batch: RecordBatch) -> Result<RecordBatch>
fn map_batch(&self, batch: RecordBatch) -> Result<RecordBatch>
Adapts a RecordBatch
to match the table_schema
Sourcefn map_partial_batch(&self, batch: RecordBatch) -> Result<RecordBatch>
fn map_partial_batch(&self, batch: RecordBatch) -> Result<RecordBatch>
Adapts a RecordBatch
that does not have all the columns from the
file schema.
This method is used, for example, when applying a filter to a subset of
the columns as part of DataFusionArrowPredicate
when filter_pushdown
is enabled.
This method is slower than map_batch
as it looks up columns by name.