Module datafusion::common::arrow::json::writer

JSON Writer

This JSON writer converts Arrow RecordBatches into arrays of JSON objects or JSON formatted byte streams.

Writing JSON Objects

To serialize RecordBatches into array of JSON objects, use record_batches_to_json_rows:

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

let json_rows = arrow_json::writer::record_batches_to_json_rows(&[&batch]).unwrap();
assert_eq!(
    serde_json::Value::Object(json_rows[1].clone()),
    serde_json::json!({"a": 2}),
);

Writing JSON formatted byte streams

To serialize RecordBatches into line-delimited JSON bytes, use LineDelimitedWriter:

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as JSON
let buf = Vec::new();
let mut writer = arrow_json::LineDelimitedWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();

// Get the underlying buffer back,
let buf = writer.into_inner();
assert_eq!(r#"{"a":1}
{"a":2}
{"a":3}
"#, String::from_utf8(buf).unwrap())

To serialize RecordBatches into a well formed JSON array, use ArrayWriter:

use arrow_schema::{DataType, Field, Schema};

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as a JSON array
let buf = Vec::new();
let mut writer = arrow_json::ArrayWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();

// Get the underlying buffer back,
let buf = writer.into_inner();
assert_eq!(r#"[{"a":1},{"a":2},{"a":3}]"#, String::from_utf8(buf).unwrap())

LineDelimitedWriter and ArrayWriter will omit writing keys with null values. In order to explicitly write null values for keys, configure a custom Writer by using a WriterBuilder to construct a Writer.

Structs

• JsonArray
  Produces JSON output as a single JSON array.
• LineDelimited
  Produces JSON output with one record per line.
• Writer
  A JSON writer which serializes RecordBatches to a stream of u8 encoded JSON objects.
• WriterBuilder
  JSON writer builder.

Traits

• JsonFormat
  This trait defines how to format a sequence of JSON objects to a byte stream.

Functions

• array_to_json_array
  Converts an arrow Array into a Vec of Serde JSON serde_json::Value’s
• record_batches_to_json_rows
  Converts an arrow RecordBatch into a Vec of Serde JSON JsonMaps (objects)

Type Aliases

• ArrayWriter
  A JSON writer which serializes RecordBatches to JSON arrays.
• LineDelimitedWriter
  A JSON writer which serializes RecordBatches to newline delimited JSON objects.