Struct FileWriter

pub struct FileWriter<W> {
    writer: W,
    write_options: IpcWriteOptions,
    schema: Arc<Schema>,
    block_offsets: usize,
    dictionary_blocks: Vec<Block>,
    record_blocks: Vec<Block>,
    finished: bool,
    dictionary_tracker: DictionaryTracker,
    custom_metadata: HashMap<String, String>,
    data_gen: IpcDataGenerator,
}

Arrow File Writer

Writes Arrow RecordBatches in the IPC File Format.

See Also

• StreamWriter for writing IPC Streams

Example

let batch = record_batch!(("a", Int32, [1, 2, 3])).unwrap();
// create a new writer, the schema must be known in advance
let mut writer = FileWriter::try_new(&mut file, &batch.schema()).unwrap();
// write each batch to the underlying writer
writer.write(&batch).unwrap();
// When all batches are written, call finish to flush all buffers
writer.finish().unwrap();

Fields

writer: W

write_options: IpcWriteOptions

schema: Arc<Schema>

block_offsets: usize

dictionary_blocks: Vec<Block>

record_blocks: Vec<Block>

finished: bool

dictionary_tracker: DictionaryTracker

custom_metadata: HashMap<String, String>

data_gen: IpcDataGenerator

Implementations

impl<W> FileWriter<BufWriter<W>>

where W: Write,

pub fn try_new_buffered( writer: W, schema: &Schema, ) -> Result<FileWriter<BufWriter<W>>, ArrowError>

Try to create a new file writer with the writer wrapped in a BufWriter.

See FileWriter::try_new for an unbuffered version.

impl<W> FileWriter<W>

where W: Write,

pub fn try_new(writer: W, schema: &Schema) -> Result<FileWriter<W>, ArrowError>

Try to create a new writer, with the schema written as part of the header

Note the created writer is not buffered. See FileWriter::try_new_buffered for details.

Errors

An ‘Err’ may be returned if writing the header to the writer fails.

pub fn try_new_with_options( writer: W, schema: &Schema, write_options: IpcWriteOptions, ) -> Result<FileWriter<W>, ArrowError>

Try to create a new writer with IpcWriteOptions

Note the created writer is not buffered. See FileWriter::try_new_buffered for details.

Errors

An ‘Err’ may be returned if writing the header to the writer fails.

pub fn write_metadata( &mut self, key: impl Into<String>, value: impl Into<String>, )

Adds a key-value pair to the FileWriter’s custom metadata

pub fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>

Write a record batch to the file

pub fn finish(&mut self) -> Result<(), ArrowError>

Write footer and closing tag, then mark the writer as done

pub fn schema(&self) -> &Arc<Schema>

Returns the arrow SchemaRef for this arrow file.

pub fn get_ref(&self) -> &W

Gets a reference to the underlying writer.

pub fn get_mut(&mut self) -> &mut W

Gets a mutable reference to the underlying writer.

It is inadvisable to directly write to the underlying writer.

pub fn flush(&mut self) -> Result<(), ArrowError>

Flush the underlying writer.

Both the BufWriter and the underlying writer are flushed.

pub fn into_inner(self) -> Result<W, ArrowError>

Unwraps the underlying writer.

The writer is flushed and the FileWriter is finished before returning.

Errors

An ‘Err’ may be returned if an error occurs while finishing the StreamWriter or while flushing the writer.

Trait Implementations

impl<W> RecordBatchWriter for FileWriter<W>

where W: Write,

fn write(&mut self, batch: &RecordBatch) -> Result<(), ArrowError>

Write a single batch to the writer.

fn close(self) -> Result<(), ArrowError>

Write footer or termination data, then mark the writer as done.