re_viewer::external::arrow

Crate ipc

Expand description

Support for the Arrow IPC Format

The Arrow IPC format defines how to read and write RecordBatches to/from a file or stream of bytes. This format can be used to serialize and deserialize data to files and over the network.

There are two variants of the IPC format:

  1. IPC Streaming Format: Supports streaming data sources, implemented by StreamReader and StreamWriter

  2. IPC File Format: Supports random access, implemented by FileReader and FileWriter.

See the reader and writer modules for more information.

Modules§

  • convert
    Utilities for converting between IPC types and native Arrow types
  • gen
    Generated code
  • reader
    Arrow IPC File and Stream Readers
  • writer
    Arrow IPC File and Stream Writers

Structs§

Enums§

Constants§

Functions§

  • finish_footer_buffer
  • finish_message_buffer
  • finish_schema_buffer
  • finish_size_prefixed_footer_buffer
  • finish_size_prefixed_message_buffer
  • finish_size_prefixed_schema_buffer
  • finish_size_prefixed_sparse_tensor_buffer
  • finish_size_prefixed_tensor_buffer
  • finish_sparse_tensor_buffer
  • finish_tensor_buffer
  • root_as_footer
    Verifies that a buffer of bytes contains a Footer and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_footer_unchecked.
  • root_as_footer_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a Footer and returns it.
  • root_as_footer_with_opts
    Verifies, with the given options, that a buffer of bytes contains a Footer and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_footer_unchecked.
  • root_as_message
    Verifies that a buffer of bytes contains a Message and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_message_unchecked.
  • root_as_message_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a Message and returns it.
  • root_as_message_with_opts
    Verifies, with the given options, that a buffer of bytes contains a Message and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_message_unchecked.
  • root_as_schema
    Verifies that a buffer of bytes contains a Schema and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_schema_unchecked.
  • root_as_schema_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a Schema and returns it.
  • root_as_schema_with_opts
    Verifies, with the given options, that a buffer of bytes contains a Schema and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_schema_unchecked.
  • root_as_sparse_tensor
    Verifies that a buffer of bytes contains a SparseTensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_sparse_tensor_unchecked.
  • root_as_sparse_tensor_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a SparseTensor and returns it.
  • root_as_sparse_tensor_with_opts
    Verifies, with the given options, that a buffer of bytes contains a SparseTensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_sparse_tensor_unchecked.
  • root_as_tensor
    Verifies that a buffer of bytes contains a Tensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_tensor_unchecked.
  • root_as_tensor_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a Tensor and returns it.
  • root_as_tensor_with_opts
    Verifies, with the given options, that a buffer of bytes contains a Tensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_tensor_unchecked.
  • size_prefixed_root_as_footer
    Verifies that a buffer of bytes contains a size prefixed Footer and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use size_prefixed_root_as_footer_unchecked.
  • size_prefixed_root_as_footer_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a size prefixed Footer and returns it.
  • size_prefixed_root_as_footer_with_opts
    Verifies, with the given verifier options, that a buffer of bytes contains a size prefixed Footer and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_footer_unchecked.
  • size_prefixed_root_as_message
    Verifies that a buffer of bytes contains a size prefixed Message and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use size_prefixed_root_as_message_unchecked.
  • size_prefixed_root_as_message_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a size prefixed Message and returns it.
  • size_prefixed_root_as_message_with_opts
    Verifies, with the given verifier options, that a buffer of bytes contains a size prefixed Message and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_message_unchecked.
  • size_prefixed_root_as_schema
    Verifies that a buffer of bytes contains a size prefixed Schema and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use size_prefixed_root_as_schema_unchecked.
  • size_prefixed_root_as_schema_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a size prefixed Schema and returns it.
  • size_prefixed_root_as_schema_with_opts
    Verifies, with the given verifier options, that a buffer of bytes contains a size prefixed Schema and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_schema_unchecked.
  • size_prefixed_root_as_sparse_tensor
    Verifies that a buffer of bytes contains a size prefixed SparseTensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use size_prefixed_root_as_sparse_tensor_unchecked.
  • size_prefixed_root_as_sparse_tensor_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a size prefixed SparseTensor and returns it.
  • size_prefixed_root_as_sparse_tensor_with_opts
    Verifies, with the given verifier options, that a buffer of bytes contains a size prefixed SparseTensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_sparse_tensor_unchecked.
  • size_prefixed_root_as_tensor
    Verifies that a buffer of bytes contains a size prefixed Tensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use size_prefixed_root_as_tensor_unchecked.
  • size_prefixed_root_as_tensor_unchecked⚠
    Assumes, without verification, that a buffer of bytes contains a size prefixed Tensor and returns it.
  • size_prefixed_root_as_tensor_with_opts
    Verifies, with the given verifier options, that a buffer of bytes contains a size prefixed Tensor and returns it. Note that verification is still experimental and may not catch every error, or be maximally performant. For the previous, unchecked, behavior use root_as_tensor_unchecked.