Struct re_viewer::external::re_chunk_store::Chunk

pub struct Chunk {
    pub(crate) id: ChunkId,
    pub(crate) entity_path: EntityPath,
    pub(crate) heap_size_bytes: AtomicU64,
    pub(crate) is_sorted: bool,
    pub(crate) row_ids: StructArray,
    pub(crate) timelines: BTreeMap<Timeline, TimeColumn>,
    pub(crate) components: BTreeMap<ComponentName, ListArray<i32>>,
}

Dense arrow-based storage of N rows of multi-component multi-temporal data for a specific entity.

This is our core datastructure for logging, storing, querying and transporting data around.

The chunk as a whole is always ascendingly sorted by RowId before it gets manipulated in any way. Its time columns might or might not be ascendingly sorted, depending on how the data was logged.

This is the in-memory representation of a chunk, optimized for efficient manipulation of the data within. For transport, see crate::TransportChunk instead.

Fields

id: ChunkId

entity_path: EntityPath

heap_size_bytes: AtomicU64

is_sorted: bool

row_ids: StructArray

timelines: BTreeMap<Timeline, TimeColumn>

components: BTreeMap<ComponentName, ListArray<i32>>

Implementations

impl Chunk

pub fn builder(entity_path: EntityPath) -> ChunkBuilder

Initializes a new ChunkBuilder.

pub fn builder_with_id(id: ChunkId, entity_path: EntityPath) -> ChunkBuilder

Initializes a new ChunkBuilder.

The final Chunk will have the specified id.

impl Chunk

pub fn with_id(self, id: ChunkId) -> Chunk

Returns a version of us with a new ChunkId.

Reminder:

• The returned Chunk will re-use the exact same RowIds as self.
• Duplicated RowIds in the ChunkStore is undefined behavior.

pub fn are_similar(lhs: &Chunk, rhs: &Chunk) -> bool

Returns true is two Chunks are similar, although not byte-for-byte equal.

In particular, this ignores chunks and row IDs, as well as temporal timestamps.

Useful for tests.

pub fn are_equal_ignoring_extension_types(&self, other: &Chunk) -> bool

Check for equality while ignoring possible Extension type information

This is necessary because arrow2 loses the Extension datatype when deserializing back from the arrow_schema::DataType representation.

In theory we could fix this, but as we’re moving away from arrow2 anyways it’s unlikely worth the effort.

impl Chunk

pub fn clone_as(&self, id: ChunkId, first_row_id: RowId) -> Chunk

Clones the chunk and assign new IDs to the resulting chunk and its rows.

first_row_id will become the RowId of the first row in the duplicated chunk. Each row after that will be monotonically increasing.

pub fn into_static(self) -> Chunk

Clones the chunk into a new chunk without any time data.

pub fn zeroed(self) -> Chunk

Clones the chunk into a new chunk where all RowIds are RowId::ZERO.

pub fn time_range_per_component( &self ) -> BTreeMap<Timeline, BTreeMap<ComponentName, ResolvedTimeRange>>

Computes the time range covered by each individual component column on each timeline.

This is different from the time range covered by the Chunk as a whole because component columns are potentially sparse.

This is crucial for indexing and queries to work properly.

pub fn num_events_cumulative(&self) -> u64

The cumulative number of events in this chunk.

I.e. how many component batches (“cells”) were logged in total?

pub fn num_events_cumulative_per_unique_time( &self, timeline: &Timeline ) -> Vec<(TimeInt, u64)>

The cumulative number of events in this chunk for each unique timestamp.

I.e. how many component batches (“cells”) were logged in total at each timestamp?

Keep in mind that a timestamp can appear multiple times in a Chunk. This method will do a sum accumulation to account for these cases (i.e. every timestamp in the returned vector is guaranteed to be unique).

pub fn num_events_for_component( &self, component_name: ComponentName ) -> Option<u64>

The number of events in this chunk for the specified component.

I.e. how many component batches (“cells”) were logged in total for this component?

pub fn row_id_range_per_component( &self ) -> BTreeMap<ComponentName, (RowId, RowId)>

Computes the RowId range covered by each individual component column on each timeline.

This is different from the RowId range covered by the Chunk as a whole because component columns are potentially sparse.

This is crucial for indexing and queries to work properly.

impl Chunk

pub fn new( id: ChunkId, entity_path: EntityPath, is_sorted: Option<bool>, row_ids: StructArray, timelines: BTreeMap<Timeline, TimeColumn>, components: BTreeMap<ComponentName, ListArray<i32>> ) -> Result<Chunk, ChunkError>

Creates a new Chunk.

This will fail if the passed in data is malformed in any way – see Self::sanity_check for details.

Iff you know for sure whether the data is already appropriately sorted or not, specify is_sorted. When left unspecified (None), it will be computed in O(n) time.

For a row-oriented constructor, see Self::builder.

pub fn from_native_row_ids( id: ChunkId, entity_path: EntityPath, is_sorted: Option<bool>, row_ids: &[RowId], timelines: BTreeMap<Timeline, TimeColumn>, components: BTreeMap<ComponentName, ListArray<i32>> ) -> Result<Chunk, ChunkError>

Creates a new Chunk.

This will fail if the passed in data is malformed in any way – see Self::sanity_check for details.

Iff you know for sure whether the data is already appropriately sorted or not, specify is_sorted. When left unspecified (None), it will be computed in O(n) time.

For a row-oriented constructor, see Self::builder.

pub fn from_auto_row_ids( id: ChunkId, entity_path: EntityPath, timelines: BTreeMap<Timeline, TimeColumn>, components: BTreeMap<ComponentName, ListArray<i32>> ) -> Result<Chunk, ChunkError>

Creates a new Chunk.

This will fail if the passed in data is malformed in any way – see Self::sanity_check for details.

The data is assumed to be sorted in RowId-order. Sequential RowIds will be generated for each row in the chunk.

pub fn new_static( id: ChunkId, entity_path: EntityPath, is_sorted: Option<bool>, row_ids: StructArray, components: BTreeMap<ComponentName, ListArray<i32>> ) -> Result<Chunk, ChunkError>

Simple helper for Self::new for static data.

For a row-oriented constructor, see Self::builder.

pub fn empty(id: ChunkId, entity_path: EntityPath) -> Chunk

pub fn add_component( &mut self, component_name: ComponentName, list_array: ListArray<i32> ) -> Result<(), ChunkError>

Unconditionally inserts an ArrowListArray as a component column.

Removes and replaces the column if it already exists.

This will fail if the end result is malformed in any way – see Self::sanity_check.

pub fn add_timeline( &mut self, chunk_timeline: TimeColumn ) -> Result<(), ChunkError>

Unconditionally inserts a TimeColumn.

Removes and replaces the column if it already exists.

This will fail if the end result is malformed in any way – see Self::sanity_check.

impl Chunk

pub fn id(&self) -> ChunkId

pub fn entity_path(&self) -> &EntityPath

pub fn num_columns(&self) -> usize

How many columns in total? Includes control, time, and component columns.

pub fn num_controls(&self) -> usize

pub fn num_timelines(&self) -> usize

pub fn num_components(&self) -> usize

pub fn num_rows(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn row_ids_array(&self) -> &StructArray

pub fn row_ids_raw(&self) -> (&PrimitiveArray<u64>, &PrimitiveArray<u64>)

Returns the RowIds in their raw-est form: a tuple of (times, counters) arrays.

pub fn row_ids(&self) -> impl Iterator<Item = RowId>

All the RowId in this chunk.

This could be in any order if this chunk is unsorted.

pub fn component_row_ids( &self, component_name: &ComponentName ) -> impl Iterator<Item = RowId>

Returns an iterator over the RowIds of a Chunk, for a given component.

This is different than Self::row_ids: it will only yield RowIds for rows at which there is data for the specified component_name.

pub fn row_id_range(&self) -> Option<(RowId, RowId)>

Returns the RowId-range covered by this Chunk.

None if the chunk is_empty.

This is O(1) if the chunk is sorted, O(n) otherwise.

pub fn is_static(&self) -> bool

pub fn timelines(&self) -> &BTreeMap<Timeline, TimeColumn>

pub fn component_names(&self) -> impl Iterator<Item = ComponentName>

pub fn components(&self) -> &BTreeMap<ComponentName, ListArray<i32>>

pub fn timepoint_max(&self) -> TimePoint

Computes the maximum value for each and every timeline present across this entire chunk, and returns the corresponding TimePoint.

impl Chunk

pub fn sanity_check(&self) -> Result<(), ChunkError>

Returns an error if the Chunk’s invariants are not upheld.

Costly checks are only run in debug builds.

impl Chunk

pub fn component_batch_raw( &self, component_name: &ComponentName, row_index: usize ) -> Option<Result<Box<dyn Array>, ChunkError>>

Returns the raw data for the specified component.

Returns an error if the row index is out of bounds.

pub fn component_batch<C>( &self, row_index: usize ) -> Option<Result<Vec<C>, ChunkError>>

where C: Component,

Returns the deserialized data for the specified component.

Returns an error if the data cannot be deserialized, or if the row index is out of bounds.

pub fn component_instance_raw( &self, component_name: &ComponentName, row_index: usize, instance_index: usize ) -> Option<Result<Box<dyn Array>, ChunkError>>

Returns the raw data for the specified component at the given instance index.

Returns an error if either the row index or instance index are out of bounds.

pub fn component_instance<C>( &self, row_index: usize, instance_index: usize ) -> Option<Result<C, ChunkError>>

where C: Component,

Returns the component data of the specified instance.

Returns an error if the data cannot be deserialized, or if either the row index or instance index are out of bounds.

pub fn component_mono_raw( &self, component_name: &ComponentName, row_index: usize ) -> Option<Result<Box<dyn Array>, ChunkError>>

Returns the raw data for the specified component, assuming a mono-batch.

Returns an error if either the row index is out of bounds, or the underlying batch is not of unit length.

pub fn component_mono<C>( &self, row_index: usize ) -> Option<Result<C, ChunkError>>

where C: Component,

Returns the deserialized data for the specified component, assuming a mono-batch.

Returns an error if the data cannot be deserialized, or if either the row index is out of bounds, or the underlying batch is not of unit length.

impl Chunk

pub fn to_unit(self: &Arc<Chunk>) -> Option<UnitChunkShared>

Turns the chunk into a UnitChunkShared, if possible.

pub fn into_unit(self) -> Option<UnitChunkShared>

Turns the chunk into a UnitChunkShared, if possible.

impl Chunk

pub fn iter_indices( &self, timeline: &Timeline ) -> impl Iterator<Item = (TimeInt, RowId)>

Returns an iterator over the indices ((TimeInt, RowId)) of a Chunk, for a given timeline.

If the chunk is static, timeline will be ignored.

See also:

• Self::iter_component_indices.
• Self::iter_indices_owned.

pub fn iter_component_indices( &self, timeline: &Timeline, component_name: &ComponentName ) -> impl Iterator<Item = (TimeInt, RowId)>

Returns an iterator over the indices ((TimeInt, RowId)) of a Chunk, for a given timeline and component.

If the chunk is static, timeline will be ignored.

This is different than Self::iter_indices in that it will only yield indices for rows at which there is data for the specified component_name.

See also Self::iter_indices.

pub fn iter_timepoints(&self) -> impl Iterator<Item = TimePoint>

Returns an iterator over the TimePoints of a Chunk.

See also:

• Self::iter_component_timepoints.

pub fn iter_component_timepoints( &self, component_name: &ComponentName ) -> impl Iterator<Item = TimePoint>

Returns an iterator over the TimePoints of a Chunk, for a given component.

This is different than Self::iter_timepoints in that it will only yield timepoints for rows at which there is data for the specified component_name.

See also Self::iter_timepoints.

pub fn iter_component_offsets( &self, component_name: &ComponentName ) -> impl Iterator<Item = (usize, usize)>

Returns an iterator over the offsets ((offset, len)) of a Chunk, for a given component.

I.e. each (offset, len) pair describes the position of a component batch in the underlying arrow array of values.

pub fn iter_component_arrays( &self, component_name: &ComponentName ) -> impl Iterator<Item = Box<dyn Array>>

Returns an iterator over the raw arrays of a Chunk, for a given component.

See also:

• Self::iter_primitive
• Self::iter_primitive_array
• Self::iter_primitive_array_list
• Self::iter_string
• Self::iter_buffer.
• Self::iter_component.

pub fn iter_primitive<T>( &self, component_name: &ComponentName ) -> impl Iterator<Item = &[T]>

where T: NativeType,

Returns an iterator over the raw primitive values of a Chunk, for a given component.

This is a very fast path: the entire column will be downcasted at once, and then every component batch will be a slice reference into that global slice. Use this when working with simple arrow datatypes and performance matters (e.g. scalars, points, etc).

See also:

• Self::iter_primitive_array
• Self::iter_primitive_array_list
• Self::iter_string
• Self::iter_buffer.
• Self::iter_component_arrays.
• Self::iter_component.

pub fn iter_primitive_array<const N: usize, T>( &self, component_name: &ComponentName ) -> impl Iterator<Item = &[[T; N]]>

where T: NativeType, [T; N]: Pod,

Returns an iterator over the raw primitive arrays of a Chunk, for a given component.

This is a very fast path: the entire column will be downcasted at once, and then every component batch will be a slice reference into that global slice. Use this when working with simple arrow datatypes and performance matters (e.g. scalars, points, etc).

See also:

• Self::iter_primitive
• Self::iter_string
• Self::iter_buffer.
• Self::iter_component_arrays.
• Self::iter_component.

pub fn iter_primitive_array_list<const N: usize, T>( &self, component_name: &ComponentName ) -> impl Iterator<Item = Vec<&[[T; N]]>>

where T: NativeType, [T; N]: Pod,

Returns an iterator over the raw list of primitive arrays of a Chunk, for a given component.

This is a very fast path: the entire column will be downcasted at once, and then every component batch will be a slice reference into that global slice. Use this when working with simple arrow datatypes and performance matters (e.g. strips, etc).

See also:

• Self::iter_primitive
• Self::iter_primitive_array
• Self::iter_string
• Self::iter_buffer.
• Self::iter_component_arrays.
• Self::iter_component.

pub fn iter_string( &self, component_name: &ComponentName ) -> impl Iterator<Item = Vec<ArrowString>>

Returns an iterator over the raw strings of a Chunk, for a given component.

This is a very fast path: the entire column will be downcasted at once, and then every component batch will be a slice reference into that global slice. Use this when working with simple arrow datatypes and performance matters (e.g. labels, etc).

See also:

• Self::iter_primitive
• Self::iter_primitive_array
• Self::iter_primitive_array_list
• Self::iter_buffer.
• Self::iter_component_arrays.
• Self::iter_component.

pub fn iter_buffer<T>( &self, component_name: &ComponentName ) -> impl Iterator<Item = Vec<ArrowBuffer<T>>>

where T: NativeType,

Returns an iterator over the raw buffers of a Chunk, for a given component.

This is a very fast path: the entire column will be downcasted at once, and then every component batch will be a slice reference into that global slice. Use this when working with simple arrow datatypes and performance matters (e.g. blobs, etc).

See also:

• Self::iter_primitive
• Self::iter_primitive_array
• Self::iter_primitive_array_list
• Self::iter_string.
• Self::iter_component_arrays.
• Self::iter_component.

impl Chunk

pub fn iter_indices_owned( self: Arc<Chunk>, timeline: &Timeline ) -> impl Iterator<Item = (TimeInt, RowId)>

Returns an iterator over the indices ((TimeInt, RowId)) of a Chunk, for a given timeline.

If the chunk is static, timeline will be ignored.

The returned iterator outlives self, thus it can be passed around freely. The tradeoff is that self must be an Arc.

See also Self::iter_indices.

impl Chunk

pub fn iter_component<C>( &self ) -> ChunkComponentIter<C, impl Iterator<Item = (usize, usize)>>

where C: Component,

Returns an iterator over the deserialized batches of a Chunk, for a given component.

This is a dedicated fast path: the entire column will be downcasted and deserialized at once, and then every component batch will be a slice reference into that global slice. Use this when working with complex arrow datatypes and performance matters (e.g. ranging through enum types across many timestamps).

See also:

• Self::iter_primitive
• Self::iter_primitive_array
• Self::iter_primitive_array_list
• Self::iter_string
• Self::iter_buffer.
• Self::iter_component_arrays.

impl Chunk

pub fn latest_at( &self, query: &LatestAtQuery, component_name: ComponentName ) -> Chunk

Runs a LatestAtQuery filter on a Chunk.

This behaves as a row-based filter: the result is a new Chunk that is vertically sliced to only contain the row relevant for the specified query.

The resulting Chunk is guaranteed to contain all the same columns has the queried chunk: there is no horizontal slicing going on.

An empty Chunk (i.e. 0 rows, but N columns) is returned if the query yields nothing.

Because the resulting chunk doesn’t discard any column information, you can find extra relevant information by inspecting the data, for examples timestamps on other timelines. See Self::timeline_sliced and Self::component_sliced if you do want to filter this extra data.

impl Chunk

pub fn concatenated(&self, rhs: &Chunk) -> Result<Chunk, ChunkError>

Concatenates two Chunks into a new one.

The order of the arguments matter: self‘s contents will precede rhs’ contents in the returned Chunk.

This will return an error if the chunks are not concatenable.

pub fn overlaps_on_row_id(&self, rhs: &Chunk) -> bool

Returns true if self and rhs overlap on their RowId range.

pub fn overlaps_on_time(&self, rhs: &Chunk) -> bool

Returns true if self and rhs overlap on any of their time range(s).

This does not imply that they share the same exact set of timelines.

pub fn same_entity_paths(&self, rhs: &Chunk) -> bool

Returns true if both chunks share the same entity path.

pub fn same_timelines(&self, rhs: &Chunk) -> bool

Returns true if both chunks contains the same set of timelines.

pub fn same_datatypes(&self, rhs: &Chunk) -> bool

Returns true if both chunks share the same datatypes for the components that they have in common.

pub fn concatenable(&self, rhs: &Chunk) -> bool

Returns true if two chunks are concatenable.

To be concatenable, two chunks must:

• Share the same entity path.
• Share the same exact set of timelines.
• Use the same datatypes for the components they have in common.

impl Chunk

pub fn range(&self, query: &RangeQuery, component_name: ComponentName) -> Chunk

Runs a RangeQuery filter on a Chunk.

This behaves as a row-based filter: the result is a new Chunk that is vertically sliced, sorted and filtered in order to only contain the row(s) relevant for the specified query.

The resulting Chunk is guaranteed to contain all the same columns has the queried chunk: there is no horizontal slicing going on.

An empty Chunk (i.e. 0 rows, but N columns) is returned if the query yields nothing.

Because the resulting chunk doesn’t discard any column information, you can find extra relevant information by inspecting the data, for examples timestamps on other timelines. See Self::timeline_sliced and Self::component_sliced if you do want to filter this extra data.

impl Chunk

pub fn is_sorted(&self) -> bool

Is the chunk currently ascendingly sorted by crate::RowId?

This is O(1) (cached).

See also Self::is_sorted_uncached.

pub fn is_time_sorted(&self) -> bool

Is the chunk ascendingly sorted by time, for all of its timelines?

This is O(1) (cached).

pub fn is_timeline_sorted(&self, timeline: &Timeline) -> bool

Is the chunk ascendingly sorted by time, for a specific timeline?

This is O(1) (cached).

See also Self::is_timeline_sorted_uncached.

pub fn sort_if_unsorted(&mut self)

Sort the chunk, if needed.

The underlying arrow data will be copied and shuffled in memory in order to make it contiguous.

pub fn sorted_by_timeline_if_unsorted(&self, timeline: &Timeline) -> Chunk

Returns a new Chunk that is sorted by (<timeline>, RowId).

The underlying arrow data will be copied and shuffled in memory in order to make it contiguous.

This is a no-op if the underlying timeline is already sorted appropriately (happy path).

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn shuffle_random(&mut self, seed: u64)

Randomly shuffles the chunk using the given seed.

The underlying arrow data will be copied and shuffled in memory in order to make it contiguous.

impl Chunk

pub fn cell( &self, row_id: RowId, component_name: &ComponentName ) -> Option<Box<dyn Array>>

Returns the cell corresponding to the specified RowId for a given ComponentName.

This is O(log(n)) if self.is_sorted(), and O(n) otherwise.

Reminder: duplicated RowIds results in undefined behavior.

pub fn row_sliced(&self, index: usize, len: usize) -> Chunk

Slices the Chunk vertically.

The result is a new Chunk with the same columns and (potentially) less rows.

This cannot fail nor panic: index and len will be capped so that they cannot run out of bounds. This can result in an empty Chunk being returned if the slice is completely OOB.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn timeline_sliced(&self, timeline: Timeline) -> Chunk

Slices the Chunk horizontally by keeping only the selected timeline.

The result is a new Chunk with the same rows and (at-most) one timeline column. All non-timeline columns will be kept as-is.

If timeline is not found within the Chunk, the end result will be the same as the current chunk but without any timeline column.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn component_sliced(&self, component_name: ComponentName) -> Chunk

Slices the Chunk horizontally by keeping only the selected component_name.

The result is a new Chunk with the same rows and (at-most) one component column. All non-component columns will be kept as-is.

If component_name is not found within the Chunk, the end result will be the same as the current chunk but without any component column.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn timelines_sliced( &self, timelines_to_keep: &HashSet<Timeline, BuildHasherDefault<NoHashHasher<Timeline>>> ) -> Chunk

Slices the Chunk horizontally by keeping only the selected timelines.

The result is a new Chunk with the same rows and (at-most) the selected timeline columns. All non-timeline columns will be kept as-is.

If none of the selected timelines exist in the Chunk, the end result will be the same as the current chunk but without any timeline column.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn components_sliced( &self, component_names: &HashSet<ComponentName, BuildHasherDefault<NoHashHasher<ComponentName>>> ) -> Chunk

Slices the Chunk horizontally by keeping only the selected component_names.

The result is a new Chunk with the same rows and (at-most) the selected component columns. All non-component columns will be kept as-is.

If none of the component_names exist in the Chunk, the end result will be the same as the current chunk but without any component column.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn densified(&self, component_name_pov: ComponentName) -> Chunk

Densifies the Chunk vertically based on the component_name column.

Densifying here means dropping all rows where the associated value in the component_name column is null.

The result is a new Chunk where the component_name column is guaranteed to be dense.

If component_name doesn’t exist in this Chunk, or if it is already dense, this method is a no-op.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn emptied(&self) -> Chunk

Empties the Chunk vertically.

The result is a new Chunk with the same columns but zero rows.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn components_removed(self) -> Chunk

Removes all component columns from the Chunk.

The result is a new Chunk with the same number of rows and the same index columns, but no components.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn deduped_latest_on_index(&self, index: &Timeline) -> Chunk

Removes duplicate rows from sections of consecutive identical indices.

• If the Chunk is sorted on that index, the remaining values in the index column will be unique.
• If the Chunk has been densified on a specific column, the resulting chunk will effectively contain the latest value of that column for each given index value.

If this is a temporal chunk and timeline isn’t present in it, this method is a no-op.

This does not obey RowId-ordering semantics (or any other kind of semantics for that matter) – it merely respects how the chunk is currently laid out: no more, no less. Sort the chunk according to the semantics you’re looking for before calling this method.

pub fn filtered(&self, filter: &BooleanArray) -> Option<Chunk>

Applies a filter kernel to the Chunk as a whole.

Returns None if the length of the filter does not match the number of rows in the chunk.

In release builds, filters are allowed to have null entries (they will be interpreted as false). In debug builds, null entries will panic.

Note: a filter kernel copies the data in order to make the resulting arrays contiguous in memory.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

pub fn taken<O>(&self, indices: &PrimitiveArray<O>) -> Chunk

where O: Index,

Applies a take kernel to the Chunk as a whole.

In release builds, indices are allowed to have null entries (they will be taken as nulls). In debug builds, null entries will panic.

Note: a take kernel copies the data in order to make the resulting arrays contiguous in memory.

Takes care of up- and down-casting the data back and forth on behalf of the caller.

WARNING: the returned chunk has the same old crate::ChunkId! Change it with Self::with_id.

impl Chunk

pub fn to_transport(&self) -> Result<TransportChunk, ChunkError>

Prepare the Chunk for transport.

It is probably a good idea to sort the chunk first.

pub fn from_transport(transport: &TransportChunk) -> Result<Chunk, ChunkError>

impl Chunk

pub fn from_arrow_msg(msg: &ArrowMsg) -> Result<Chunk, ChunkError>

pub fn to_arrow_msg(&self) -> Result<ArrowMsg, ChunkError>

Trait Implementations

impl Clone for Chunk

fn clone(&self) -> Chunk

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Chunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Display for Chunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl PartialEq for Chunk

fn eq(&self, other: &Chunk) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl SizeBytes for Chunk

fn heap_size_bytes(&self) -> u64

Returns the total size of self on the heap, in bytes.

fn total_size_bytes(&self) -> u64

Returns the total size of self in bytes, accounting for both stack and heap space.

fn stack_size_bytes(&self) -> u64

Returns the total size of self on the stack, in bytes.

fn is_pod() -> bool

Is Self just plain old data?