Experimental

rerun.experimental

Experimental features for Rerun.

These features are not yet stable and may change in future releases without going through the normal deprecation cycle.

Lens `module-attribute`

Lens: TypeAlias = DeriveLens | MutateLens

Union of all lens types.

Chunk

A single chunk of data from a recording.

entity_path `property`

entity_path: str

The entity path this chunk belongs to.

id `property`

id: str

The unique ID of this chunk.

is_empty `property`

is_empty: bool

Whether the chunk has zero rows.

is_static `property`

is_static: bool

Whether the chunk contains only static data (no timelines).

num_columns `property`

num_columns: int

The number of columns in this chunk.

num_rows `property`

num_rows: int

The number of rows in this chunk.

timeline_names `property`

timeline_names: list[str]

The names of all timelines in this chunk.

apply_lenses

def apply_lenses(lenses: Sequence[Lens] | Lens) -> list[Chunk]

Apply one or more lenses to this chunk, returning transformed chunks.

Each lens matches by input component. Columns not consumed by any matching lens are forwarded unchanged as a separate chunk.

If no lens matches the chunk (including when an empty list of lenses is passed), the original chunk is returned unchanged.

PARAMETER	DESCRIPTION
`lenses`	One or more `Lens` objects. TYPE: `Sequence[Lens] \| Lens`

RETURNS	DESCRIPTION
A list of [`Chunk`][] objects.

apply_selector

def apply_selector(
    source: ComponentDescriptor | str,
    selector: Selector | str,
) -> Chunk

Apply a selector to a single component, returning a new chunk with the component transformed.

All other columns (timelines, other components) are preserved unchanged. The source component's existing descriptor is preserved.

For better performance, prefer MutateLens with apply_lenses which processes multiple transformations in a single pass.

PARAMETER	DESCRIPTION
`source`	A `ComponentDescriptor` or component identifier string for the input column to transform. TYPE: `ComponentDescriptor \| str`
`selector`	A `Selector` or selector query string to apply to the component. TYPE: `Selector \| str`

RETURNS	DESCRIPTION
A new [`Chunk`][rerun.experimental.Chunk] with the component transformed.

RAISES	DESCRIPTION
`ValueError`	If the source component is not found in the chunk or the selector fails to evaluate.

format

def format(
    *,
    width: int = 240,
    redact: bool = False,
    trim_metadata_keys: bool = True,
) -> str

Format this chunk as a human-readable table string.

PARAMETER	DESCRIPTION
`width`	Fixed width for the table. Default: 240. TYPE: `int` DEFAULT: `240`
`redact`	If True, redact non-deterministic values (RowIds, ChunkIds, etc.) for stable snapshot testing. Default: False. TYPE: `bool` DEFAULT: `False`
`trim_metadata_keys`	If True, trim the `rerun:` / `sorbet:` prefix from metadata keys. Default: True. TYPE: `bool` DEFAULT: `True`

from_columns `classmethod`

def from_columns(
    entity_path: str,
    indexes: Iterable[TimeColumnLike],
    columns: Iterable[ComponentColumn],
) -> Chunk

Create a Chunk from columns, mirroring the rerun.send_columns API.

A fresh chunk ID and sequential row IDs are auto-generated.

PARAMETER	DESCRIPTION
`entity_path`	The entity path for this chunk (e.g., "/camera/image"). TYPE: `str`
`indexes`	The time columns for this chunk. Each `TimeColumnLike` provides a timeline name and a PyArrow array of timestamps. You typically use `TimeColumn` here. Pass an empty iterable for static data. TYPE: `Iterable[TimeColumnLike]`
`columns`	The component columns for this chunk. Each `ComponentColumn` provides a component descriptor and a PyArrow array of component data. TYPE: `Iterable[ComponentColumn]`

RAISES	DESCRIPTION
`ValueError`	If timeline and component column lengths don't match.

Example

chunk = Chunk.from_columns(
    "/robots/arm",
    indexes=[rr.TimeColumn("frame", sequence=[0, 1, 2])],
    columns=rr.Points3D.columns(positions=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]),
)

from_dataframe `classmethod`

def from_dataframe(
    dataframe: DataframeLike,
    *,
    index: str | list[str] | None | _AutoIndex = AUTO_INDEX,
    entity_path: str | None = None,
) -> Iterator[Chunk]

Lazily turn an Arrow-backed dataframe into chunks.

Accepts a Table, a RecordBatch, a RecordBatchReader, or any object implementing the Arrow C stream interface (__arrow_c_stream__) — most notably a datafusion.DataFrame (an optional dependency).

Yields each chunk of Chunk.from_record_batch applied to every record batch in turn. See that method for the index and entity_path semantics.

RAISES	DESCRIPTION
`TypeError`	If `dataframe` is not a pyarrow `Table`, a pyarrow `RecordBatch`, a pyarrow `RecordBatchReader`, or an Arrow-C-stream object (such as a `datafusion.DataFrame`).
`ValueError`	See `Chunk.from_record_batch`.

from_record_batch `classmethod`

def from_record_batch(
    record_batch: RecordBatch,
    *,
    index: str | list[str] | None | _AutoIndex = AUTO_INDEX,
    entity_path: str | None = None,
) -> list[Chunk]

Interpret an Arrow RecordBatch as Rerun chunk data.

Each column of the batch is classified as a row-id column, index (timeline) column, or a component column. Component columns are then grouped per entity path, and one chunk per entity path is emitted.

The rerun:* arrow metadata, if it exists, drives the kind of each input column, as well as the entity/archetype/component type for component columns.

If present, the row id column and chunk id metadata indicate that the batch represents a fully identified chunk, e.g. as produced by Chunk.to_record_batch. Both the row ids and chunk id are preserved under the following conditions: - both are present in the input batch - index is omitted - entity_path is omitted

If any of these conditions are not met, it means that either the batch is not fully identified, or that the chunk data is reinterpreted (e.g. entity path rewriting). In that case, fresh row ids and chunk id are generated and used instead of the input ones.

PARAMETER	DESCRIPTION
`record_batch`	The Arrow record batch to interpret. Component columns may be either lists (one component batch per row) or plain arrays (wrapped as single-element lists automatically). TYPE: `RecordBatch`
`index`	Determines which columns are index (timeline) columns. Each promoted column's time type is taken from its Arrow datatype: `int64` → sequence, `timestamp(ns)` → timestamp, `duration(ns)` → duration. Omitted (the default): derive the index columns from the batch's Rerun metadata. The batch is treated as temporal if it carries index metadata. A batch with no index metadata is ambiguous and raises an error — unless it is an already-identified chunk (it carries a row-id column and a chunk id), which round-trips as-is and may therefore be static. Pass `index=None` to force a static interpretation. A column name, or list of column names: treat exactly these columns as timelines. The remaining (non-row-id) columns become components. `None`: produce static chunks (no timeline). Any index metadata or promoted index column is then a contradiction and is rejected. Note Static chunks with multiple rows are legitimate in some cases, but only the last row is visible from typical latest-at queries. An info-level message is emitted when this happens — except for an already-identified chunk that is preserved as-is (see above), which is passed through without this check. TYPE: `str \| list[str] \| None \| _AutoIndex` DEFAULT: `AUTO_INDEX`
`entity_path`	Default entity path for component columns that do not otherwise specify one. Resolution order per component column is: its `rerun:entity_path` metadata, then the batch-level `rerun:entity_path` metadata, then the column-name convention (see Notes), then this argument, then the root entity (`/`). TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`One chunk per distinct entity path described by the batch, in first-seen column order.`

RAISES DESCRIPTION

ValueError

In any of the following cases:

index was omitted and the batch carries no index metadata (an ambiguous raw batch). Pass index=<column> for temporal data or index=None for static data.
index=None was given but the batch also carries index metadata or names an index column (contradiction).
index names a column that is not present in the batch.
The batch contains no component columns (there is nothing to log).
A column promoted to an index contains null values. Time columns must be dense; static data is expressed with index=None, not with null times.
An index column has an Arrow datatype that is not a supported time type.
The batch is a fully-identified chunk (it carries both a row-id column and a chunk id) but resolves to more than one entity path. An identified chunk is preserved as a single chunk; drop the chunk-id metadata and/or the row-id column to reinterpret it into one chunk per entity (with freshly-minted ids).

Notes

Column-name convention. When a component column carries no rerun:entity_path / rerun:component metadata, its entity path is read from the column name: if the name starts with / and contains a :, the first part of the column name is interpreted as the entity path and the rest as the component identifier. Example: /point:Points3D:positions and /metadata:foo.

Limitations/Future work

A batch that mixes static and temporal rows — aka where some index values are null — are rejected. Handling this case requires row-splitting and generating a mix of temporal and static chunks.

Recording-property columns (named property:…, mapping to the /__properties entity) are not recognized by the column-name convention and are not mapped back to that entity.

to_record_batch

def to_record_batch() -> RecordBatch

Convert this chunk to an Arrow RecordBatch.

with_entity_path

def with_entity_path(entity_path: str) -> Chunk

Return a copy of this chunk with a new entity path.

A fresh chunk ID is generated to avoid aliasing the original chunk in downstream caches and indices. Row IDs, timelines, and components are preserved as-is.

PARAMETER	DESCRIPTION
`entity_path`	The new entity path for the returned chunk (e.g. `"/left/camera/image"`). TYPE: `str`

ChunkStore

A fully-materialized, in-memory chunk store.

Build one from chunks via ChunkStore.from_chunks, or fully materialize an IndexedReader via reader.stream().collect(). For lazy, on-demand chunk loading, see LazyStore.

Use stream() to process chunks through the lazy pipeline, or write_rrd() to persist to disk.

len

def __len__() -> int

Return the number of chunks in this store.

from_chunks `staticmethod`

def from_chunks(chunks: Sequence[Chunk]) -> ChunkStore

Build a ChunkStore from a sequence of chunks.

reader

def reader(
    index: str | None,
    *,
    contents: ContentFilter | str | list[str] | None = None,
    include_semantically_empty_columns: bool = False,
    include_tombstone_columns: bool = False,
    fill_latest_at: bool = False,
    using_index_values: IndexValuesLike | None = None,
    ctx: SessionContext | None = None,
) -> DataFrame

Build a DataFusion DataFrame over this store.

The returned DataFrame is data-equivalent to the result of round-tripping the same chunks through write_rrd → rr.server.Server → dataset.reader(), modulo the rerun_segment_id column (absent here because a single ChunkStore has no segment concept).

PARAMETER	DESCRIPTION
`index`	The index (timeline) column to use, or `None` for the static-only view. TYPE: `str \| None`
`contents`	Entity-path filter. A `ContentFilter` built with the fluent API, a single entity-path expression, a list of expressions, or `None` for everything. An empty list returns no rows. TYPE: `ContentFilter \| str \| list[str] \| None` DEFAULT: `None`
`include_semantically_empty_columns`	Whether to include columns that are semantically empty. TYPE: `bool` DEFAULT: `False`
`include_tombstone_columns`	Whether to include tombstone columns. TYPE: `bool` DEFAULT: `False`
`fill_latest_at`	Whether to fill null values with the latest valid data. TYPE: `bool` DEFAULT: `False`
`using_index_values`	Index values at which to resample data. When specified, this argument changes the way rows are returned. Instead of returning the rows that exist in the data, one row is returned per `index_value` you provide. If the segment has no row at that index value, nulls are returned — or the latest prior value if fill_latest_at=True` (which is typically what you want for resampling). Don't use this argument for plain index slicing — use a DataFusion filter on the index column instead. For example: `from datafusion import col, lit # All rows in a time window. store.reader(index="real_time").filter( (col("real_time") >= lit(t0)) & (col("real_time") <= lit(t1)) )` TYPE: `IndexValuesLike \| None` DEFAULT: `None`
`ctx`	DataFusion `SessionContext` to register the table into. When `None`, uses `datafusion.SessionContext.global_ctx()` — the process-wide default. Pass an explicit `ctx` for isolation or a custom `SessionConfig`. TYPE: `SessionContext \| None` DEFAULT: `None`

schema

def schema() -> Schema

The schema describing all columns in this store.

stream

def stream() -> LazyChunkStream

Return a lazy stream over all chunks in this store.

summary

def summary() -> str

Compact, deterministic summary of every chunk in the store.

Each line describes one chunk:

{entity_path}  rows={n}  static={True|False}  timelines=[…]  cols=[…]

Useful for snapshot testing.

write_rrd

def write_rrd(
    path: str | Path,
    *,
    application_id: str,
    recording_id: str,
) -> None

Write all chunks to an RRD file.

The caller must provide application_id and recording_id explicitly.

DatasetInfo `dataclass`

Structural metadata for a single HDF5 dataset.

ATTRIBUTE	DESCRIPTION
`path`	Full path of the dataset within the file (e.g. `/observations/qpos`). TYPE: `str`
`shape`	Dataset dimensions (e.g. `(272, 128, 128, 3)`). TYPE: `tuple[int, ...]`
`dtype`	Element type name (e.g. `"uint8"`, `"float64"`). TYPE: `str`

DeriveLens

A derive lens that creates new component/time columns from an input component.

Derive lenses extract fields from a component and produce new columns, optionally at a different entity and/or with new time columns.

Pass scatter=True to enable 1:N row mapping (exploding lists).

Example usage::

lens = (
    DeriveLens("Imu:accel")
    .to_component(rr.Scalars.descriptor_scalars(), Selector(".x"))
)

To write to an explicit target entity::

lens = (
    DeriveLens("Imu:accel", output_entity="/out/x")
    .to_component(rr.Scalars.descriptor_scalars(), Selector(".x"))
)

init

def __init__(
    input_component: str,
    *,
    output_entity: str | None = None,
    scatter: bool = False,
) -> None

Create a new derive lens.

PARAMETER	DESCRIPTION
`input_component`	The component identifier to match (e.g. `"Imu:accel"`). TYPE: `str`
`output_entity`	Optional target entity path. When set, output is written to this entity instead of the input entity. TYPE: `str \| None` DEFAULT: `None`
`scatter`	When `True`, use 1:N row mapping (explode lists). TYPE: `bool` DEFAULT: `False`

to_component

def to_component(
    component: ComponentDescriptor | str,
    selector: Selector | str,
    *,
    cast_to: DataType | Literal["auto"] | None = None,
) -> DeriveLens

Add a component output column.

PARAMETER	DESCRIPTION
`component`	A `ComponentDescriptor` or a component identifier string for the output column (e.g. `"Scalars:scalars"`). TYPE: `ComponentDescriptor \| str`
`selector`	A `Selector` or selector query string to apply to the input column. TYPE: `Selector \| str`
`cast_to`	How to cast the produced column to match the target component. By default (`None`) the column is emitted as-is. Pass `"auto"` to cast it to the component's canonical Arrow datatype, or an explicit pyarrow `DataType` to cast it to that type. Casting errors if the conversion is unsupported. TYPE: `DataType \| Literal['auto'] \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the component added.

to_packed_component

def to_packed_component(
    component: ComponentDescriptor | str,
    *fields: str,
    cast_to: DataType | Literal["auto"] | None = "auto",
) -> DeriveLens

Add a component output column by packing the provided fields in a fixed-size list.

PARAMETER	DESCRIPTION
`component`	A `ComponentDescriptor` or a component identifier string for the output column (e.g. `"Points3D:positions"`). TYPE: `ComponentDescriptor \| str`
`*fields`	Names of the struct fields to pack, in order. They must all resolve to the same datatype. At least one field is required. TYPE: `str` DEFAULT: `()`
`cast_to`	How to cast the packed column to match the target component. Defaults to `"auto"`, which casts to the component's canonical Arrow datatype (e.g. the `f64` columns a parquet file typically holds → the `f32` a `Transform3D:translation` expects). Pass an explicit pyarrow `DataType` to cast to that type, or `None` to emit the packed list as-is. TYPE: `DataType \| Literal['auto'] \| None` DEFAULT: `'auto'`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the packed component added.

to_quaternion

def to_quaternion(x: str, y: str, z: str, w: str) -> DeriveLens

Add a Transform3D:quaternion component from the provided paths.

PARAMETER	DESCRIPTION
`x`	Paths of the struct fields holding the quaternion components, in `xyzw` order. TYPE: `str`
`y`	Paths of the struct fields holding the quaternion components, in `xyzw` order. TYPE: `str`
`z`	Paths of the struct fields holding the quaternion components, in `xyzw` order. TYPE: `str`
`w`	Paths of the struct fields holding the quaternion components, in `xyzw` order. TYPE: `str`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the quaternion added.

to_rotation_axis_angle

def to_rotation_axis_angle(
    axis_x: str, axis_y: str, axis_z: str, angle: str
) -> DeriveLens

Add a Transform3D:rotation_axis_angle component from the provided paths.

PARAMETER	DESCRIPTION
`axis_x`	Paths of the struct fields holding the rotation axis components. TYPE: `str`
`axis_y`	Paths of the struct fields holding the rotation axis components. TYPE: `str`
`axis_z`	Paths of the struct fields holding the rotation axis components. TYPE: `str`
`angle`	Path of the struct field holding the rotation angle, in radians. TYPE: `str`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the rotation added.

to_scalars

def to_scalars(*fields: str) -> DeriveLens

Add a Scalars:scalars component from the provided path(s).

Each path becomes one scalar instance per row, so a single path yields one series and multiple paths yield one series each at the same entity.

PARAMETER	DESCRIPTION
`*fields`	Paths of the struct fields to read as scalars, in order. At least one is required. TYPE: `str` DEFAULT: `()`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the scalars added.

to_scale

def to_scale(x: str, y: str, z: str) -> DeriveLens

Add a Transform3D:scale component from the provided paths.

PARAMETER	DESCRIPTION
`x`	Paths of the struct fields holding the per-axis scale factors. TYPE: `str`
`y`	Paths of the struct fields holding the per-axis scale factors. TYPE: `str`
`z`	Paths of the struct fields holding the per-axis scale factors. TYPE: `str`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the scale added.

to_timeline

def to_timeline(
    timeline_name: str,
    timeline_type: Literal[
        "sequence", "duration_ns", "timestamp_ns"
    ],
    selector: Selector | str,
) -> DeriveLens

Add a time extraction column.

PARAMETER	DESCRIPTION
`timeline_name`	Name of the timeline to create. TYPE: `str`
`timeline_type`	Type of the timeline: `"sequence"`, `"duration_ns"`, or `"timestamp_ns"`. TYPE: `Literal['sequence', 'duration_ns', 'timestamp_ns']`
`selector`	A `Selector` or selector query string to extract time values (must produce `Int64` arrays). TYPE: `Selector \| str`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the time column added.

to_translation

def to_translation(x: str, y: str, z: str) -> DeriveLens

Add a Transform3D:translation component from the provided paths.

PARAMETER	DESCRIPTION
`x`	Paths of the struct fields holding the translation components. TYPE: `str`
`y`	Paths of the struct fields holding the translation components. TYPE: `str`
`z`	Paths of the struct fields holding the translation components. TYPE: `str`

RETURNS	DESCRIPTION
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the translation added.

Hdf5Reader

Read chunks from an HDF5 file.

The reader is a lightweight handle over the file: inspect the raw structure with groups(), datasets(), and attributes(), and produce chunks with stream(...). All loading options live on stream(), so one reader can drive several differently-configured streams over the same file.

Each HDF5 group is mapped to a Rerun entity, and the group's leaf datasets become the columns of that entity. The file root maps to the entity /, and nested groups map to nested entity paths (/observations/images becomes the entity /observations/images). See stream() for how datasets, timelines, and attributes are turned into chunks.

PARAMETER	DESCRIPTION
`path`	Path to the `.hdf5` / `.h5` file. TYPE: `str \| Path`

RAISES	DESCRIPTION
`FileNotFoundError`	If `path` does not exist.

path `property`

path: Path

The file path of the HDF5 file.

attributes

def attributes(
    path: str = "/",
) -> dict[
    str, int | float | str | bytes | list[int | float | str]
]

Read the HDF5 attributes attached to an object as a typed Python dict.

This is a convenience accessor for the same attributes that stream() emits under __hdf5_properties (see the class docstring). It reads the raw file directly.

PARAMETER	DESCRIPTION
`path`	Path to the object whose attributes are read. Defaults to the root group `/`, i.e. the file-level (global) attributes. May reference any group or dataset. TYPE: `str` DEFAULT: `'/'`

RETURNS	DESCRIPTION
`A mapping from attribute name to value. Scalar attributes are returned`
as Python scalars (`int`, `float`, `str`, `bytes`); array-valued
`attributes are returned as lists. Empty if the object has no attributes.`

RAISES	DESCRIPTION
`KeyError`	If `path` does not exist in the file.

datasets

def datasets(path: str = '/') -> list[DatasetInfo]

List the datasets under path, recursively, with their shape and dtype.

Metadata only — no dataset values are read; reflects the raw file.

PARAMETER	DESCRIPTION
`path`	Group under which to list. Defaults to the root group `/`, i.e. the whole file. TYPE: `str` DEFAULT: `'/'`

groups

def groups(path: str = '/') -> list[str]

List the group paths under path, recursively.

Metadata only — no dataset values are read; reflects the raw file.

PARAMETER	DESCRIPTION
`path`	Group under which to list. Defaults to the root group `/`, i.e. the whole file. TYPE: `str` DEFAULT: `'/'`

stream

def stream(
    *,
    entity_path_prefix: str | None = None,
    index_column: IndexColumn | None = None,
    ignore_datasets: list[str] | None = None,
    use_structs: bool = True,
) -> LazyChunkStream

Return a lazy stream over all chunks in the HDF5 file.

Each call is independent: the same reader can be streamed several times with different configurations.

Datasets are loaded according to their dimensionality:

A 0-D (scalar) dataset is loaded as static data — a single value with no timeline.
A 1-D dataset [N] becomes a column of N scalar rows.
A 2-D dataset [N, K] becomes a column of N rows, each a fixed-size list of K elements.
A 3-D-or-higher dataset [N, d1, …, dk] becomes a column of N rows, each a single blob of the matching type (an Arrow List<PRIMITIVE_TYPE>) holding the row's raw row-major values. The original per-row shape is not recorded in the emitted data; recover it via datasets.

For 1-D and higher-dimensional datasets the leading dimension is always the row axis. Element types are mapped to their natural Arrow equivalents (signed and unsigned integers, floats, and strings); no semantic interpretation is applied.

HDF5 attributes are emitted as static chunks under a dedicated __hdf5_properties entity, mirroring the source layout: root attributes land on __hdf5_properties, and attributes on object /a/b on __hdf5_properties/a/b. Each attribute becomes one static component named after it, typed with the same mapping as datasets. This keeps the general __properties entity free for user-defined property layers.

Row alignment

Every loaded, non-ignored, non-scalar dataset is aligned positionally to the file-wide timeline and must therefore share the same number of rows (scalar datasets are static and exempt):

With an index_column, that shared count is the index dataset's length.
Without one, the datasets must all agree on a single row count, which becomes the length of the generated row_index timeline.

A dataset that violates this raises unless it is listed in ignore_datasets; nothing is dropped automatically to satisfy alignment.

PARAMETER	DESCRIPTION
`entity_path_prefix`	Optional prefix prepended to every entity path (for example `"/world"`). TYPE: `str \| None` DEFAULT: `None`
`index_column`	Dataset to use as the file-wide timeline index, built with `IndexColumn`, e.g. `IndexColumn.timestamp("/time", input_unit="s")` or `IndexColumn.sequence("/frame_id")`. The referenced dataset must be 1-dimensional. When omitted, a single `row_index` sequence timeline (0, 1, …) is generated for the whole file and every loaded dataset must align to it (see Row alignment). TYPE: `IndexColumn \| None` DEFAULT: `None`
`ignore_datasets`	Datasets or groups to exclude entirely. Each entry is a dataset path or a group path (which excludes the whole subtree). Ignored datasets are neither loaded nor considered for row alignment. TYPE: `list[str] \| None` DEFAULT: `None`
`use_structs`	When `True` (default), all columns of an entity are packed into a single Arrow `Struct` component, with one field per dataset named after that dataset. When `False`, each dataset becomes a separate component on the same entity. A group holding a single dataset always emits that dataset as a bare component, never as a one-field struct. TYPE: `bool` DEFAULT: `True`

RAISES DESCRIPTION

ValueError

If a loaded, non-ignored, non-scalar dataset cannot be aligned to the applicable row count (the index length, or the file's shared row count when no index_column is set). Resolve by adding the offending dataset to ignore_datasets or by choosing a compatible index_column.

Also raised when the file exists but cannot be parsed as HDF5: the layout is validated eagerly here, so such failures surface at stream() rather than lazily mid-iteration.

IndexColumn `dataclass`

A dataset/column to use as a timeline index, and how to interpret it.

Construct one with timestamp, duration, or sequence — the timeline kind is the constructor you pick, so there is nothing to mistype:

IndexColumn.timestamp("/time", input_unit="s")
IndexColumn.duration("/elapsed", input_unit="us")
IndexColumn.sequence("/frame_id")

input_unit `class-attribute` `instance-attribute`

input_unit: TimeUnit | None = None

What the raw integer/float values represent (not a desired output unit); values are scaled to nanoseconds internally. None for sequence.

kind `instance-attribute`

kind: Literal['timestamp', 'duration', 'sequence']

The timeline kind: time since epoch, elapsed time, or an ordinal integer index.

path `instance-attribute`

path: str

Path of the 1-D dataset (HDF5) or name of the column (Parquet) to use as the index.

duration `classmethod`

def duration(
    path: str, *, input_unit: TimeUnit = "ns"
) -> IndexColumn

An elapsed-time timeline. input_unit describes the raw values (default "ns").

sequence `classmethod`

def sequence(path: str) -> IndexColumn

An ordinal integer-index timeline. No unit applies.

timestamp `classmethod`

def timestamp(
    path: str, *, input_unit: TimeUnit = "ns"
) -> IndexColumn

A time-since-epoch timeline. input_unit describes the raw values (default "ns").

IndexedReader

Bases: StreamingReader, Protocol

Protocol for readers backed by an index/manifest.

Extends StreamingReader: every IndexedReader also supports stream() -> LazyChunkStream for pure-streaming processing.

Indexed readers expose a LazyStore view over the source via store() — the manifest is read up-front; chunks load on demand. To fully materialize into a ChunkStore, call stream().collect().

store

def store() -> LazyStore

Return a LazyStore view of this source.

stream

def stream() -> LazyChunkStream

Return a lazy stream over all chunks from this source.

LazyChunkStream

A lazy, composable pipeline over chunks.

Builder methods (filter, drop, split, map, flat_map, lenses, merge) consume the input stream(s) and return new stream(s). A consumed stream cannot be used again; attempting to do so raises a ValueError. This prevents accidental reuse that would result in duplicate use of the same stream in a pipeline.

Terminal methods (to_chunks, __iter__, collect, write_rrd) do not consume the stream — they run the pipeline and leave the stream usable. Each call creates a fresh execution.

iter

def __iter__() -> Iterator[Chunk]

Iterate over chunks one at a time (triggers execution).

collect

def collect(
    *, optimize: OptimizationProfile | None = None
) -> ChunkStore

Run the pipeline and materialize all chunks into a ChunkStore.

By default, only the single-pass compaction that happens naturally during chunk insertion is applied. Pass optimize=OptimizationProfile.LIVE or optimize=OptimizationProfile.OBJECT_STORE to run additional optimization (extra convergence passes, video GoP rebatching) tuned for the chosen target.

PARAMETER	DESCRIPTION
`optimize`	If `None` (default), no extra optimization is performed beyond the single pass that happens on insert. Otherwise, apply the given profile after insertion. TYPE: `OptimizationProfile \| None` DEFAULT: `None`

Examples:

Run with the object-store-tuned profile:

store = reader.stream().collect(optimize=OptimizationProfile.OBJECT_STORE)

drop

def drop(
    *,
    content: ContentFilter
    | str
    | Sequence[str]
    | None = None,
    has_timeline: str | None = None,
    is_static: bool | None = None,
    components: ComponentDescriptor
    | str
    | Sequence[ComponentDescriptor | str]
    | None = None,
) -> LazyChunkStream

Drop the matching portion of each chunk; keep the rest. Consumes this stream.

Complement of filter(): what filter() would keep is discarded, what it would discard is kept.

PARAMETER	DESCRIPTION
`content`	Entity path filter. Accepts a single expression, a list of expressions, or a `ContentFilter` object. TYPE: `ContentFilter \| str \| Sequence[str] \| None` DEFAULT: `None`
`has_timeline`	Only drop chunks that have a column for this timeline. TYPE: `str \| None` DEFAULT: `None`
`is_static`	If `True`, drop only static chunks. If `False`, drop only temporal chunks. TYPE: `bool \| None` DEFAULT: `None`
`components`	Drop the listed component columns. Accepts `ComponentDescriptor` objects or `str` component identifiers (e.g. `"Points3D:positions"`). A single value or a list are both accepted. TYPE: `ComponentDescriptor \| str \| Sequence[ComponentDescriptor \| str] \| None` DEFAULT: `None`

filter

def filter(
    *,
    content: ContentFilter
    | str
    | Sequence[str]
    | None = None,
    has_timeline: str | None = None,
    is_static: bool | None = None,
    components: ComponentDescriptor
    | str
    | Sequence[ComponentDescriptor | str]
    | None = None,
) -> LazyChunkStream

Keep the matching portion of each chunk; drop the rest. Consumes this stream.

All criteria are combined with AND. For chunk-level predicates (content, has_timeline, is_static) the chunk either passes or is dropped entirely. For components, the chunk is split by component columns: only matching component columns are kept (timelines and entity path are preserved). When a list is given, any column matching any of the listed components is kept (OR semantics). Chunks that contain none of the listed components are dropped entirely.

If a chunk fails any predicate, it is dropped entirely -- no component splitting occurs.

PARAMETER	DESCRIPTION
`content`	Entity path filter. Accepts a single expression, a list of expressions, or a `ContentFilter` object. TYPE: `ContentFilter \| str \| Sequence[str] \| None` DEFAULT: `None`
`has_timeline`	Only keep chunks that have a column for this timeline. TYPE: `str \| None` DEFAULT: `None`
`is_static`	If `True`, keep only static chunks. If `False`, keep only temporal chunks. TYPE: `bool \| None` DEFAULT: `None`
`components`	Keep only the listed component columns. Accepts `ComponentDescriptor` objects or `str` component identifiers (e.g. `"Points3D:positions"`). A single value or a list are both accepted. TYPE: `ComponentDescriptor \| str \| Sequence[ComponentDescriptor \| str] \| None` DEFAULT: `None`

flat_map

def flat_map(
    fn: Callable[[Chunk], Iterable[Chunk]],
) -> LazyChunkStream

Apply a Python function to each chunk, producing zero or more output chunks. Consumes this stream.

Runs in Python (GIL-bound, sequential).

from_iter `staticmethod`

def from_iter(chunks: Iterable[Chunk]) -> LazyChunkStream

Wrap a Python iterable of Chunks into a LazyChunkStream.

Enables user-defined sources and the generator escape hatch.

lenses

def lenses(
    lenses: Sequence[Lens] | Lens,
    *,
    output_mode: Literal[
        "drop_unmatched", "forward_unmatched", "forward_all"
    ] = "drop_unmatched",
    content: ContentFilter
    | str
    | Sequence[str]
    | None = None,
) -> LazyChunkStream

Apply lenses to transform chunk data. Consumes this stream.

Each lens matches chunks by entity path and input component, then transforms the data according to its output specifications.

PARAMETER	DESCRIPTION
`lenses`	One or more `Lens` objects. TYPE: `Sequence[Lens] \| Lens`
`output_mode`	How to handle unmatched chunks: `"forward_all"`: forward both transformed and original data `"forward_unmatched"`: forward transformed if matched, otherwise original `"drop_unmatched"`: only forward transformed data (default) TYPE: `Literal['drop_unmatched', 'forward_unmatched', 'forward_all']` DEFAULT: `'drop_unmatched'`
`content`	Optional entity path filter. When set, lenses are applied only to chunks whose entity path matches; non-matching chunks pass through unchanged regardless of `output_mode`. TYPE: `ContentFilter \| str \| Sequence[str] \| None` DEFAULT: `None`

map

def map(fn: Callable[[Chunk], Chunk]) -> LazyChunkStream

Apply a Python function to each chunk, producing exactly one output chunk. Consumes this stream.

Runs in Python (GIL-bound, sequential). For transforms that may produce zero or many chunks, use flat_map instead.

merge `staticmethod`

def merge(*streams: LazyChunkStream) -> LazyChunkStream

Merge chunks from multiple streams into one. Consumes all input streams.

All inputs execute concurrently. Chunks are yielded as they become available. Within each input, chunk order is preserved. Across inputs, ordering is non-deterministic.

split

def split(
    *,
    content: ContentFilter
    | str
    | Sequence[str]
    | None = None,
    has_timeline: str | None = None,
    is_static: bool | None = None,
    components: ComponentDescriptor
    | str
    | Sequence[ComponentDescriptor | str]
    | None = None,
) -> tuple[LazyChunkStream, LazyChunkStream]

Split into (matching, non_matching). Consumes this stream.

Equivalent to (stream.filter(…), stream.drop(…)), but the upstream executes only once. merge(matching, non_matching) reconstructs the original stream in a semantically lossless way (component-wise chunk splitting is not undone).

Both branches share the same upstream -- it executes once. Both branches MUST be consumed for the pipeline to complete (dropping an unconsumed branch is fine and unblocks the other).

PARAMETER	DESCRIPTION
`content`	Entity path filter. Accepts a single expression, a list of expressions, or a `ContentFilter` object. TYPE: `ContentFilter \| str \| Sequence[str] \| None` DEFAULT: `None`
`has_timeline`	Only match chunks that have a column for this timeline. TYPE: `str \| None` DEFAULT: `None`
`is_static`	If `True`, match only static chunks. If `False`, match only temporal chunks. TYPE: `bool \| None` DEFAULT: `None`
`components`	Match the listed component columns. Accepts `ComponentDescriptor` objects or `str` component identifiers (e.g. `"Points3D:positions"`). A single value or a list are both accepted. TYPE: `ComponentDescriptor \| str \| Sequence[ComponentDescriptor \| str] \| None` DEFAULT: `None`

to_chunks

def to_chunks() -> list[Chunk]

Run the pipeline and return all chunks as a list.

write_rrd

def write_rrd(
    path: str | Path,
    *,
    application_id: str,
    recording_id: str,
) -> None

Run the pipeline and write all chunks to an RRD file.

The caller must provide application_id and recording_id explicitly.

LazyStore

Index-based, on-demand chunk store.

The manifest is held in memory (so schema(), summary(), and __len__ work without loading any chunks), but chunk data is loaded only when requested.

Example: lazy = RrdReader("recording.rrd").store()

Use stream() to process chunks through the lazy pipeline, or write_rrd() to persist to disk. To fully materialize into a ChunkStore, call lazy.stream().collect().

len

def __len__() -> int

Return the number of chunks described by the manifest.

schema

def schema() -> Schema

The schema describing all columns in this store, derived from the manifest.

stream

def stream() -> LazyChunkStream

Return a lazy stream over all chunks in this store.

summary

def summary() -> str

Compact, deterministic summary of every chunk in the store.

Built from the manifest; no chunk data is loaded. Each line describes one chunk:

{entity_path}  rows={n}  static={True|False}  timelines=[…]  cols=[…]

Useful for snapshot testing.

write_rrd

def write_rrd(
    path: str | Path,
    *,
    application_id: str,
    recording_id: str,
) -> None

Write all chunks to an RRD file.

The caller must provide application_id and recording_id explicitly.

McapReader

Read chunks from an MCAP file.

path `property`

path: Path

The file path of the MCAP file.

init

def __init__(
    path: str | Path,
    *,
    timeline_type: Literal[
        "timestamp", "duration"
    ] = "timestamp",
    timestamp_offset_ns: int | None = None,
    decoders: Sequence[str] | None = None,
    include_topic_regex: Sequence[str] | None = None,
    exclude_topic_regex: Sequence[str] | None = None,
    start_time_ns: int | None = None,
    end_time_ns: int | None = None,
    recover: bool = False,
) -> None

Construct a new MCAP reader.

PARAMETER	DESCRIPTION
`path`	Path to the `.mcap` file to read. TYPE: `str \| Path`
`timeline_type`	Whether to interpret the MCAP `log_time` column as wall-clock timestamps ("timestamp") or as nanosecond durations ("duration"). TYPE: `Literal['timestamp', 'duration']` DEFAULT: `'timestamp'`
`timestamp_offset_ns`	Optional offset in nanoseconds to add to all `TimestampNs` time columns. TYPE: `int \| None` DEFAULT: `None`
`decoders`	Optional list of MCAP decoder identifiers to enable. If omitted, all available decoders are enabled. Use `McapReader.available_decoders` to enumerate them. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`include_topic_regex`	Optional list of regex patterns. If provided, only topics matching at least one pattern are decoded. Patterns use RE2 syntax and are not implicitly anchored. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`exclude_topic_regex`	Optional list of regex patterns. Topics matching any pattern are skipped. Applied after includes. Same syntax as `include_topic_regex`. TYPE: `Sequence[str] \| None` DEFAULT: `None`
`start_time_ns`	Optional inclusive lower bound on the raw MCAP `log_time` (nanoseconds). Messages before this time are skipped. `None` leaves the range open at the start. TYPE: `int \| None` DEFAULT: `None`
`end_time_ns`	Optional exclusive upper bound on the raw MCAP `log_time` (nanoseconds). Messages at or after this time are skipped. `None` leaves the range open at the end. TYPE: `int \| None` DEFAULT: `None`
`recover`	Whether to recover a missing or invalid MCAP summary in memory. Our reader normally requires the summary + chunk index that live at the end of the file, so an interrupted recording (valid start, truncated tail, no footer/summary) fails to read. When `recover` is set, the summary is reconstructed from a front-to-back scan instead: the incomplete tail chunk/record is dropped with a warning, and any channel declared only in the dropped tail is lost. The recovered statistics only count the channels and messages that could be recovered. Healthy files are unaffected. TYPE: `bool` DEFAULT: `False`

available_decoders `staticmethod`

def available_decoders() -> list[str]

Return the list of all supported MCAP decoder identifiers.

stream

def stream(
    *,
    start_time_ns: int | None = None,
    end_time_ns: int | None = None,
) -> LazyChunkStream

Return a lazy stream over the chunks in the MCAP file.

start_time_ns and end_time_ns override the values passed to the constructor, for this scan only. If either start_time_ns or end_time_ns are provided both are reset.

time_bounds

def time_bounds() -> tuple[int, int]

Return the (min, max) MCAP log_time bounds (nanoseconds, inclusive).

MetricsCollector

Accumulator yielded by query_metrics on __enter__.

Use last_query() / queries to read snapshots accumulated so far; both are non-destructive. On context-manager exit any remaining snapshots are drained into this collector and the scope is unbound from the ContextVar, so the collector is still readable after the scope ends.

queries `property`

queries: list[QueryMetrics]

Non-destructive snapshot of all queries captured so far.

clear

def clear() -> None

Drop all captured snapshots from both the Rust buffer and this collector.

last_query

def last_query() -> QueryMetrics | None

Most recently captured query, or None if none yet.

Mp4Reader

Read chunks from an MP4 file.

entity_path `property`

entity_path: str

The entity path under which chunks are emitted.

path `property`

path: Path

The file path of the MP4 file.

init

def __init__(
    path: str | Path,
    *,
    mode: Literal["stream"] = "stream",
    chunk_by_gop: bool = True,
    timeline_name: str = "video",
    timeline_type: Literal[
        "duration", "timestamp"
    ] = "duration",
    transcode: Mp4TranscodeOptions | None = None,
    entity_path: str | None = None,
) -> None

def __init__(
    path: str | Path,
    *,
    mode: Literal["asset"],
    timeline_name: str = "video",
    timeline_type: Literal[
        "duration", "timestamp"
    ] = "duration",
    entity_path: str | None = None,
) -> None

def __init__(
    path: str | Path,
    *,
    mode: Literal["asset", "stream"] = "stream",
    chunk_by_gop: bool = True,
    timeline_name: str = "video",
    timeline_type: Literal[
        "duration", "timestamp"
    ] = "duration",
    transcode: Mp4TranscodeOptions | None = None,
    entity_path: str | None = None,
) -> None

Construct a new MP4 reader.

PARAMETER	DESCRIPTION
`path`	Path to the `.mp4` file to read. TYPE: `str \| Path`
`mode`	How to convert the mp4 into chunks. `"stream"` (default): emit a static `VideoStream(codec=…)` chunk followed by per-GOP (or per-sample) `VideoSample` chunks. The mp4 must use a codec representable as `VideoCodec`. A source containing B-frames — or any source for which a transform is requested via `transcode` — is transcoded with FFmpeg into an equivalent B-frame-free stream before emission, which requires an `ffmpeg` executable. `"asset"`: emit an `AssetVideo` blob chunk plus a `VideoFrameReference` index chunk, matching the behavior of `rerun video.mp4`. TYPE: `Literal['asset', 'stream']` DEFAULT: `'stream'`
`chunk_by_gop`	Only meaningful when `mode="stream"`. When `True` (default), each emitted Rerun chunk contains a keyframe plus all dependent samples up to (but not including) the next keyframe. When `False`, each sample becomes its own one-row Rerun chunk. Passing `chunk_by_gop=False` together with `mode="asset"` raises `ValueError`. TYPE: `bool` DEFAULT: `True`
`timeline_name`	Name of the timeline used for stream-mode samples and for the `VideoFrameReference` index chunk in asset mode. Defaults to `"video"`. TYPE: `str` DEFAULT: `'video'`
`timeline_type`	How to interpret the timeline values. The emitted values are the mp4 PTS (nanoseconds since the start of the video) only the declared Arrow type changes: `"duration"` (default): the values are typed as a duration, the natural mp4 PTS interpretation. `"timestamp"`: the same PTS values, typed as nanoseconds since the Unix epoch. The reader does not shift them, so until you retag them — via a downstream `.map(...)` on the chunk stream with caller-supplied wall-clock times (e.g. from a trajectory file) — they render as timestamps near 1970. TYPE: `Literal['duration', 'timestamp']` DEFAULT: `'duration'`
`transcode`	Only meaningful when `mode="stream"`. An `Mp4TranscodeOptions` describing an optional re-encode. TYPE: `Mp4TranscodeOptions \| None` DEFAULT: `None`
`entity_path`	Entity path under which chunks are emitted. When `None` (default), the entity path is derived from the absolute file path (e.g. `foo/video.mp4` run from `/data` becomes `/data/foo/video.mp4`). The path is resolved to absolute up front, so the result is independent of any later change to the working directory. TYPE: `str \| None` DEFAULT: `None`

stream

def stream() -> LazyChunkStream

Return a lazy stream over all chunks in the MP4 file.

Mp4TranscodeOptions `dataclass`

How to transcode an mp4.

ffmpeg_override `class-attribute` `instance-attribute`

ffmpeg_override: str | Path | None = None

Override the ffmpeg executable used to transcode. When None (default), ffmpeg is looked up on PATH. Ignored when no transcode happens.

gop_size `class-attribute` `instance-attribute`

gop_size: int | None = None

Force a keyframe every gop_size frames in the transcoded output. Requesting it triggers a re-encode. None (default) keeps the encoder's default GOP.

output_codec `class-attribute` `instance-attribute`

output_codec: VideoCodec | None = None

Re-encode to this VideoCodec instead of keeping the source codec; the emitted VideoStream codec follows it. None (default) keeps the source codec.

try_gpu `class-attribute` `instance-attribute`

try_gpu: bool = False

Try to use a hardware (GPU) encoder if the local FFmpeg provides one for the output codec, otherwise fall back to software (best-effort). GPU encoding is drawn from the NVENC and VideoToolbox families only, so it realistically applies to H264/H265 and, on newer NVIDIA hardware, AV1. VP8/VP9 always fall back to software — their only GPU encoders are Intel QSV/VAAPI, which are not yet used. Has no effect unless a transcode is already happening.

MutateLens

A mutate lens that modifies the input component in-place.

Mutate lenses apply a selector transformation to the input component, replacing it in the chunk. By default, new row IDs are generated. Pass keep_row_ids=True to preserve original row IDs.

Example usage::

lens = MutateLens("Imu:accel", Selector(".x"))

init

def __init__(
    input_component: str,
    selector: Selector | str,
    *,
    keep_row_ids: bool = False,
) -> None

Create a new mutate lens.

PARAMETER	DESCRIPTION
`input_component`	The component identifier to modify in-place. TYPE: `str`
`selector`	A `Selector` or selector query string to apply. TYPE: `Selector \| str`
`keep_row_ids`	When `True`, preserve the original row IDs. TYPE: `bool` DEFAULT: `False`

OptimizationProfile `dataclass`

Named optimization profile passed to LazyChunkStream.collect(optimize=...).

Two presets:

OptimizationProfile.LIVE: small chunks tuned for the live Viewer workflow.
OptimizationProfile.OBJECT_STORE: large chunks tuned for object-store-backed query and streaming (e.g. a catalog server).

The presets are fully concrete: every field has a value. Custom profiles built by calling OptimizationProfile(...) directly may pass None on the threshold fields to fall back to the SDK's internal default (OptimizationProfile.LIVE's thresholds).

LIVE `class-attribute`

LIVE: OptimizationProfile

Optimized for the live Viewer workflow: small chunks for low-latency rendering and fine-grained time-panel precision.

OBJECT_STORE `class-attribute`

OBJECT_STORE: OptimizationProfile

Optimized for object-store-backed storage (e.g. a catalog server): larger chunks tuned for query throughput and streaming over the network.

extra_passes `class-attribute` `instance-attribute`

extra_passes: int = 50

Number of extra convergence passes run after the initial insert.

fix_keyframe `class-attribute` `instance-attribute`

fix_keyframe: bool = False

If True, any user-supplied VideoStream:is_keyframe data is dropped and re-derived from the encoded samples during video rebatching.

gop_batching `class-attribute` `instance-attribute`

gop_batching: bool = True

If True (default), video stream chunks are rebatched to align with GoP (keyframe) boundaries after normal compaction.

GoP rebatching never splits a GoP across chunks, so streams with long keyframe intervals can produce chunks much larger than max_bytes.

max_bytes `class-attribute` `instance-attribute`

max_bytes: int | None = None

Chunk size threshold in bytes. None means use LIVE's default.

max_rows `class-attribute` `instance-attribute`

max_rows: int | None = None

Maximum rows per sorted chunk. None means use LIVE's default.

max_rows_if_unsorted `class-attribute` `instance-attribute`

max_rows_if_unsorted: int | None = None

Maximum rows per unsorted chunk. None means use LIVE's default.

split_size_ratio `class-attribute` `instance-attribute`

split_size_ratio: float | None = None

If set, split chunks so no two archetype groups sharing a chunk differ in byte size by more than this factor. Values should be >= 1; at 1.0, every archetype is forced into its own chunk.

This keeps large columns (images, videos, blobs) out of the same chunk as small columns (scalars, transforms, text), so the viewer can fetch just the small columns without dragging along the large payload. Components belonging to the same archetype are always kept together.

A good starting value is 10.0. If None (default), no splitting is performed.

ParquetReader

Read chunks from a Parquet file.

The reader turns raw parquet columns into grouped, time-indexed Chunks of struct/scalar components. To map those struct fields into Rerun archetypes (translation, rotation, scalars, …), apply lenses to the resulting .stream() — see DeriveLens:

Example

from rerun.experimental import ParquetReader, DeriveLens, IndexColumn

store = (
    ParquetReader(path, index_columns=[IndexColumn.sequence("frame_index")])
    .stream()
    .lenses(
        [
            DeriveLens("data", output_entity="/pose")
            .to_translation("pos_x", "pos_y", "pos_z")
            .to_quaternion("quat_x", "quat_y", "quat_z", "quat_w")
        ],
        content="/transform",
    )
    .collect()
)

path `property`

path: Path

The file path of the Parquet file.

init

def __init__(
    path: str | Path,
    *,
    entity_path_prefix: str | None = None,
    column_grouping: str = "prefix",
    delimiter: str = "_",
    prefixes: list[str] | None = None,
    use_structs: bool = True,
    static_columns: list[str] | None = None,
    index_columns: list[IndexColumn] | None = None,
) -> None

Load a parquet file with configurable column grouping.

PARAMETER	DESCRIPTION
`path`	Path to the `.parquet` file. TYPE: `str \| Path`
`entity_path_prefix`	Optional prefix for all entity paths (e.g. `"/world"`). TYPE: `str \| None` DEFAULT: `None`
`column_grouping`	How to group columns into chunks. `"prefix"` splits column names on `delimiter` and groups by the first segment. `"individual"` gives each column its own chunk. `"explicit_prefixes"` groups columns by the explicit prefix strings in `prefixes`. TYPE: `str` DEFAULT: `'prefix'`
`delimiter`	Character used to split column names when `column_grouping="prefix"`. TYPE: `str` DEFAULT: `'_'`
`prefixes`	Explicit prefix strings for grouping columns. Required when `column_grouping="explicit_prefixes"`. Columns starting with a prefix are grouped together; the prefix is stripped from the component name. Prefixes are tried longest-first to avoid ambiguity. TYPE: `list[str] \| None` DEFAULT: `None`
`use_structs`	When `True` (default) and `column_grouping="prefix"` or `"explicit_prefixes"`, columns sharing a prefix are packed into a single Arrow `Struct` component. When `False`, each column becomes a separate component (the pre-struct layout). Ignored when `column_grouping="individual"`. TYPE: `bool` DEFAULT: `True`
`static_columns`	Column names whose values are constant across all rows. These are emitted once as timeless/static data. An error is raised if a listed column contains varying values. TYPE: `list[str] \| None` DEFAULT: `None`
`index_columns`	Columns to use as timeline indices, each built with `IndexColumn`, e.g. `IndexColumn.timestamp("ts", input_unit="ms")` or `IndexColumn.sequence("frame_index")`. When omitted, a synthetic `row_index` sequence timeline is generated automatically (one entry per row). TYPE: `list[IndexColumn] \| None` DEFAULT: `None`

stream

def stream() -> LazyChunkStream

Return a lazy stream over all chunks in the Parquet file.

QueryMetrics `dataclass`

One query's metrics, captured at the moment its last per-partition stream finished.

Mirrors the Rust-side re_datafusion::QuerySnapshot. The same numbers are produced via three transports: this dataclass (Python), DataFusion's EXPLAIN ANALYZE, and the PostHog analytics OTLP span. Field naming differs across the three:

Timing fields here are datetime.timedelta (total_duration, time_to_first_chunk, …). EXPLAIN ANALYZE uses DataFusion Time metrics, which print their own units. The OTLP analytics attributes keep an explicit _us suffix and carry integer microseconds (total_duration_us, time_to_first_chunk_us, …) because OTLP attribute values are scalar (i64 / f64 / bool / string) and can't carry a duration natively.
query_chunks_per_segment_mean is a float and does not appear in EXPLAIN ANALYZE, since DataFusion Count metrics are integer-only. The corresponding _min / _max integer fields are surfaced in all three transports.

fetch_direct_max_attempt is the true maximum attempt number across all partitions.

fetch_bytes `property`

fetch_bytes: int

Total bytes fetched across both gRPC and direct transports.

fetch_requests `property`

fetch_requests: int

Total fetch requests across both gRPC and direct transports.

RrdReader

Read chunks from an RRD file.

Use recordings() or blueprints() to discover what stores exist in the file, then stream() or store() to access a specific one. When no store is specified, the first recording store is used.

path `property`

path: Path

The file path of the RRD file.

blueprints

def blueprints() -> list[StoreEntry]

List the blueprint entries in this RRD file.

recordings

def recordings() -> list[StoreEntry]

List the recording entries in this RRD file.

store

def store(*, store: StoreEntry | None = None) -> LazyStore

Open a specific store as a LazyStore.

Reads the manifest immediately; chunk data is loaded on demand. Legacy RRDs without a footer/manifest are not supported here — use RrdReader(...).stream().collect() for those.

PARAMETER	DESCRIPTION
`store`	Which store to load. If `None`, uses the first recording store. TYPE: `StoreEntry \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`ValueError`	If the specified store is not in this RRD file, or `None` was passed and the file contains no recording stores.

stream

def stream(
    *, store: StoreEntry | None = None
) -> LazyChunkStream

Return a lazy stream over chunks from a store.

PARAMETER	DESCRIPTION
`store`	Which store to stream. If `None`, uses the first recording store. TYPE: `StoreEntry \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`ValueError`	If the specified store is not in this RRD file, or `None` was passed and the file contains no recording stores.

Selector

A jq-like query selector for Arrow arrays.

Selectors provide a path-based query language (inspired by jq) that operates on Arrow arrays in a columnar fashion.

Syntax overview:

.field — access a named field in a struct
[] — iterate over every element of a list
[N] — index into a list by position
? — error suppression / optional operator
! — assert non-null
| — pipe the output of one expression to another

Example usage::

selector = Selector(".location")
result = selector.execute(my_struct_array)

Selectors can also be piped into Python functions::

selector = Selector(".values").pipe(lambda arr: pa.compute.multiply(arr, 2))
result = selector.execute(my_struct_array)

init

def __init__(query: str) -> None

Parse a selector from a query string.

PARAMETER	DESCRIPTION
`query`	The selector query string (e.g. ".field", ".foo \| .bar"). TYPE: `str`

execute

def execute(source: Array) -> Array | None

Execute this selector against a pyarrow array.

PARAMETER	DESCRIPTION
`source`	The input Arrow array to query. TYPE: `Array`

RETURNS	DESCRIPTION
`The result array, or None if the selector's error was suppressed.`

execute_per_row

def execute_per_row(source: Array) -> Array | None

Execute this selector against each row of a pyarrow list array.

The output is guaranteed to have the same number of rows as the input.

PARAMETER	DESCRIPTION
`source`	The input Arrow list array to query. TYPE: `Array`

RETURNS	DESCRIPTION
`The result list array, or None if the selector's error was suppressed.`

pipe

def pipe(
    func: Callable[[Array], Array | None] | Selector,
) -> Selector

Pipe the output of this selector through a transformation function or another selector.

Returns a new selector; the original is not modified.

PARAMETER	DESCRIPTION
`func`	A callable that accepts a `pyarrow.Array` and returns a `pyarrow.Array` or `None`, or another `Selector` to chain. TYPE: `Callable[[Array], Array \| None] \| Selector`

RETURNS	DESCRIPTION
A new [`Selector`][rerun.experimental.Selector] with the transformation applied.

StoreEntry

Describes a store found in an RRD file.

application_id `property`

application_id: str

The application ID of the store.

kind `property`

kind: Literal['recording', 'blueprint']

Store kind: "recording" or "blueprint".

recording_id `property`

recording_id: str

The recording ID of the store.

StreamingReader

Bases: Protocol

Protocol for readers that produce a sequential stream of chunks.

All readers provide stream() -> LazyChunkStream. Readers for indexable formats will additionally satisfy IndexedReader, which adds store() -> LazyStore and load() -> ChunkStore.

stream

def stream() -> LazyChunkStream

Return a lazy stream over all chunks from this source.

ViewerClient

A connection to an instance of a Rerun viewer.

Use the connect classmethod to attach to an already-running viewer, or spawn to start a fresh one (e.g. in headless mode for CI screenshots).

Spawned-viewer teardown:

Explicit close always terminates the spawned viewer.
For an attached viewer (detach_process=False), exiting a with block or garbage-collecting the client also terminates the viewer.
A detached viewer keeps running through with exits and garbage collection. Only an explicit close() shuts it down.

Warning

This API is experimental and may change or be removed in future versions.

url `property`

url: str

The rerun+http://…/proxy URL of the viewer this client is connected to.

init

def __init__(
    url: str = _DEFAULT_URL,
    *,
    _pid: int | None = None,
    _kill_on_exit: bool = False,
) -> None

Low-level constructor.

Prefer ViewerClient.connect or ViewerClient.spawn.

PARAMETER	DESCRIPTION
`url`	The URL to connect to. The scheme must be one of `rerun://`, `rerun+http://`, or `rerun+https://`, and the pathname must be `/proxy` — the same form accepted by `rerun.connect_grpc`. Defaults to `rerun+http://127.0.0.1:9876/proxy`. TYPE: `str` DEFAULT: `_DEFAULT_URL`
`_pid`	Internal — set by `spawn()` to the pid of the launched viewer so that `close()` can terminate it. TYPE: `int \| None` DEFAULT: `None`
`_kill_on_exit`	Internal — set by `spawn()` to indicate that implicit teardown (`__exit__`, `__del__`) should call `close()`. See the class docstring for the full teardown rules. TYPE: `bool` DEFAULT: `False`

close

def close() -> None

Close the client, terminating the spawned viewer.

Emits a UserWarning and is a no-op if there is no spawned viewer to terminate (either the client never spawned one, or it has already been closed). Safe to call multiple times — only the first call has an effect.

connect `classmethod`

def connect(url: str = _DEFAULT_URL) -> ViewerClient

Connect to an already-running viewer.

PARAMETER	DESCRIPTION
`url`	The URL to connect to. The scheme must be one of `rerun://`, `rerun+http://`, or `rerun+https://`, and the pathname must be `/proxy` — the same form accepted by `rerun.connect_grpc`. Defaults to `rerun+http://127.0.0.1:9876/proxy`. TYPE: `str` DEFAULT: `_DEFAULT_URL`

save_screenshot

def save_screenshot(
    file_path: str, view_id: str | UUID | None = None
) -> None

Save a screenshot to a file.

Warning

This API is experimental and may change or be removed in future versions.

PARAMETER	DESCRIPTION
`file_path`	The path where the screenshot will be saved. Important This path is relative to the viewer's filesystem, not the client's. If your viewer runs on a different machine, the screenshot will be saved there. TYPE: `str`
`view_id`	Optional view ID to screenshot. If None, screenshots the entire viewer. TYPE: `str \| UUID \| None` DEFAULT: `None`

send_table

def send_table(
    name: str,
    table: RecordBatch | list[RecordBatch] | DataFrame,
) -> None

Send a table to the viewer.

A table is represented as a dataframe defined by an Arrow record batch.

PARAMETER	DESCRIPTION
`name`	The table name. Note The table name serves as an identifier. If you send a table with the same name twice, the second table will replace the first one. TYPE: `str`
`table`	The Arrow RecordBatch containing the table data to send. TYPE: `RecordBatch \| list[RecordBatch] \| DataFrame`

spawn `classmethod`

def spawn(
    *,
    headless: bool = False,
    port: int = 9876,
    memory_limit: str = "75%",
    server_memory_limit: str = "1GiB",
    hide_welcome_screen: bool = False,
    detach_process: bool | None = None,
    executable_name: str = "rerun",
    executable_path: str | None = None,
) -> ViewerClient

Spawn a fresh viewer process and connect to it.

PARAMETER	DESCRIPTION
`headless`	Run the spawned viewer in headless mode (no OS window). The viewer still listens for gRPC connections, so the SDK can keep logging data and request screenshots via `save_screenshot`. A working graphics stack must be present — either a real GPU/driver or a software rasterizer like Mesa's `lavapipe`. In a bare CI container with no Vulkan adapter, the viewer panics on startup with "No graphics adapter was found". TYPE: `bool` DEFAULT: `False`
`port`	The port to listen on. TYPE: `int` DEFAULT: `9876`
`memory_limit`	An upper limit on how much memory the Rerun Viewer should use. When this limit is reached, Rerun will drop the oldest data. Example: `16GB` or `50%` (of system total). TYPE: `str` DEFAULT: `'75%'`
`server_memory_limit`	An upper limit on how much memory the gRPC server running in the same process as the Rerun Viewer should use. When this limit is reached, Rerun will drop the oldest data. Example: `16GB` or `50%` (of system total). Defaults to `1GiB`. TYPE: `str` DEFAULT: `'1GiB'`
`hide_welcome_screen`	Hide the normal Rerun welcome screen. TYPE: `bool` DEFAULT: `False`
`detach_process`	Detach the spawned viewer from this Python process. A detached viewer survives unexpected parent termination (e.g. crashes or terminal hang-up), `with` block exits, and garbage collection — to take it down you must call `close` explicitly. An attached viewer is killed by all of those. Defaults to `True` for a regular GUI viewer and `False` when `headless=True`, since a leftover invisible viewer is rarely what you want. TYPE: `bool \| None` DEFAULT: `None`
`executable_name`	Specifies the name of the Rerun executable. You can omit the `.exe` suffix on Windows. Defaults to `rerun`. TYPE: `str` DEFAULT: `'rerun'`
`executable_path`	Enforce a specific executable to use instead of searching through PATH for `executable_name`. Unspecified by default. TYPE: `str \| None` DEFAULT: `None`

query_metrics

def query_metrics() -> Iterator[MetricsCollector]

Capture DataFusion query metrics for every query that runs inside the with block.

Yields a MetricsCollector; read .last_query() or .queries mid-scope or after the scope exits.

The scope is bound to the current contextvars.Context: every re_datafusion query built from dataset.reader(…) while this scope is open contributes a QueryMetrics record. Nested query_metrics() scopes each see queries built inside them. Queries built in another thread or asyncio task that did not inherit this context (e.g. a raw threading.Thread rather than one started via contextvars.copy_context()) are not captured.

The collectors are bound to a query at reader() time, so a df built inside the with block whose .collect() runs after __exit__ still flows to the collector; a df built outside but executed inside does not.

Examples:

import rerun as rr
from rerun.experimental import query_metrics

client = rr.catalog.CatalogClient("rerun://…")
dataset = client.get_dataset(name="…")

with query_metrics() as m:
    df = dataset.reader(index="time_1").limit(100)
    df.collect()
    print(m.last_query())

send_chunks

def send_chunks(
    chunks: Chunk
    | LazyChunkStream
    | LazyStore
    | ChunkStore
    | Iterable[Chunk],
    *,
    recording: RecordingStream | None = None,
) -> None

Send chunks to a recording stream. Blocks until every chunk has been queued.

Note

For a LazyChunkStream and LazyStore inputs, this call triggers execution and/or loading and will block for the duration of this process.

PARAMETER DESCRIPTION

chunks

One of:

A single Chunk.
A LazyChunkStream — consume the stream and forward all chunks to the recording stream.
A LazyStore — send all chunks to the recording stream. This triggers loading all chunks from the source.
A ChunkStore — send all chunks to the recording stream (fast since all chunks are already loaded).
Any iterable of Chunk objects.

Source store identity (application_id, recording_id) is not preserved: chunks adopt the destination recording's identity.

TYPE: Chunk | LazyChunkStream | LazyStore | ChunkStore | Iterable[Chunk]

recording

Recording stream to send into. Defaults to the current active recording.

TYPE: RecordingStream | None DEFAULT: None

Experimental

rerun.experimental

Lens module-attribute

Chunk

entity_path property

id property

is_empty property

is_static property

num_columns property

num_rows property

timeline_names property

apply_lenses

apply_selector

format

from_columns classmethod

from_dataframe classmethod

from_record_batch classmethod

to_record_batch

with_entity_path

ChunkStore

__len__

from_chunks staticmethod

reader

schema

stream

summary

write_rrd

DatasetInfo dataclass

DeriveLens

__init__

to_component

to_packed_component

to_quaternion

to_rotation_axis_angle

to_scalars

to_scale

to_timeline

to_translation

Hdf5Reader

path property

attributes

datasets

groups

stream

IndexColumn dataclass

input_unit class-attribute instance-attribute

kind instance-attribute

path instance-attribute

duration classmethod

sequence classmethod

timestamp classmethod

IndexedReader

store

stream

LazyChunkStream

__iter__

collect

drop

filter

flat_map

from_iter staticmethod

lenses

map

merge staticmethod

split

to_chunks

write_rrd

LazyStore

__len__

schema

stream

summary

write_rrd

McapReader

path property

__init__

available_decoders staticmethod

stream

time_bounds

MetricsCollector

Lens `module-attribute`

entity_path `property`

id `property`

is_empty `property`

is_static `property`

num_columns `property`

num_rows `property`

timeline_names `property`

from_columns `classmethod`

from_dataframe `classmethod`

from_record_batch `classmethod`

len

from_chunks `staticmethod`

DatasetInfo `dataclass`

init

path `property`

IndexColumn `dataclass`

input_unit `class-attribute` `instance-attribute`

kind `instance-attribute`

path `instance-attribute`

duration `classmethod`

sequence `classmethod`

timestamp `classmethod`

iter

from_iter `staticmethod`

merge `staticmethod`

len

path `property`

init

available_decoders `staticmethod`

queries `property`

entity_path `property`

path `property`

init

Mp4TranscodeOptions `dataclass`

ffmpeg_override `class-attribute` `instance-attribute`

gop_size `class-attribute` `instance-attribute`

output_codec `class-attribute` `instance-attribute`

try_gpu `class-attribute` `instance-attribute`

init

OptimizationProfile `dataclass`

LIVE `class-attribute`

OBJECT_STORE `class-attribute`

extra_passes `class-attribute` `instance-attribute`

fix_keyframe `class-attribute` `instance-attribute`

gop_batching `class-attribute` `instance-attribute`

max_bytes `class-attribute` `instance-attribute`

max_rows `class-attribute` `instance-attribute`

max_rows_if_unsorted `class-attribute` `instance-attribute`

split_size_ratio `class-attribute` `instance-attribute`

path `property`

init

QueryMetrics `dataclass`

fetch_bytes `property`

fetch_requests `property`

path `property`

init

application_id `property`

kind `property`

recording_id `property`

url `property`

init

connect `classmethod`

spawn `classmethod`