Experimental
rerun.experimental
Experimental features for Rerun.
These features are not yet stable and may change in future releases without going through the normal deprecation cycle.
Chunk
A single chunk of data from a recording.
apply_lenses
Apply one or more lenses to this chunk, returning transformed chunks.
Each lens matches by input component. Columns not consumed by any matching lens are forwarded unchanged as a separate chunk.
If no lens matches the chunk (including when an empty list of lenses is passed), the original chunk is returned unchanged.
| PARAMETER | DESCRIPTION |
|---|---|
lenses
|
One or more |
| RETURNS | DESCRIPTION |
|---|---|
A list of [`Chunk`][] objects.
|
|
apply_selector
def apply_selector(
source: ComponentDescriptor | str,
selector: Selector | str,
) -> Chunk
Apply a selector to a single component, returning a new chunk with the component transformed.
All other columns (timelines, other components) are preserved unchanged. The source component's existing descriptor is preserved.
For better performance, prefer MutateLens
with apply_lenses
which processes multiple transformations in a single pass.
| PARAMETER | DESCRIPTION |
|---|---|
source
|
A
TYPE:
|
selector
|
A |
| RETURNS | DESCRIPTION |
|---|---|
A new [`Chunk`][rerun.experimental.Chunk] with the component transformed.
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the source component is not found in the chunk or the selector fails to evaluate. |
format
Format this chunk as a human-readable table string.
| PARAMETER | DESCRIPTION |
|---|---|
width
|
Fixed width for the table. Default: 240.
TYPE:
|
redact
|
If True, redact non-deterministic values (RowIds, ChunkIds, etc.) for stable snapshot testing. Default: False.
TYPE:
|
trim_metadata_keys
|
If True, trim the
TYPE:
|
from_columns
classmethod
def from_columns(
entity_path: str,
indexes: Iterable[TimeColumnLike],
columns: Iterable[ComponentColumn],
) -> Chunk
Create a Chunk from columns, mirroring the rerun.send_columns API.
A fresh chunk ID and sequential row IDs are auto-generated.
| PARAMETER | DESCRIPTION |
|---|---|
entity_path
|
The entity path for this chunk (e.g., "/camera/image").
TYPE:
|
indexes
|
The time columns for this chunk. Each
TYPE:
|
columns
|
The component columns for this chunk. Each
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If timeline and component column lengths don't match. |
Example
chunk = Chunk.from_columns(
"/robots/arm",
indexes=[rr.TimeColumn("frame", sequence=[0, 1, 2])],
columns=rr.Points3D.columns(positions=[[1, 2, 3], [4, 5, 6], [7, 8, 9]]),
)
from_dataframe
classmethod
def from_dataframe(
dataframe: DataframeLike,
*,
index: str | list[str] | None | _AutoIndex = AUTO_INDEX,
entity_path: str | None = None,
) -> Iterator[Chunk]
Lazily turn an Arrow-backed dataframe into chunks.
Accepts a Table, a RecordBatch, a
RecordBatchReader, or any object implementing the Arrow C
stream interface (__arrow_c_stream__) — most notably a datafusion.DataFrame (an optional
dependency).
Yields each chunk of
Chunk.from_record_batch applied to every
record batch in turn. See that method for the index and entity_path semantics.
| RAISES | DESCRIPTION |
|---|---|
TypeError
|
If |
ValueError
|
from_record_batch
classmethod
def from_record_batch(
record_batch: RecordBatch,
*,
index: str | list[str] | None | _AutoIndex = AUTO_INDEX,
entity_path: str | None = None,
) -> list[Chunk]
Interpret an Arrow RecordBatch as Rerun chunk data.
Each column of the batch is classified as a row-id column, index (timeline) column, or a component column. Component columns are then grouped per entity path, and one chunk per entity path is emitted.
The rerun:* arrow metadata, if it exists, drives the kind of each input column,
as well as the entity/archetype/component type for component columns.
If present, the row id column and chunk id metadata indicate that the batch represents
a fully identified chunk, e.g. as produced by Chunk.to_record_batch.
Both the row ids and chunk id are preserved under the following conditions:
- both are present in the input batch
- index is omitted
- entity_path is omitted
If any of these conditions are not met, it means that either the batch is not fully identified, or that the chunk data is reinterpreted (e.g. entity path rewriting). In that case, fresh row ids and chunk id are generated and used instead of the input ones.
| PARAMETER | DESCRIPTION |
|---|---|
record_batch
|
The Arrow record batch to interpret. Component columns may be either lists (one component batch per row) or plain arrays (wrapped as single-element lists automatically).
TYPE:
|
index
|
Determines which columns are index (timeline) columns. Each promoted column's
time type is taken from its Arrow datatype:
Note Static chunks with multiple rows are legitimate in some cases, but only the last row is visible from typical latest-at queries. An info-level message is emitted when this happens — except for an already-identified chunk that is preserved as-is (see above), which is passed through without this check.
TYPE:
|
entity_path
|
Default entity path for component columns that do not otherwise specify one.
Resolution order per component column is: its
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
One chunk per distinct entity path described by the batch, in first-seen column order.
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
In any of the following cases:
|
Notes
Column-name convention. When a component column carries no rerun:entity_path /
rerun:component metadata, its entity path is read from the column name: if the name starts
with / and contains a :, the first part of the column name is interpreted as the entity
path and the rest as the component identifier. Example: /point:Points3D:positions and
/metadata:foo.
Limitations/Future work
A batch that mixes static and temporal rows — aka where some index values are null — are
rejected. Handling this case requires row-splitting and generating a mix of temporal and
static chunks.
Recording-property columns (named property:…, mapping to the /__properties entity) are
not recognized by the column-name convention and are not mapped back to that entity.
with_entity_path
Return a copy of this chunk with a new entity path.
A fresh chunk ID is generated to avoid aliasing the original chunk in downstream caches and indices. Row IDs, timelines, and components are preserved as-is.
| PARAMETER | DESCRIPTION |
|---|---|
entity_path
|
The new entity path for the returned chunk (e.g.
TYPE:
|
ChunkStore
A fully-materialized, in-memory chunk store.
Build one from chunks via
ChunkStore.from_chunks, or
fully materialize an IndexedReader
via reader.stream().collect().
For lazy, on-demand chunk loading, see LazyStore.
Use stream() to process chunks through the lazy pipeline, or
write_rrd() to persist to disk.
from_chunks
staticmethod
def from_chunks(chunks: Sequence[Chunk]) -> ChunkStore
Build a ChunkStore from a sequence of chunks.
reader
def reader(
index: str | None,
*,
contents: ContentFilter | str | list[str] | None = None,
include_semantically_empty_columns: bool = False,
include_tombstone_columns: bool = False,
fill_latest_at: bool = False,
using_index_values: IndexValuesLike | None = None,
ctx: SessionContext | None = None,
) -> DataFrame
Build a DataFusion DataFrame over this store.
The returned DataFrame is data-equivalent to the result of round-tripping
the same chunks through write_rrd → rr.server.Server → dataset.reader(),
modulo the rerun_segment_id column (absent here because a single
ChunkStore has no segment concept).
| PARAMETER | DESCRIPTION |
|---|---|
index
|
The index (timeline) column to use, or
TYPE:
|
contents
|
Entity-path filter. A
TYPE:
|
include_semantically_empty_columns
|
Whether to include columns that are semantically empty.
TYPE:
|
include_tombstone_columns
|
Whether to include tombstone columns.
TYPE:
|
fill_latest_at
|
Whether to fill null values with the latest valid data.
TYPE:
|
using_index_values
|
Index values at which to resample data. When specified, this argument changes the way rows are returned. Instead
of returning the rows that exist in the data, one row is returned per
Don't use this argument for plain index slicing — use a DataFusion filter on the index column instead. For example:
TYPE:
|
ctx
|
DataFusion
TYPE:
|
summary
def summary() -> str
Compact, deterministic summary of every chunk in the store.
Each line describes one chunk:
{entity_path} rows={n} static={True|False} timelines=[…] cols=[…]
Useful for snapshot testing.
DeriveLens
A derive lens that creates new component/time columns from an input component.
Derive lenses extract fields from a component and produce new columns, optionally at a different entity and/or with new time columns.
Pass scatter=True to enable 1:N row mapping (exploding lists).
Example usage::
lens = (
DeriveLens("Imu:accel")
.to_component(rr.Scalars.descriptor_scalars(), Selector(".x"))
)
To write to an explicit target entity::
lens = (
DeriveLens("Imu:accel", output_entity="/out/x")
.to_component(rr.Scalars.descriptor_scalars(), Selector(".x"))
)
__init__
def __init__(
input_component: str,
*,
output_entity: str | None = None,
scatter: bool = False,
) -> None
Create a new derive lens.
| PARAMETER | DESCRIPTION |
|---|---|
input_component
|
The component identifier to match (e.g.
TYPE:
|
output_entity
|
Optional target entity path. When set, output is written to this entity instead of the input entity.
TYPE:
|
scatter
|
When
TYPE:
|
to_component
def to_component(
component: ComponentDescriptor | str,
selector: Selector | str,
*,
cast_to: DataType | Literal["auto"] | None = None,
) -> DeriveLens
Add a component output column.
| PARAMETER | DESCRIPTION |
|---|---|
component
|
A
TYPE:
|
selector
|
A |
cast_to
|
How to cast the produced column to match the target component. By default
( |
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the component added.
|
|
to_packed_component
def to_packed_component(
component: ComponentDescriptor | str,
*fields: str,
cast_to: DataType | Literal["auto"] | None = "auto",
) -> DeriveLens
Add a component output column by packing the provided fields in a fixed-size list.
| PARAMETER | DESCRIPTION |
|---|---|
component
|
A
TYPE:
|
*fields
|
Names of the struct fields to pack, in order. They must all resolve to the same datatype. At least one field is required.
TYPE:
|
cast_to
|
How to cast the packed column to match the target component. Defaults to |
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the packed component added.
|
|
to_quaternion
def to_quaternion(x: str, y: str, z: str, w: str) -> DeriveLens
Add a Transform3D:quaternion component from the provided paths.
| PARAMETER | DESCRIPTION |
|---|---|
x
|
Paths of the struct fields holding the quaternion components, in
TYPE:
|
y
|
Paths of the struct fields holding the quaternion components, in
TYPE:
|
z
|
Paths of the struct fields holding the quaternion components, in
TYPE:
|
w
|
Paths of the struct fields holding the quaternion components, in
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the quaternion added.
|
|
to_rotation_axis_angle
def to_rotation_axis_angle(
axis_x: str, axis_y: str, axis_z: str, angle: str
) -> DeriveLens
Add a Transform3D:rotation_axis_angle component from the provided paths.
| PARAMETER | DESCRIPTION |
|---|---|
axis_x
|
Paths of the struct fields holding the rotation axis components.
TYPE:
|
axis_y
|
Paths of the struct fields holding the rotation axis components.
TYPE:
|
axis_z
|
Paths of the struct fields holding the rotation axis components.
TYPE:
|
angle
|
Path of the struct field holding the rotation angle, in radians.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the rotation added.
|
|
to_scalars
def to_scalars(*fields: str) -> DeriveLens
Add a Scalars:scalars component from the provided path(s).
Each path becomes one scalar instance per row, so a single path yields one series and multiple paths yield one series each at the same entity.
| PARAMETER | DESCRIPTION |
|---|---|
*fields
|
Paths of the struct fields to read as scalars, in order. At least one is required.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the scalars added.
|
|
to_scale
def to_scale(x: str, y: str, z: str) -> DeriveLens
Add a Transform3D:scale component from the provided paths.
| PARAMETER | DESCRIPTION |
|---|---|
x
|
Paths of the struct fields holding the per-axis scale factors.
TYPE:
|
y
|
Paths of the struct fields holding the per-axis scale factors.
TYPE:
|
z
|
Paths of the struct fields holding the per-axis scale factors.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the scale added.
|
|
to_timeline
def to_timeline(
timeline_name: str,
timeline_type: Literal[
"sequence", "duration_ns", "timestamp_ns"
],
selector: Selector | str,
) -> DeriveLens
Add a time extraction column.
| PARAMETER | DESCRIPTION |
|---|---|
timeline_name
|
Name of the timeline to create.
TYPE:
|
timeline_type
|
Type of the timeline:
TYPE:
|
selector
|
A |
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the time column added.
|
|
to_translation
def to_translation(x: str, y: str, z: str) -> DeriveLens
Add a Transform3D:translation component from the provided paths.
| PARAMETER | DESCRIPTION |
|---|---|
x
|
Paths of the struct fields holding the translation components.
TYPE:
|
y
|
Paths of the struct fields holding the translation components.
TYPE:
|
z
|
Paths of the struct fields holding the translation components.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
A new [`DeriveLens`][rerun.experimental.DeriveLens] with the translation added.
|
|
IndexedReader
Bases: StreamingReader, Protocol
Protocol for readers backed by an index/manifest.
Extends StreamingReader: every IndexedReader also supports
stream() -> LazyChunkStream for pure-streaming processing.
Indexed readers expose a LazyStore view
over the source via store() — the manifest is read up-front; chunks load
on demand. To fully materialize into a
ChunkStore, call stream().collect().
LazyChunkStream
A lazy, composable pipeline over chunks.
Builder methods (filter, drop, split, map, flat_map, lenses, merge)
consume the input stream(s) and return new stream(s). A consumed stream cannot be
used again; attempting to do so raises a ValueError. This prevents accidental reuse
that would result in duplicate use of the same stream in a pipeline.
Terminal methods (to_chunks, __iter__, collect, write_rrd) do not consume
the stream — they run the pipeline and leave the stream usable. Each call creates a
fresh execution.
collect
def collect(
*, optimize: OptimizationProfile | None = None
) -> ChunkStore
Run the pipeline and materialize all chunks into a ChunkStore.
By default, only the single-pass compaction that happens naturally
during chunk insertion is applied. Pass optimize=OptimizationProfile.LIVE
or optimize=OptimizationProfile.OBJECT_STORE to run additional
optimization (extra convergence passes, video GoP rebatching) tuned for
the chosen target.
| PARAMETER | DESCRIPTION |
|---|---|
optimize
|
If Otherwise, apply the given profile after insertion.
TYPE:
|
Examples:
Run with the object-store-tuned profile:
store = reader.stream().collect(optimize=OptimizationProfile.OBJECT_STORE)
drop
def drop(
*,
content: ContentFilter
| str
| Sequence[str]
| None = None,
has_timeline: str | None = None,
is_static: bool | None = None,
components: ComponentDescriptor
| str
| Sequence[ComponentDescriptor | str]
| None = None,
) -> LazyChunkStream
Drop the matching portion of each chunk; keep the rest. Consumes this stream.
Complement of filter(): what filter() would keep is
discarded, what it would discard is kept.
| PARAMETER | DESCRIPTION |
|---|---|
content
|
Entity path filter. Accepts a single expression, a list of expressions,
or a
TYPE:
|
has_timeline
|
Only drop chunks that have a column for this timeline.
TYPE:
|
is_static
|
If
TYPE:
|
components
|
Drop the listed component columns. Accepts
TYPE:
|
filter
def filter(
*,
content: ContentFilter
| str
| Sequence[str]
| None = None,
has_timeline: str | None = None,
is_static: bool | None = None,
components: ComponentDescriptor
| str
| Sequence[ComponentDescriptor | str]
| None = None,
) -> LazyChunkStream
Keep the matching portion of each chunk; drop the rest. Consumes this stream.
All criteria are combined with AND. For chunk-level predicates (content,
has_timeline, is_static) the chunk either passes or is dropped
entirely. For components, the chunk is split by component columns:
only matching component columns are kept (timelines and entity
path are preserved). When a list is given, any column matching
any of the listed components is kept (OR semantics). Chunks that
contain none of the listed components are dropped entirely.
If a chunk fails any predicate, it is dropped entirely -- no component splitting occurs.
| PARAMETER | DESCRIPTION |
|---|---|
content
|
Entity path filter. Accepts a single expression, a list of expressions,
or a
TYPE:
|
has_timeline
|
Only keep chunks that have a column for this timeline.
TYPE:
|
is_static
|
If
TYPE:
|
components
|
Keep only the listed component columns. Accepts
TYPE:
|
flat_map
def flat_map(
fn: Callable[[Chunk], Iterable[Chunk]],
) -> LazyChunkStream
Apply a Python function to each chunk, producing zero or more output chunks. Consumes this stream.
Runs in Python (GIL-bound, sequential).
from_iter
staticmethod
def from_iter(chunks: Iterable[Chunk]) -> LazyChunkStream
Wrap a Python iterable of Chunks into a LazyChunkStream.
Enables user-defined sources and the generator escape hatch.
lenses
def lenses(
lenses: Sequence[Lens] | Lens,
*,
output_mode: Literal[
"drop_unmatched", "forward_unmatched", "forward_all"
] = "drop_unmatched",
content: ContentFilter
| str
| Sequence[str]
| None = None,
) -> LazyChunkStream
Apply lenses to transform chunk data. Consumes this stream.
Each lens matches chunks by entity path and input component, then transforms the data according to its output specifications.
| PARAMETER | DESCRIPTION |
|---|---|
lenses
|
One or more |
output_mode
|
How to handle unmatched chunks:
TYPE:
|
content
|
Optional entity path filter. When set, lenses are applied only to chunks
whose entity path matches; non-matching chunks pass through unchanged
regardless of
TYPE:
|
map
def map(fn: Callable[[Chunk], Chunk]) -> LazyChunkStream
Apply a Python function to each chunk, producing exactly one output chunk. Consumes this stream.
Runs in Python (GIL-bound, sequential). For transforms that may produce
zero or many chunks, use flat_map instead.
merge
staticmethod
def merge(*streams: LazyChunkStream) -> LazyChunkStream
Merge chunks from multiple streams into one. Consumes all input streams.
All inputs execute concurrently. Chunks are yielded as they become available. Within each input, chunk order is preserved. Across inputs, ordering is non-deterministic.
split
def split(
*,
content: ContentFilter
| str
| Sequence[str]
| None = None,
has_timeline: str | None = None,
is_static: bool | None = None,
components: ComponentDescriptor
| str
| Sequence[ComponentDescriptor | str]
| None = None,
) -> tuple[LazyChunkStream, LazyChunkStream]
Split into (matching, non_matching). Consumes this stream.
Equivalent to (stream.filter(…), stream.drop(…)), but the
upstream executes only once. merge(matching, non_matching)
reconstructs the original stream in a semantically lossless way
(component-wise chunk splitting is not undone).
Both branches share the same upstream -- it executes once. Both branches MUST be consumed for the pipeline to complete (dropping an unconsumed branch is fine and unblocks the other).
| PARAMETER | DESCRIPTION |
|---|---|
content
|
Entity path filter. Accepts a single expression, a list of expressions,
or a
TYPE:
|
has_timeline
|
Only match chunks that have a column for this timeline.
TYPE:
|
is_static
|
If
TYPE:
|
components
|
Match the listed component columns. Accepts
TYPE:
|
LazyStore
Index-based, on-demand chunk store.
The manifest is held in memory (so schema(), summary(), and __len__
work without loading any chunks), but chunk data is loaded only when
requested.
Example: lazy = RrdReader("recording.rrd").store()
Use stream() to process chunks through the lazy pipeline, or write_rrd()
to persist to disk. To fully materialize into a
ChunkStore, call lazy.stream().collect().
schema
def schema() -> Schema
The schema describing all columns in this store, derived from the manifest.
summary
def summary() -> str
Compact, deterministic summary of every chunk in the store.
Built from the manifest; no chunk data is loaded. Each line describes one chunk:
{entity_path} rows={n} static={True|False} timelines=[…] cols=[…]
Useful for snapshot testing.
McapReader
Read chunks from an MCAP file.
__init__
def __init__(
path: str | Path,
*,
timeline_type: Literal[
"timestamp", "duration"
] = "timestamp",
timestamp_offset_ns: int | None = None,
decoders: Sequence[str] | None = None,
include_topic_regex: Sequence[str] | None = None,
exclude_topic_regex: Sequence[str] | None = None,
) -> None
Construct a new MCAP reader.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the |
timeline_type
|
Whether to interpret the MCAP
TYPE:
|
timestamp_offset_ns
|
Optional offset in nanoseconds to add to all
TYPE:
|
decoders
|
Optional list of MCAP decoder identifiers to enable. If omitted, all
available decoders are enabled. Use
|
include_topic_regex
|
Optional list of regex patterns. If provided, only topics matching at least one pattern are decoded. Patterns use RE2 syntax and are not implicitly anchored. |
exclude_topic_regex
|
Optional list of regex patterns. Topics matching any pattern are
skipped. Applied after includes. Same syntax as |
available_decoders
staticmethod
Return the list of all supported MCAP decoder identifiers.
MetricsCollector
Accumulator yielded by query_metrics on __enter__.
Use last_query() / queries to read snapshots accumulated so far; both
are non-destructive. On context-manager exit any remaining snapshots are
drained into this collector and the scope is unbound from the
ContextVar, so the collector is still readable after the scope ends.
queries
property
queries: list[QueryMetrics]
Non-destructive snapshot of all queries captured so far.
clear
def clear() -> None
Drop all captured snapshots from both the Rust buffer and this collector.
last_query
def last_query() -> QueryMetrics | None
Most recently captured query, or None if none yet.
Mp4Reader
Read chunks from an MP4 file.
__init__
def __init__(
path: str | Path,
*,
mode: Literal["asset", "stream"] = "stream",
chunk_by_gop: bool = True,
timeline_name: str = "video",
timeline_type: Literal[
"duration", "timestamp"
] = "duration",
ffmpeg_override: str | Path | None = None,
entity_path: str | None = None,
) -> None
Construct a new MP4 reader.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the |
mode
|
How to convert the mp4 into chunks.
TYPE:
|
chunk_by_gop
|
Only meaningful when Passing
TYPE:
|
timeline_name
|
Name of the timeline used for stream-mode samples and for the
TYPE:
|
timeline_type
|
How to interpret the timeline values. Applies to both modes (the
stream-mode sample timeline and the asset-mode The emitted values are the mp4 PTS (nanoseconds since the start of the video) in both cases — only the declared Arrow type changes:
TYPE:
|
ffmpeg_override
|
Only meaningful when |
entity_path
|
Entity path under which chunks are emitted. When
TYPE:
|
MutateLens
A mutate lens that modifies the input component in-place.
Mutate lenses apply a selector transformation to the input component,
replacing it in the chunk. By default, new row IDs are generated.
Pass keep_row_ids=True to preserve original row IDs.
Example usage::
lens = MutateLens("Imu:accel", Selector(".x"))
OptimizationProfile
dataclass
Named optimization profile passed to LazyChunkStream.collect(optimize=...).
Two presets:
OptimizationProfile.LIVE: small chunks tuned for the live Viewer workflow.OptimizationProfile.OBJECT_STORE: large chunks tuned for object-store-backed query and streaming (e.g. a catalog server).
The presets are fully concrete: every field has a value. Custom profiles
built by calling OptimizationProfile(...) directly may pass None on the
threshold fields to fall back to the SDK's internal default
(OptimizationProfile.LIVE's thresholds).
LIVE
class-attribute
LIVE: OptimizationProfile
Optimized for the live Viewer workflow: small chunks for low-latency rendering and fine-grained time-panel precision.
OBJECT_STORE
class-attribute
OBJECT_STORE: OptimizationProfile
Optimized for object-store-backed storage (e.g. a catalog server): larger chunks tuned for query throughput and streaming over the network.
extra_passes
class-attribute
instance-attribute
extra_passes: int = 50
Number of extra convergence passes run after the initial insert.
fix_keyframe
class-attribute
instance-attribute
fix_keyframe: bool = False
If True, any user-supplied VideoStream:is_keyframe data is dropped and
re-derived from the encoded samples during video rebatching.
gop_batching
class-attribute
instance-attribute
gop_batching: bool = True
If True (default), video stream chunks are rebatched to align with GoP
(keyframe) boundaries after normal compaction.
GoP rebatching never splits a GoP across chunks, so streams with long
keyframe intervals can produce chunks much larger than max_bytes.
max_bytes
class-attribute
instance-attribute
max_bytes: int | None = None
Chunk size threshold in bytes. None means use LIVE's default.
max_rows
class-attribute
instance-attribute
max_rows: int | None = None
Maximum rows per sorted chunk. None means use LIVE's default.
max_rows_if_unsorted
class-attribute
instance-attribute
max_rows_if_unsorted: int | None = None
Maximum rows per unsorted chunk. None means use LIVE's default.
split_size_ratio
class-attribute
instance-attribute
split_size_ratio: float | None = None
If set, split chunks so no two archetype groups sharing a chunk differ in
byte size by more than this factor. Values should be >= 1; at 1.0,
every archetype is forced into its own chunk.
This keeps large columns (images, videos, blobs) out of the same chunk as small columns (scalars, transforms, text), so the viewer can fetch just the small columns without dragging along the large payload. Components belonging to the same archetype are always kept together.
A good starting value is 10.0. If None (default), no splitting is
performed.
ParquetReader
Read chunks from a Parquet file.
The reader turns raw parquet columns into grouped, time-indexed
Chunks of struct/scalar components. To map those
struct fields into Rerun archetypes (translation, rotation, scalars, …), apply
lenses to the resulting .stream() — see
DeriveLens:
Example
from rerun.experimental import ParquetReader, DeriveLens
store = (
ParquetReader(path, index_columns=[("frame_index", "sequence")])
.stream()
.lenses(
[
DeriveLens("data", output_entity="/pose")
.to_translation("pos_x", "pos_y", "pos_z")
.to_quaternion("quat_x", "quat_y", "quat_z", "quat_w")
],
content="/transform",
)
.collect()
)
__init__
def __init__(
path: str | Path,
*,
entity_path_prefix: str | None = None,
column_grouping: str = "prefix",
delimiter: str = "_",
prefixes: list[str] | None = None,
use_structs: bool = True,
static_columns: list[str] | None = None,
index_columns: list[
tuple[str, str] | tuple[str, str, str]
]
| None = None,
) -> None
Load a parquet file with configurable column grouping.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the |
entity_path_prefix
|
Optional prefix for all entity paths (e.g.
TYPE:
|
column_grouping
|
How to group columns into chunks.
TYPE:
|
delimiter
|
Character used to split column names when
TYPE:
|
prefixes
|
Explicit prefix strings for grouping columns. Required when
|
use_structs
|
When
TYPE:
|
static_columns
|
Column names whose values are constant across all rows. These are emitted once as timeless/static data. An error is raised if a listed column contains varying values. |
index_columns
|
List of columns to use as timeline indices. Each entry is a tuple:
The
The When omitted, a synthetic
TYPE:
|
QueryMetrics
dataclass
One query's metrics, captured at the moment its last per-partition stream finished.
Mirrors the Rust-side re_datafusion::QuerySnapshot. The same numbers are
produced via three transports: this dataclass (Python), DataFusion's
EXPLAIN ANALYZE, and the PostHog analytics OTLP span. Field naming
differs across the three:
- Timing fields here are
datetime.timedelta(total_duration,time_to_first_chunk, …).EXPLAIN ANALYZEuses DataFusionTimemetrics, which print their own units. The OTLP analytics attributes keep an explicit_ussuffix and carry integer microseconds (total_duration_us,time_to_first_chunk_us, …) because OTLP attribute values are scalar (i64/f64/bool/string) and can't carry a duration natively. query_chunks_per_segment_meanis afloatand does not appear inEXPLAIN ANALYZE, since DataFusionCountmetrics are integer-only. The corresponding_min/_maxinteger fields are surfaced in all three transports.
Note: fetch_direct_max_attempt is the per-partition max summed across
partitions (rather than the cross-partition true max), because
DataFusion's MetricsSet::Count lacks a fetch_max aggregation. For
single-partition queries the two are identical.
RrdReader
Read chunks from an RRD file.
Use recordings() or blueprints() to discover what stores exist in the file,
then stream() or store() to access a specific one. When no store is
specified, the first recording store is used.
store
def store(*, store: StoreEntry | None = None) -> LazyStore
Open a specific store as a LazyStore.
Reads the manifest immediately; chunk data is loaded on demand.
Legacy RRDs without a footer/manifest are not supported here — use
RrdReader(...).stream().collect() for those.
| PARAMETER | DESCRIPTION |
|---|---|
store
|
Which store to load. If
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the specified store is not in this RRD file, or |
stream
def stream(
*, store: StoreEntry | None = None
) -> LazyChunkStream
Return a lazy stream over chunks from a store.
| PARAMETER | DESCRIPTION |
|---|---|
store
|
Which store to stream. If
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the specified store is not in this RRD file, or |
Selector
A jq-like query selector for Arrow arrays.
Selectors provide a path-based query language (inspired by jq) that operates on Arrow arrays in a columnar fashion.
Syntax overview:
.field— access a named field in a struct[]— iterate over every element of a list[N]— index into a list by position?— error suppression / optional operator!— assert non-null|— pipe the output of one expression to another
Example usage::
selector = Selector(".location")
result = selector.execute(my_struct_array)
Selectors can also be piped into Python functions::
selector = Selector(".values").pipe(lambda arr: pa.compute.multiply(arr, 2))
result = selector.execute(my_struct_array)
__init__
def __init__(query: str) -> None
Parse a selector from a query string.
| PARAMETER | DESCRIPTION |
|---|---|
query
|
The selector query string (e.g. ".field", ".foo | .bar").
TYPE:
|
execute
Execute this selector against a pyarrow array.
| PARAMETER | DESCRIPTION |
|---|---|
source
|
The input Arrow array to query.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
The result array, or None if the selector's error was suppressed.
|
|
execute_per_row
Execute this selector against each row of a pyarrow list array.
The output is guaranteed to have the same number of rows as the input.
| PARAMETER | DESCRIPTION |
|---|---|
source
|
The input Arrow list array to query.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
The result list array, or None if the selector's error was suppressed.
|
|
pipe
Pipe the output of this selector through a transformation function or another selector.
Returns a new selector; the original is not modified.
| PARAMETER | DESCRIPTION |
|---|---|
func
|
A callable that accepts a |
| RETURNS | DESCRIPTION |
|---|---|
A new [`Selector`][rerun.experimental.Selector] with the transformation applied.
|
|
StoreEntry
StreamingReader
Bases: Protocol
Protocol for readers that produce a sequential stream of chunks.
All readers provide stream() -> LazyChunkStream. Readers for indexable
formats will additionally satisfy
IndexedReader, which adds
store() -> LazyStore and load() -> ChunkStore.
ViewerClient
A connection to an instance of a Rerun viewer.
Use the connect classmethod
to attach to an already-running viewer, or
spawn to start a fresh one
(e.g. in headless mode for CI screenshots).
Spawned-viewer teardown:
- Explicit
closealways terminates the spawned viewer. - For an attached viewer (
detach_process=False), exiting awithblock or garbage-collecting the client also terminates the viewer. - A detached viewer keeps running through
withexits and garbage collection. Only an explicitclose()shuts it down.
Warning
This API is experimental and may change or be removed in future versions.
__init__
def __init__(
url: str = _DEFAULT_URL,
*,
_pid: int | None = None,
_kill_on_exit: bool = False,
) -> None
Low-level constructor.
Prefer
ViewerClient.connect or
ViewerClient.spawn.
| PARAMETER | DESCRIPTION |
|---|---|
url
|
The URL to connect to. The scheme must be one of
TYPE:
|
_pid
|
Internal — set by
TYPE:
|
_kill_on_exit
|
Internal — set by
TYPE:
|
close
def close() -> None
Close the client, terminating the spawned viewer.
Emits a UserWarning and is a no-op if there is no spawned viewer to
terminate (either the client never spawned one, or it has already
been closed). Safe to call multiple times — only the first call has
an effect.
connect
classmethod
def connect(url: str = _DEFAULT_URL) -> ViewerClient
Connect to an already-running viewer.
| PARAMETER | DESCRIPTION |
|---|---|
url
|
The URL to connect to. The scheme must be one of
TYPE:
|
save_screenshot
Save a screenshot to a file.
Warning
This API is experimental and may change or be removed in future versions.
| PARAMETER | DESCRIPTION |
|---|---|
file_path
|
The path where the screenshot will be saved. Important This path is relative to the viewer's filesystem, not the client's. If your viewer runs on a different machine, the screenshot will be saved there.
TYPE:
|
view_id
|
Optional view ID to screenshot. If None, screenshots the entire viewer. |
send_table
def send_table(
name: str,
table: RecordBatch | list[RecordBatch] | DataFrame,
) -> None
Send a table to the viewer.
A table is represented as a dataframe defined by an Arrow record batch.
| PARAMETER | DESCRIPTION |
|---|---|
name
|
The table name. Note The table name serves as an identifier. If you send a table with the same name twice, the second table will replace the first one.
TYPE:
|
table
|
The Arrow RecordBatch containing the table data to send.
TYPE:
|
spawn
classmethod
def spawn(
*,
headless: bool = False,
port: int = 9876,
memory_limit: str = "75%",
server_memory_limit: str = "1GiB",
hide_welcome_screen: bool = False,
detach_process: bool | None = None,
executable_name: str = "rerun",
executable_path: str | None = None,
) -> ViewerClient
Spawn a fresh viewer process and connect to it.
| PARAMETER | DESCRIPTION |
|---|---|
headless
|
Run the spawned viewer in headless mode (no OS window).
The viewer still listens for gRPC connections, so the SDK can keep
logging data and request screenshots via
A working graphics stack must be present — either a real GPU/driver or a
software rasterizer like Mesa's
TYPE:
|
port
|
The port to listen on.
TYPE:
|
memory_limit
|
An upper limit on how much memory the Rerun Viewer should use.
When this limit is reached, Rerun will drop the oldest data.
Example:
TYPE:
|
server_memory_limit
|
An upper limit on how much memory the gRPC server running
in the same process as the Rerun Viewer should use.
When this limit is reached, Rerun will drop the oldest data.
Example: Defaults to
TYPE:
|
hide_welcome_screen
|
Hide the normal Rerun welcome screen.
TYPE:
|
detach_process
|
Detach the spawned viewer from this Python process. A detached viewer survives unexpected parent termination
(e.g. crashes or terminal hang-up), Defaults to
TYPE:
|
executable_name
|
Specifies the name of the Rerun executable.
You can omit the Defaults to
TYPE:
|
executable_path
|
Enforce a specific executable to use instead of searching
through PATH for Unspecified by default.
TYPE:
|
query_metrics
def query_metrics() -> Iterator[MetricsCollector]
Capture DataFusion query metrics for every query that runs inside the with block.
Yields a MetricsCollector; read .last_query() or
.queries mid-scope or after the scope exits.
The scope is bound to the current contextvars.Context: every
re_datafusion query built from dataset.reader(…) while this scope
is open contributes a QueryMetrics record. Nested query_metrics()
scopes each see queries built inside them. Queries built in another
thread or asyncio task that did not inherit this context (e.g. a
raw threading.Thread rather than one started via
contextvars.copy_context()) are not captured.
The collectors are bound to a query at reader() time, so a df built
inside the with block whose .collect() runs after __exit__ still
flows to the collector; a df built outside but executed inside does
not.
Examples:
import rerun as rr
from rerun.experimental import query_metrics
client = rr.catalog.CatalogClient("rerun://…")
dataset = client.get_dataset(name="…")
with query_metrics() as m:
df = dataset.reader(index="time_1").limit(100)
df.collect()
print(m.last_query())
send_chunks
def send_chunks(
chunks: Chunk
| LazyChunkStream
| LazyStore
| ChunkStore
| Iterable[Chunk],
*,
recording: RecordingStream | None = None,
) -> None
Send chunks to a recording stream. Blocks until every chunk has been queued.
Note
For a LazyChunkStream and LazyStore inputs, this call triggers execution
and/or loading and will block for the duration of this process.
| PARAMETER | DESCRIPTION |
|---|---|
chunks
|
One of:
Source store identity (
TYPE:
|
recording
|
Recording stream to send into. Defaults to the current active recording.
TYPE:
|