Experimental
rerun.experimental
Experimental features for Rerun.
These features are not yet stable and may change in future releases without going through the normal deprecation cycle.
The stable chunk API and readers (RrdReader, McapReader, …) now live in rerun.chunk.
The old rerun.experimental names still work for one release but warn on use.
StoreId
module-attribute
StoreId = str
Identifies one recording open in the viewer, as {kind}:{application_id}:{recording_id}.
kind is Recording or Blueprint. The application id is the application that logged it, or
the dataset id for a catalog-backed recording; the recording id is the recording itself, or its
segment id. Pass the whole string back to the methods that take a recording.
Both ids may contain a colon, so the application id's are escaped as \: (and a backslash as
\\): the kind runs to the first colon, the application id to the next unescaped one, and the
recording id is the rest.
DatasetInfo
dataclass
Hdf5Reader
Read chunks from an HDF5 file.
The reader is a lightweight handle over the file: inspect the raw structure
with groups(), datasets(), and attributes(), and produce chunks with
stream(...). All loading options live on stream(), so one reader can drive
several differently-configured streams over the same file.
Each HDF5 group is mapped to a Rerun entity, and the group's leaf datasets
become the columns of that entity. The file root maps to the entity /, and
nested groups map to nested entity paths (/observations/images becomes the
entity /observations/images). See stream() for how datasets, timelines, and
attributes are turned into chunks.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the |
| RAISES | DESCRIPTION |
|---|---|
FileNotFoundError
|
If |
attributes
def attributes(
path: str = "/",
) -> dict[
str, int | float | str | bytes | list[int | float | str]
]
Read the HDF5 attributes attached to an object as a typed Python dict.
This is a convenience accessor for the same attributes that stream()
emits under __hdf5_properties (see the class docstring). It reads the
raw file directly.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the object whose attributes are read. Defaults to the root
group
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
A mapping from attribute name to value. Scalar attributes are returned
|
|
as Python scalars (`int`, `float`, `str`, `bytes`); array-valued
|
|
attributes are returned as lists. Empty if the object has no attributes.
|
|
| RAISES | DESCRIPTION |
|---|---|
KeyError
|
If |
datasets
def datasets(path: str = '/') -> list[DatasetInfo]
List the datasets under path, recursively, with their shape and dtype.
Metadata only — no dataset values are read; reflects the raw file.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Group under which to list. Defaults to the root group
TYPE:
|
groups
List the group paths under path, recursively.
Metadata only — no dataset values are read; reflects the raw file.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Group under which to list. Defaults to the root group
TYPE:
|
stream
def stream(
*,
root_group: str | None = None,
entity_path_prefix: str | None = None,
index_column: IndexColumn | None = None,
ignore_datasets: list[str] | None = None,
use_structs: bool = True,
) -> LazyChunkStream
Return a lazy stream over all chunks in the HDF5 file.
Each call is independent: the same reader can be streamed several times with different configurations.
Datasets are loaded according to their dimensionality:
- A 0-D (scalar) dataset is loaded as static data — a single value with no timeline.
- A 1-D dataset
[N]becomes a column ofNscalar rows. - A 2-D dataset
[N, K]becomes a column ofNrows, each a fixed-size list ofKelements. - A 3-D-or-higher dataset
[N, d1, …, dk]becomes a column ofNrows, each a single blob of the matching type (an ArrowList<PRIMITIVE_TYPE>) holding the row's raw row-major values. The original per-row shape is not recorded in the emitted data; recover it viadatasets.
For 1-D and higher-dimensional datasets the leading dimension is always the row axis. Element types are mapped to their natural Arrow equivalents (signed and unsigned integers, floats, and strings); no semantic interpretation is applied.
HDF5 attributes are emitted as static chunks under a dedicated
__hdf5_properties entity, mirroring the source layout: root attributes land
on __hdf5_properties, and attributes on object /a/b on
__hdf5_properties/a/b. Each attribute becomes one static component named
after it, typed with the same mapping as datasets. This keeps the general
__properties entity free for user-defined property layers.
Row alignment
Every loaded, non-ignored, non-scalar dataset is aligned positionally to the file-wide timeline and must therefore share the same number of rows (scalar datasets are static and exempt):
- With an
index_column, that shared count is the index dataset's length. - Without one, the datasets must all agree on a single row count, which
becomes the length of the generated
row_indextimeline.
A dataset that violates this raises unless it is listed in ignore_datasets;
nothing is dropped automatically to satisfy alignment.
| PARAMETER | DESCRIPTION |
|---|---|
root_group
|
The group to treat as the file root (default: the whole file). Only its
subtree is loaded and aligned, its own attributes act as the root
attributes, and every other path — Attributes on groups above
TYPE:
|
entity_path_prefix
|
Optional prefix prepended to every entity path (for example
TYPE:
|
index_column
|
Dataset to use as the file-wide timeline index, built with
The referenced dataset must be 1-dimensional. When omitted, a single
TYPE:
|
ignore_datasets
|
Datasets or groups to exclude entirely. Each entry is a dataset path or
a group path (which excludes the whole subtree), interpreted relative to
|
use_structs
|
When
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If a loaded, non-ignored, non-scalar dataset cannot be aligned to the
applicable row count (the index length, or the file's shared row count
when no Also raised when |
LeRobotReader
Read chunks from a LeRobot dataset, one episode at a time.
The reader is a lightweight handle over the dataset directory: constructing it
validates that the path is a v2 or v3 LeRobot dataset and loads its metadata.
Enumerate episodes with episodes() and produce chunks with stream(episode, ...).
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the LeRobot dataset directory (the one containing |
| RAISES | DESCRIPTION |
|---|---|
FileNotFoundError
|
If |
ValueError
|
If |
stream
def stream(
episode: int,
*,
entity_path_prefix: str | None = None,
timeline: str | None = None,
video_mode: Literal["native", "skip"] = "native",
) -> LazyChunkStream
Return a lazy stream over one episode's chunks.
Most video streams directly; a stream that must be re-encoded — H.264 with
B-frames, or an episode window starting mid-GOP — needs ffmpeg on the system
PATH.
Each call is independent: the same reader can stream several episodes (or the same episode several times) with different configurations. The typical loop is:
reader = LeRobotReader("path/to/dataset")
for episode in reader.episodes():
reader.stream(episode).write_rrd(
f"episode_{episode}.rrd",
application_id="my_dataset",
recording_id=f"episode_{episode}",
)
| PARAMETER | DESCRIPTION |
|---|---|
episode
|
The episode index to stream; must be one of
TYPE:
|
entity_path_prefix
|
Prepended to every feature's entity path.
TYPE:
|
timeline
|
Overrides the derived timeline name (
TYPE:
|
video_mode
|
TYPE:
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If |
LogEntry
dataclass
One message the viewer logged.
MetricsCollector
Accumulator yielded by query_metrics on __enter__.
Use last_query() / queries to read snapshots accumulated so far; both
are non-destructive. On context-manager exit any remaining snapshots are
drained into this collector and the scope is unbound from the
ContextVar, so the collector is still readable after the scope ends.
queries
property
queries: list[QueryMetrics]
Non-destructive snapshot of all queries captured so far.
clear
def clear() -> None
Drop all captured snapshots from both the Rust buffer and this collector.
last_query
def last_query() -> QueryMetrics | None
Most recently captured query, or None if none yet.
Mp4Reader
Read chunks from an MP4 file.
__init__
def __init__(
path: str | Path,
*,
mode: Literal["asset", "stream"] = "stream",
chunk_by_gop: bool = True,
timeline_name: str = "video",
timeline_type: TemporalTimelineType = "duration_ns",
transcode: Mp4TranscodeOptions | None = None,
entity_path: str | None = None,
) -> None
Construct a new MP4 reader.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the |
mode
|
How to convert the mp4 into chunks.
TYPE:
|
chunk_by_gop
|
Only meaningful when Passing
TYPE:
|
timeline_name
|
Name of the timeline used for stream-mode samples and for the
TYPE:
|
timeline_type
|
How to interpret the timeline values.
The emitted values are the mp4 PTS (nanoseconds since the start of the video) only the declared Arrow type changes:
TYPE:
|
transcode
|
Only meaningful when
TYPE:
|
entity_path
|
Entity path under which chunks are emitted. When
TYPE:
|
Mp4TranscodeOptions
dataclass
How to transcode an mp4.
ffmpeg_override
class-attribute
instance-attribute
Override the ffmpeg executable used to transcode. When None (default),
ffmpeg is looked up on PATH. Ignored when no transcode happens.
gop_size
class-attribute
instance-attribute
gop_size: int | None = None
Force a keyframe every gop_size frames in the transcoded output. Requesting it
triggers a re-encode. None (default) keeps the encoder's default GOP.
output_codec
class-attribute
instance-attribute
output_codec: VideoCodec | None = None
Re-encode to this VideoCodec instead of keeping
the source codec; the emitted VideoStream codec follows it. None (default)
keeps the source codec.
try_gpu
class-attribute
instance-attribute
try_gpu: bool = False
Try to use a hardware (GPU) encoder if the local FFmpeg provides one for the output codec, otherwise fall back to software (best-effort). GPU encoding is drawn from the NVENC and VideoToolbox families only, so it realistically applies to H264/H265 and, on newer NVIDIA hardware, AV1. VP8/VP9 always fall back to software — their only GPU encoders are Intel QSV/VAAPI, which are not yet used. Has no effect unless a transcode is already happening.
ParquetReader
Read chunks from a Parquet file.
The reader turns raw parquet columns into grouped, time-indexed
Chunks of struct/scalar components. To map those
struct fields into Rerun archetypes (translation, rotation, scalars, …), apply
lenses to the resulting .stream() — see
DeriveLens:
Example
from rerun.chunk import DeriveLens, IndexColumn
from rerun.experimental import ParquetReader
store = (
ParquetReader(path)
.stream(index_columns=[IndexColumn.sequence("frame_index")])
.lenses(
[
DeriveLens("data", output_entity="/pose")
.to_translation("pos_x", "pos_y", "pos_z")
.to_quaternion("quat_x", "quat_y", "quat_z", "quat_w")
],
content="/transform",
)
.collect()
)
| PARAMETER | DESCRIPTION |
|---|---|
path
|
Path to the |
| RAISES | DESCRIPTION |
|---|---|
FileNotFoundError
|
If |
stream
def stream(
*,
entity_path_prefix: str | None = None,
column_grouping: str = "prefix",
delimiter: str = "_",
prefixes: list[str] | None = None,
use_structs: bool = True,
static_columns: list[str] | None = None,
index_columns: list[IndexColumn] | None = None,
) -> LazyChunkStream
Return a lazy stream over all chunks in the Parquet file.
| PARAMETER | DESCRIPTION |
|---|---|
entity_path_prefix
|
Optional prefix for all entity paths (e.g.
TYPE:
|
column_grouping
|
How to group columns into chunks.
TYPE:
|
delimiter
|
Character used to split column names when
TYPE:
|
prefixes
|
Explicit prefix strings for grouping columns. Required when
|
use_structs
|
When
TYPE:
|
static_columns
|
Column names whose values are constant across all rows. These are emitted once as timeless/static data. An error is raised if a listed column contains varying values. |
index_columns
|
Columns to use as timeline indices, each built with
When omitted, a synthetic
TYPE:
|
QueryMetrics
dataclass
One query's metrics, captured at the moment its last per-partition stream finished.
Mirrors the Rust-side re_datafusion::QuerySnapshot. The same numbers are
produced via three transports: this dataclass (Python), DataFusion's
EXPLAIN ANALYZE, and the PostHog analytics OTLP span. Field naming
differs across the three:
- Timing fields here are
datetime.timedelta(total_duration,time_to_first_chunk, …).EXPLAIN ANALYZEuses DataFusionTimemetrics, which print their own units. The OTLP analytics attributes keep an explicit_ussuffix and carry integer microseconds (total_duration_us,time_to_first_chunk_us, …) because OTLP attribute values are scalar (i64/f64/bool/string) and can't carry a duration natively. query_chunks_per_segment_meanis afloatand does not appear inEXPLAIN ANALYZE, since DataFusionCountmetrics are integer-only. The corresponding_min/_maxinteger fields are surfaced in all three transports.
fetch_direct_max_attempt is the true maximum attempt number across all
partitions.
fetch_requests
property
fetch_requests: int
Total fetch requests across both gRPC and direct transports.
waste_ratio
property
waste_ratio: float | None
Bytes fetched per byte delivered — the over-fetch signal.
A large value means the query pulled far more data than it handed back
(e.g. "fetched 4 GB, delivered 200 rows"). None when nothing was
delivered, so the ratio is undefined.
Recording
dataclass
Timeline
dataclass
One timeline of a recording, with the range of times it holds.
start
instance-attribute
start: int | None
First time on the timeline, or None if it holds no data yet.
View
dataclass
One view of the viewer's current blueprint.
reports
instance-attribute
reports: list[ViewReport]
What failed to visualize. Empty when the view is healthy.
ViewReport
dataclass
A warning or an error a view reported the last time it was shown.
ViewerClient
A connection to an instance of a Rerun viewer.
Use the connect classmethod
to attach to an already-running viewer, or
spawn to start a fresh one
(e.g. in headless mode for CI screenshots).
Spawned-viewer teardown:
- Explicit
closealways terminates the spawned viewer. - For an attached viewer (
detach_process=False), exiting awithblock or garbage-collecting the client also terminates the viewer. - A detached viewer keeps running through
withexits and garbage collection. Only an explicitclose()shuts it down.
Warning
This API is experimental and may change or be removed in future versions.
__init__
def __init__(
url: str = _DEFAULT_URL,
*,
_pid: int | None = None,
_kill_on_exit: bool = False,
) -> None
Low-level constructor.
Prefer
ViewerClient.connect or
ViewerClient.spawn.
| PARAMETER | DESCRIPTION |
|---|---|
url
|
The URL to connect to. The scheme must be one of
TYPE:
|
_pid
|
Internal — set by
TYPE:
|
_kill_on_exit
|
Internal — set by
TYPE:
|
close
def close() -> None
Close the client, terminating the spawned viewer.
Emits a UserWarning and is a no-op if there is no spawned viewer to
terminate (either the client never spawned one, or it has already
been closed). Safe to call multiple times — only the first call has
an effect.
close_recordings
Close recordings in the viewer, and return what was closed.
This only removes them from the viewer. Files on disk are untouched, and registered recordings stay in the catalog and can be reopened, but unsaved blueprint edits are lost.
Warning
This API is experimental and may change or be removed in future versions.
| PARAMETER | DESCRIPTION |
|---|---|
target
|
|
connect
classmethod
def connect(url: str = _DEFAULT_URL) -> ViewerClient
Connect to an already-running viewer.
| PARAMETER | DESCRIPTION |
|---|---|
url
|
The URL to connect to. The scheme must be one of
TYPE:
|
open_url
def open_url(url: str) -> None
Open a URL in the viewer.
Warning
This API is experimental and may change or be removed in future versions.
| PARAMETER | DESCRIPTION |
|---|---|
url
|
A recording or blueprint file, a
TYPE:
|
save_screenshot
Save a screenshot to a file.
Warning
This API is experimental and may change or be removed in future versions.
| PARAMETER | DESCRIPTION |
|---|---|
file_path
|
The path where the screenshot will be saved. Important This path is relative to the viewer's filesystem, not the client's. If your viewer runs on a different machine, the screenshot will be saved there.
TYPE:
|
view_id
|
Optional view ID to screenshot. If None, screenshots the entire viewer. |
send_table
def send_table(
name: str,
table: RecordBatch | list[RecordBatch] | DataFrame,
) -> None
Send a table to the viewer.
A table is represented as a dataframe defined by an Arrow record batch.
| PARAMETER | DESCRIPTION |
|---|---|
name
|
The table name. Note The table name serves as an identifier. If you send a table with the same name twice, the second table will replace the first one.
TYPE:
|
table
|
The Arrow RecordBatch containing the table data to send.
TYPE:
|
set_time
def set_time(
timeline: str | None = None,
*,
sequence: int,
play: bool = False,
recording: StoreId | None = None,
) -> None
def set_time(
timeline: str | None = None,
*,
sequence: int | None = None,
duration: int
| float
| timedelta
| timedelta64
| None = None,
timestamp: int
| float
| datetime
| datetime64
| None = None,
play: bool = False,
recording: StoreId | None = None,
) -> None
Set the viewer's time cursor.
| PARAMETER | DESCRIPTION |
|---|---|
timeline
|
The timeline to seek on. If omitted, the viewer uses its active timeline.
TYPE:
|
sequence
|
A sequence index.
TYPE:
|
duration
|
A duration in seconds, or a duration value with nanosecond precision.
TYPE:
|
timestamp
|
Seconds since Unix epoch, or a timestamp value with nanosecond precision.
TYPE:
|
play
|
Start playing from the new position. The viewer pauses by default.
TYPE:
|
recording
|
The recording to seek, as reported by
TYPE:
|
spawn
classmethod
def spawn(
*,
headless: bool = False,
port: int = 9876,
memory_limit: str = "75%",
server_memory_limit: str = "1GiB",
hide_welcome_screen: bool = False,
detach_process: bool | None = None,
executable_name: str = "rerun",
executable_path: str | None = None,
) -> ViewerClient
Spawn a fresh viewer process and connect to it.
| PARAMETER | DESCRIPTION |
|---|---|
headless
|
Run the spawned viewer in headless mode (no OS window).
The viewer still listens for gRPC connections, so the SDK can keep
logging data and request screenshots via
A working graphics stack must be present — either a real GPU/driver or a
software rasterizer like Mesa's
TYPE:
|
port
|
The port to listen on.
TYPE:
|
memory_limit
|
An upper limit on how much memory the Rerun Viewer should use.
When this limit is reached, Rerun will drop the oldest data.
Example:
TYPE:
|
server_memory_limit
|
An upper limit on how much memory the gRPC server running
in the same process as the Rerun Viewer should use.
When this limit is reached, Rerun will drop the oldest data.
Example: Defaults to
TYPE:
|
hide_welcome_screen
|
Hide the normal Rerun welcome screen.
TYPE:
|
detach_process
|
Detach the spawned viewer from this Python process. A detached viewer survives unexpected parent termination
(e.g. crashes or terminal hang-up), Defaults to
TYPE:
|
executable_name
|
Specifies the name of the Rerun executable.
You can omit the Defaults to
TYPE:
|
executable_path
|
Enforce a specific executable to use instead of searching
through PATH for Unspecified by default.
TYPE:
|
viewer_logs
Return the viewer's recent log messages, oldest first.
The viewer keeps a bounded buffer, so old entries drop out.
Warning
This API is experimental and may change or be removed in future versions.
| PARAMETER | DESCRIPTION |
|---|---|
after_sequence
|
Only return entries newer than this sequence number. Pass the last one you saw to fetch only what is new. None returns everything buffered.
TYPE:
|
viewer_state
def viewer_state() -> ViewerState
Report what the viewer is currently showing.
Call this to learn which recording and timeline to drive, and which time values are valid, before moving the time cursor. A view's reports say what failed to visualize.
Warning
This API is experimental and may change or be removed in future versions.
ViewerState
dataclass
A snapshot of what the viewer is currently showing.
catalog_url
instance-attribute
catalog_url: str | None
Origin of the catalog server the viewer hosts.
Hand this to CatalogClient to read the data behind the open
recordings; this API drives the viewer and deliberately does not serve data itself.
url
instance-attribute
url: str
The current page, as a sharable URL. Empty for a page that has none.
query_metrics
def query_metrics() -> Iterator[MetricsCollector]
Capture DataFusion query metrics for every query that runs inside the with block.
Yields a MetricsCollector; read .last_query() or
.queries mid-scope or after the scope exits.
The scope is bound to the current contextvars.Context: every
re_datafusion query built from dataset.reader(…) while this scope
is open contributes a QueryMetrics record. Nested query_metrics()
scopes each see queries built inside them. Queries built in another
thread or asyncio task that did not inherit this context (e.g. a
raw threading.Thread rather than one started via
contextvars.copy_context()) are not captured.
The collectors are bound to a query at reader() time, so a df built
inside the with block whose .collect() runs after __exit__ still
flows to the collector; a df built outside but executed inside does
not.
Examples:
import rerun as rr
from rerun.experimental import query_metrics
client = rr.catalog.CatalogClient("rerun://…")
dataset = client.get_dataset(name="…")
with query_metrics() as m:
df = dataset.reader(index="time_1").limit(100)
df.collect()
print(m.last_query())