Skip to content

Catalog

rerun.catalog

IndexValuesLike module-attribute

IndexValuesLike: TypeAlias = (
    NDArray[int_] | NDArray[datetime64] | Int64Array
)

A type alias for index values.

This can be any numpy-compatible array of integers, or a pyarrow.Int64Array

AlreadyExistsError

Bases: Exception

Raised when trying to create a resource that already exists.

CatalogClient

Client for a remote Rerun catalog server.

Note: the datafusion package is required to use this client. Initialization will fail with an error if the package is not installed.

ctx property
ctx: SessionContext

Returns a DataFusion session context for querying the catalog.

url property
url: str

Returns the catalog URL.

__init__
def __init__(
    url: str,
    *,
    token: str | None = None,
    addr: str | None = None,
) -> None

Connect to a remote Rerun catalog server.

PARAMETER DESCRIPTION
url

The URL of the catalog server to connect to.

TYPE: str

token

An optional authentication token to use when connecting to the server.

TYPE: str | None DEFAULT: None

addr

Deprecated: Renamed to url

TYPE: str | None DEFAULT: None

benchmark
def benchmark(
    *, num_bytes: int = 16 * 1024 * 1024, num_pings: int = 5
) -> BenchmarkResult

Measure round-trip time and download bandwidth to the server.

The RTT is estimated as the minimum elapsed time across num_pings 1-byte requests (using the minimum rejects latency spikes from scheduling jitter or transient network congestion). Bandwidth is measured by downloading num_bytes of pseudo-random (incompressible) bytes, subtracting the RTT from the elapsed time, and dividing by the payload size.

PARAMETER DESCRIPTION
num_bytes

Total payload size to download from the server when measuring bandwidth.

TYPE: int DEFAULT: 16 * 1024 * 1024

num_pings

How many 1-byte requests to send when estimating RTT.

TYPE: int DEFAULT: 5

Examples:

client = rr.catalog.CatalogClient("…")
print(client.benchmark())  # BenchmarkResult(rtt=12.0 ms, bandwidth=112.0 MiB/s)
create_dataset
def create_dataset(
    name: str, *, exist_ok: bool = False
) -> DatasetEntry

Creates a new dataset with the given name.

Entry names may only contain ASCII alphanumeric characters, underscores, hyphens, dots, colons and spaces, and must be at most 180 characters long.

PARAMETER DESCRIPTION
name

The name of the dataset to create.

TYPE: str

exist_ok

If True, return the existing dataset if it already exists; otherwise, raise an error.

TYPE: bool DEFAULT: False

RAISES DESCRIPTION
AlreadyExistsError

If a dataset with the given name already exists and exist_ok is False.

create_table
def create_table(
    name: str, schema: Schema, url: str | None = None
) -> TableEntry

Create and register a new table.

PARAMETER DESCRIPTION
name

The name of the table entry to create. It must be unique within all entries in the catalog. An exception will be raised if an entry with the same name already exists.

Entry names may only contain ASCII alphanumeric characters, underscores, hyphens, dots, colons and spaces, and must be at most 180 characters long.

TYPE: str

schema

The schema of the table to create.

TYPE: Schema

url

The URL of the directory for where to store the Lance table. If provided, the table will be stored in a globally unique subdirectory. If not provided, the server will use an automatically generated URL based on its configured writable storage. On Rerun Hub, custom table URLs are currently not supported: the request will be rejected unless this parameter is None.

TYPE: str | None DEFAULT: None

dataset_names
def dataset_names(*, include_hidden: bool = False) -> list[str]

Returns a list of all dataset names in the catalog.

PARAMETER DESCRIPTION
include_hidden

If True, include blueprint datasets.

TYPE: bool DEFAULT: False

datasets
def datasets(
    *, include_hidden: bool = False
) -> list[DatasetEntry]

Returns a list of all dataset entries in the catalog.

PARAMETER DESCRIPTION
include_hidden

If True, include blueprint datasets.

TYPE: bool DEFAULT: False

do_global_maintenance
def do_global_maintenance() -> None

Perform maintenance tasks on the whole system.

entries
def entries(
    *, include_hidden: bool = False
) -> list[DatasetEntry | TableEntry]

Returns a list of all entries in the catalog.

PARAMETER DESCRIPTION
include_hidden

If True, include hidden entries (blueprint datasets and system tables like __entries).

TYPE: bool DEFAULT: False

entry_names
def entry_names(*, include_hidden: bool = False) -> list[str]

Returns a list of all entry names in the catalog.

PARAMETER DESCRIPTION
include_hidden

If True, include hidden entries (blueprint datasets and system tables like __entries).

TYPE: bool DEFAULT: False

get_dataset
def get_dataset(*, id: EntryId | str) -> DatasetEntry
def get_dataset(name: str) -> DatasetEntry
def get_dataset(
    name: str | None = None,
    *,
    id: EntryId | str | None = None,
) -> DatasetEntry

Returns a dataset by its ID or name.

Exactly one of id or name must be provided.

PARAMETER DESCRIPTION
name

The name of the dataset.

TYPE: str | None DEFAULT: None

id

The unique identifier of the dataset. Can be an EntryId object or its string representation.

TYPE: EntryId | str | None DEFAULT: None

get_table
def get_table(*, id: EntryId | str) -> TableEntry
def get_table(name: str) -> TableEntry
def get_table(
    name: str | None = None,
    *,
    id: EntryId | str | None = None,
) -> TableEntry

Returns a table by its ID or name.

Exactly one of id or name must be provided.

PARAMETER DESCRIPTION
name

The name of the table.

TYPE: str | None DEFAULT: None

id

The unique identifier of the table. Can be an EntryId object or its string representation.

TYPE: EntryId | str | None DEFAULT: None

register_table
def register_table(name: str, url: str) -> TableEntry

Registers a foreign Lance table (identified by its URL) as a new table entry with the given name.

PARAMETER DESCRIPTION
name

The name of the table entry to create. It must be unique within all entries in the catalog. An exception will be raised if an entry with the same name already exists.

Entry names may only contain ASCII alphanumeric characters, underscores, hyphens, dots, colons and spaces, and must be at most 180 characters long.

TYPE: str

url

The URL of the Lance table to register.

TYPE: str

table_names
def table_names(*, include_hidden: bool = False) -> list[str]

Returns a list of all table names in the catalog.

PARAMETER DESCRIPTION
include_hidden

If True, include system tables (e.g., __entries).

TYPE: bool DEFAULT: False

tables
def tables(*, include_hidden: bool = False) -> list[TableEntry]

Returns a list of all table entries in the catalog.

PARAMETER DESCRIPTION
include_hidden

If True, include system tables (e.g., __entries).

TYPE: bool DEFAULT: False

version_info
def version_info() -> VersionInfo

Returns version and deployment information from the server.

Returns a VersionInfo object with version, cloud_provider, and cloud_region fields. Cloud fields are None if the server is not deployed on a cloud provider.

ComponentColumnDescriptor

The descriptor of a component column.

Component columns contain the data for a specific component of an entity.

Column descriptors are used to describe the columns in a Schema. They are read-only. To select a component column, use ComponentColumnSelector.

archetype property
archetype: str | None

The archetype name, if any.

This property is read-only.

component property
component: str

The component.

This property is read-only.

component_type property
component_type: str | None

The component type, if any.

This property is read-only.

entity_path property
entity_path: str

The entity path.

This property is read-only.

is_property property
is_property: bool

Is this column a property?

is_static property
is_static: bool

Whether the column is static.

This property is read-only.

name property
name: str

The name of this column.

This property is read-only.

ComponentColumnSelector

A selector for a component column.

Component columns contain the data for a specific component of an entity.

component property
component: str

The component.

This property is read-only.

entity_path property
entity_path: str

The entity path.

This property is read-only.

__init__
def __init__(entity_path: str, component: str) -> None

Create a new ComponentColumnSelector.

PARAMETER DESCRIPTION
entity_path

The entity path to select.

TYPE: str

component

The component to select. Example: Points3D:positions.

TYPE: str

ContentFilter

Immutable builder for entity path filter expressions passed to filter_contents().

Start from everything() or nothing(), then chain include(), exclude(), and include_properties() calls to build up the filter. Each method returns a new ContentFilter, leaving the original unchanged.

Examples:

# Include everything except raw robot data, but keep one specific path:
view = dataset.filter_contents(
    ContentFilter.everything()
    .exclude("/robot/raw/**")
    .include("/robot/raw/i_need_this")
)

# Include nothing, then allow specific subtrees:
view = dataset.filter_contents(
    ContentFilter.nothing()
    .include("/points", subtree=True)
    .include("/world/camera")
)

# Path segments with special characters are auto-escaped via include_path/exclude_path:
f = ContentFilter.everything().exclude_path(["robot", "raw data", "sensor!"], subtree=True)
# equivalent to: exclude("/robot/raw\ data/sensor\!/**")
everything classmethod
def everything() -> ContentFilter

Start with all entity paths included (auto-excludes __properties, see include_properties()).

Equivalent to filter_contents("/**").

exclude
def exclude(
    path: str, *, subtree: bool = False
) -> ContentFilter

Exclude entity paths matching path.

PARAMETER DESCRIPTION
path

A pre-formed entity path expression string (e.g. "/robot/raw/**"). Must start with "/".

TYPE: str

subtree

If True, appends /** to match the path and all its descendants. Ignored if the path string already ends with /**.

TYPE: bool DEFAULT: False

include
def include(
    path: str, *, subtree: bool = False
) -> ContentFilter

Include entity paths matching path.

PARAMETER DESCRIPTION
path

A pre-formed entity path expression string (e.g. "/robot/raw/**"). Must start with "/".

TYPE: str

subtree

If True, appends /** to match the path and all its descendants. Ignored if the path string already ends with /**.

TYPE: bool DEFAULT: False

include_properties
def include_properties() -> ContentFilter

Include the __properties/** subtree.

By default __properties is auto-excluded. Calling this method suppresses that auto-exclusion for the entire subtree. See the filter_contents() docs for details on __properties handling.

nothing classmethod
def nothing() -> ContentFilter

Start with all entity paths excluded.

Equivalent to filter_contents([]).

to_exprs
def to_exprs() -> list[str]

Return the accumulated list of filter expression strings.

DatasetEntry

Bases: Entry[DatasetEntryInternal]

A dataset entry in the catalog.

catalog property
catalog: CatalogClient

The catalog client that this entry belongs to.

created_at property
created_at: datetime

The entry's creation date and time.

id property
id: EntryId

The entry's id.

kind property
kind: EntryKind

The entry's kind.

manifest_url property
manifest_url: str

Return the dataset manifest URL.

name property
name: str

The entry's name.

updated_at property
updated_at: datetime

The entry's last updated date and time.

__eq__
def __eq__(other: object) -> bool

Compare this entry to another object.

Supports comparison with str and EntryId to enable the following patterns:

"entry_name" in client.entries()
entry_id in client.entries()

arrow_schema
def arrow_schema() -> Schema

Return the Arrow schema of the data contained in the dataset.

asset_dataset
def asset_dataset() -> DatasetEntry | None

The associated asset dataset, if any.

The associated asset dataset is owned by this dataset for lifecycle purposes. Deleting this dataset also deletes the associated asset dataset and its storage.

assets
def assets() -> list[str]

Lists all assets currently registered with this dataset.

blueprint_dataset
def blueprint_dataset() -> DatasetEntry | None

The associated blueprint dataset, if any.

The associated blueprint dataset is owned by this dataset for lifecycle purposes. Deleting this dataset also deletes the associated blueprint dataset and its storage.

blueprints
def blueprints() -> list[str]

Lists all blueprints currently registered with this dataset.

default_blueprint
def default_blueprint() -> str | None

Return the name currently set blueprint.

default_segment_table_blueprint
def default_segment_table_blueprint() -> str | None

Return the name of the currently set segment table blueprint.

delete
def delete() -> None

Delete this entry from the catalog.

do_maintenance
def do_maintenance(
    optimize_indexes: bool = False,
    retrain_indexes: bool = False,
    compact_fragments: bool = False,
    cleanup_before: datetime | None = None,
    unsafe_allow_recent_cleanup: bool = False,
) -> None

Perform maintenance tasks on the datasets.

filter_contents
def filter_contents(
    exprs: ContentFilter | str | Sequence[str],
) -> DatasetView

Return a new DatasetView filtered to the given entity paths.

Entity path expressions support wildcards: - "/points/**" matches all entities under /points - "-/text/**" excludes all entities under /text

PARAMETER DESCRIPTION
exprs

A ContentFilter built with the fluent builder API, an entity path expression or list of entity path expressions. Passing [] results in filtering out all contents.

TYPE: ContentFilter | str | Sequence[str]

RETURNS DESCRIPTION
DatasetView

A new view filtered to the matching entity paths.

Examples:

# Filter to a single entity path
view = dataset.filter_contents("/points/**")
view = dataset.filter_contents(
    ContentFilter.nothing()
    .include("/points/**")
)

# Filter to specific entity paths
view = dataset.filter_contents(["/points/**"])

# Exclude certain paths
view = dataset.filter_contents(["/points/**", "-/text/**"])
view = dataset.filter_contents(
    ContentFilter.nothing()
    .include("/points", subtree=True)
    .exclude("/text/**")
)

# Chain with segment filters
view = dataset.filter_segments(["recording_0"]).filter_contents("/points/**")
filter_segments
def filter_segments(
    segment_ids: str | Sequence[str] | DataFrame,
) -> DatasetView

Return a new DatasetView filtered to the given segment IDs.

PARAMETER DESCRIPTION
segment_ids

A segment ID string, a list of segment ID strings, or a DataFusion DataFrame with a column named 'rerun_segment_id'. When passing a DataFrame, if there are additional columns, they will be ignored.

TYPE: str | Sequence[str] | DataFrame

RETURNS DESCRIPTION
DatasetView

A new view filtered to the given segments.

Examples:

# Filter to a single segment
view = dataset.filter_segments("recording_0")

# Filter to specific segments
view = dataset.filter_segments(["recording_0", "recording_1"])

# Filter using a DataFrame
good_segments = segment_table.filter(col("success"))
view = dataset.filter_segments(good_segments)

# Read data from the filtered view
df = view.reader(index="timeline")
get_index_ranges
def get_index_ranges() -> DataFrame

Returns the range bounds of all indexes per segment.

manifest deprecated
def manifest(
    include_diagnostic_data: bool = False,
) -> DataFrame
Deprecated

DatasetEntry.manifest() is deprecated and will be removed in a future release. It was intended for internal and debugging use only.

Return the dataset manifest as a DataFusion DataFrame.

PARAMETER DESCRIPTION
include_diagnostic_data

Include diagnostic data in the manifest. That may include rows that correspond to layers which failed registration, were deleted, or are in pending states.

Note

Diagnostic data is subject to change in any release and should not be relied on for production.

TYPE: bool DEFAULT: False

reader
def reader(
    index: str | None,
    *,
    include_semantically_empty_columns: bool = False,
    include_tombstone_columns: bool = False,
    fill_latest_at: bool = False,
    using_index_values: dict[str, IndexValuesLike]
    | DataFrame
    | IndexValuesLike
    | None = None,
) -> DataFrame

Create a reader over this dataset.

Returns a DataFusion DataFrame.

Server side filters

The returned DataFrame supports server side filtering for both rerun_segment_id and the index (timeline) column, which can greatly improve performance. For example, the following filters will effectively be handled by the Rerun server.

dataset.reader(index="real_time").filter(col("rerun_segment_id") == "aabbccddee")
dataset.reader(index="real_time").filter(col("real_time") == "1234567890")
dataset.reader(index="real_time").filter(
    (col("rerun_segment_id") == "aabbccddee") & (col("real_time") == "1234567890")
)
PARAMETER DESCRIPTION
index

The index (timeline) to use for the view. Pass None to read only static data.

TYPE: str | None

include_semantically_empty_columns

Whether to include columns that are semantically empty.

TYPE: bool DEFAULT: False

include_tombstone_columns

Whether to include tombstone columns.

TYPE: bool DEFAULT: False

fill_latest_at

Whether to fill null values with the latest valid data.

TYPE: bool DEFAULT: False

using_index_values

Index values at which to resample data.

When specified, this argument changes the way rows are returned. Instead of returning the rows that exist in the data, one row is returned per (segment, index_value) pair you provide. If the segment has no row at that index value, nulls are returned — or the latest prior value if fill_latest_at=True (which is typically what you want for resampling).

Don't use this argument for plain index slicing — use a DataFusion filter on the index column instead. For example:

from datafusion import col, lit

# All rows in a time window.
ds.reader(index="real_time").filter(
    (col("real_time") >= lit(t0)) & (col("real_time") <= lit(t1))
)

This argument accepts the following shapes: - plain array: values are applied only to segments whose index range covers them (segments outside the range are excluded). - dict: keys are segment IDs, values are per-segment index values to sample at. - DataFrame: must have rerun_segment_id and index columns; treated as a per-segment value list.

Note

The plain array form requires a scan of the segment table to map values to the segments whose index range covers them. On datasets with many segments this can be expensive. Prefer the dict or DataFrame form when the per-segment values are already known on the client side.

Note

Unknown segment IDs are silently ignored — they contribute no rows to the result. Validate client-side if you need to catch unknown segment IDs.

TYPE: dict[str, IndexValuesLike] | DataFrame | IndexValuesLike | None DEFAULT: None

RETURNS DESCRIPTION
DataFrame

A DataFusion DataFrame.

register
def register(
    recording_uri: list[str],
    *,
    layer_name: str | Sequence[str] = "base",
    on_duplicate: OnDuplicateSegmentLayer = ERROR,
) -> RegistrationHandle

Register RRD URIs to the dataset and return a handle to track progress.

This method initiates the registration of recordings to the dataset, and returns a handle that can be used to wait for completion or iterate over results.

Prefer batching many URIs into a single register call rather than calling register repeatedly in a loop, which is much slower.

PARAMETER DESCRIPTION
recording_uri

The URIs of the RRDs to register, as a sequence of strings.

TYPE: list[str]

layer_name

The layer(s) to which the recordings will be registered to. Can be a single layer name (applied to all recordings) or a sequence of layer names (must match the length of recording_uri). Defaults to "base".

TYPE: str | Sequence[str] DEFAULT: 'base'

on_duplicate

How to handle the cases where the segment id and layer name already exist in the dataset? Defaults to OnDuplicateSegmentLayer.ERROR.

TYPE: OnDuplicateSegmentLayer DEFAULT: ERROR

RETURNS DESCRIPTION
RegistrationHandle

A handle to track and wait on the registration tasks.

register_asset
def register_asset(uri: str) -> str

Register an existing .rrd visible to the server as an asset.

Asset datasets hold a small set of static blobs shared across a dataset's segments, so they are kept deliberately small. The server enforces a few limits on the .rrd you register:

  • it must contain only static data, temporal chunks are rejected,
  • each asset segment must stay under a per-segment size limit,
  • the asset dataset may only hold a limited number of segments.
PARAMETER DESCRIPTION
uri

The URI of the .rrd file to register. It must be visible to the server.

TYPE: str

RETURNS DESCRIPTION
str

The segment id of the registered asset.

register_blueprint
def register_blueprint(
    uri: str,
    set_default: bool = True,
    *,
    segment_table: bool = False,
) -> None

Register an existing .rbl visible to the server.

By default, also set this blueprint as default.

Set segment_table=True (and set_default=True) to register it as this dataset's default for the segment table blueprint.

The associated blueprint dataset is owned by this dataset for lifecycle purposes. Deleting this dataset also deletes the associated blueprint dataset and its storage.

register_prefix
def register_prefix(
    recordings_prefix: str,
    layer_name: str | None = None,
    on_duplicate: OnDuplicateSegmentLayer = ERROR,
) -> RegistrationHandle

Register all RRDs under a given prefix to the dataset and return a handle to track progress.

A prefix is a directory-like path in an object store (e.g. an S3 bucket or ABS container). All RRDs that are recursively found under the given prefix will be registered to the dataset.

This method initiates the registration of the recordings to the dataset, and returns a handle that can be used to wait for completion or iterate over results.

PARAMETER DESCRIPTION
recordings_prefix

The prefix under which to register all RRDs.

TYPE: str

layer_name

The layer to which the recordings will be registered to. If None, this defaults to "base".

TYPE: str | None DEFAULT: None

on_duplicate

How to handle the cases where the segment id and layer name already exist in the dataset? Defaults to OnDuplicateSegmentLayer.ERROR.

TYPE: OnDuplicateSegmentLayer DEFAULT: ERROR

RETURNS DESCRIPTION
A handle to track and wait on the registration tasks.
schema
def schema() -> Schema

Return the schema of the data contained in the dataset.

segment_ids
def segment_ids() -> list[str]

Returns a list of segment IDs for the dataset.

segment_store
def segment_store(segment_id: str) -> LazyStore

Open a remote segment as a LazyStore.

The manifest is fetched immediately; chunk data is loaded on demand via LazyStore.stream. To fully materialize into a ChunkStore, call lazy.stream().collect().

segment_table
def segment_table(
    join_meta: TableEntry | DataFrame | None = None,
    join_key: str = "rerun_segment_id",
) -> DataFrame

Return the segment table as a DataFusion DataFrame.

The segment table contains metadata about each segment in the dataset, including segment IDs, layer names, storage URLs, and size information.

PARAMETER DESCRIPTION
join_meta

Optional metadata table or DataFrame to join with the segment table. If a TableEntry is provided, it will be converted to a DataFrame using reader().

TYPE: TableEntry | DataFrame | None DEFAULT: None

join_key

The column name to use for joining, defaults to "rerun_segment_id". Both the segment table and join_meta must contain this column.

TYPE: str DEFAULT: 'rerun_segment_id'

RETURNS DESCRIPTION
DataFrame

The segment metadata table, optionally joined with join_meta.

segment_url
def segment_url(
    segment_id: str,
    timeline: str | None = None,
    start: datetime | timedelta | int | None = None,
    end: datetime | timedelta | int | None = None,
) -> str

Return the URL for the given segment.

PARAMETER DESCRIPTION
segment_id

The ID of the segment to get the URL for.

TYPE: str

timeline

The name of the timeline to display.

TYPE: str | None DEFAULT: None

start

The start selected time for the segment. Integer for ticks, datetime/nanoseconds for timestamps, or timedelta for durations.

TYPE: datetime | timedelta | int | None DEFAULT: None

end

The end selected time for the segment. Integer for ticks, datetime/nanoseconds for timestamps, or timedelta for durations. If omitted, no time range selection is emitted (only the #when cursor).

TYPE: datetime | timedelta | int | None DEFAULT: None

Examples:

With ticks
>>> start_tick, end_time = 0, 10
>>> dataset.segment_url("some_id", "log_tick", start_tick, end_time)
With timestamps
>>> start_time, end_time = datetime.now() - timedelta(seconds=4), datetime.now()
>>> dataset.segment_url("some_id", "real_time", start_time, end_time)
RETURNS DESCRIPTION
str

The URL for the given segment.

set_default_blueprint
def set_default_blueprint(blueprint_name: str | None) -> None

Set an already-registered blueprint as default for this dataset.

set_default_segment_table_blueprint
def set_default_segment_table_blueprint(
    blueprint_name: str | None,
) -> None

Set an already-registered blueprint as the default segment table blueprint for this dataset.

set_name
def set_name(name: str) -> None

Change the name of this entry.

Note: entry names must be unique within the catalog. If the new name is not unique, an error will be raised.

Entry names may only contain ASCII alphanumeric characters, underscores, hyphens, dots, colons and spaces, and must be at most 180 characters long.

PARAMETER DESCRIPTION
name

New name for the entry

TYPE: str

unregister
def unregister(
    *,
    segments_to_drop: str | Sequence[str],
    layers_to_drop: str | Sequence[str],
    force: bool = False,
) -> UnregistrationHandle

Unregisters segments and layers from the dataset.

Excluding IO errors, this will always succeed as long the target dataset exists. Corollary: unregistering data that doesn't exist is a no-op.

This method acts as a product filter: * empty segments_to_drop + empty layers_to_drop: invalid argument error * empty segments_to_drop + non-empty layers_to_drop: remove specified layers for all segments * non-empty segments_to_drop + empty layers_to_drop: remove all layers for specified segments * non-empty segments_to_drop + non-empty layers_to_drop: delete all specified layers for all specified segments

PARAMETER DESCRIPTION
segments_to_drop

The segment IDs to drop. All of them if empty. The final filter will be the outer product of this and layers_to_drop.

TYPE: str | Sequence[str]

layers_to_drop

The layer names to drop. All of them if empty. The final filter will be the outer product of this and segments_to_drop.

TYPE: str | Sequence[str]

force

If true, deletion will go through regardless of the segments/layers' current statuses. This is only useful in the very specific, catatrophic scenario where the contents of the task queue were lost and some tasks are now stuck in status=pending forever. Do not use this unless you know exactly what you're doing.

TYPE: bool DEFAULT: False

unregister_asset
def unregister_asset(segment_id: str) -> None

Unregister a previously registered asset.

Since assets are shared across all of a dataset's segments, there is no way to scope an asset to a subset of them, so removing one means unregistering it here.

Unregistering an asset that doesn't exist is a no-op.

PARAMETER DESCRIPTION
segment_id

The segment id of the asset to unregister, as returned by register_asset.

TYPE: str

update deprecated
def update(*, name: str | None = None) -> None
Deprecated

Entry.update() is deprecated. Use Entry.set_name() instead.

Update this entry's properties.

PARAMETER DESCRIPTION
name

New name for the entry

TYPE: str | None DEFAULT: None

DatasetView

A filtered view over a dataset in the catalog.

A DatasetView provides lazy filtering over a dataset's segments and entity paths. Filters are composed lazily and only applied when data is actually read.

Create a DatasetView by calling filter_segments() or filter_contents() on a DatasetEntry.

Examples:

# Filter to specific segments
view = dataset.filter_segments(["recording_0", "recording_1"])

# Filter to specific entity paths
view = dataset.filter_contents(["/points/**"])

# Chain filters
view = dataset.filter_segments(["recording_0"]).filter_contents(["/points/**"])

# Read data
df = view.reader(index="timeline")
__init__
def __init__(internal: DatasetViewInternal) -> None

Create a new DatasetView wrapper.

PARAMETER DESCRIPTION
internal

The internal Rust-side DatasetView object.

TYPE: DatasetViewInternal

__repr__
def __repr__() -> str

Return a string representation of the DatasetView.

arrow_schema
def arrow_schema() -> Schema

Return the filtered Arrow schema for this view.

RETURNS DESCRIPTION
Schema

The filtered Arrow schema.

filter_contents
def filter_contents(
    exprs: ContentFilter | str | Sequence[str],
) -> DatasetView

Return a new DatasetView filtered to the given entity paths.

Entity path expressions support wildcards: - "/points/**" matches all entities under /points - "-/text/**" excludes all entities under /text

PARAMETER DESCRIPTION
exprs

A ContentFilter built with the fluent builder API, an entity path expression or list of entity path expressions. Passing [] results in filtering out all contents.

TYPE: ContentFilter | str | Sequence[str]

RETURNS DESCRIPTION
DatasetView

A new view filtered to the matching entity paths.

Examples:

# Filter to a single entity path
view = dataset.filter_contents("/points/**")

# Filter to specific entity paths
view = dataset.filter_contents(["/points/**"])

# Exclude certain paths
view = dataset.filter_contents(["/points/**", "-/text/**"])

# Using ContentFilter builder
view = dataset.filter_contents(
    ContentFilter.everything()
    .exclude("/robot/raw/**")
    .include("/robot/raw/i_need_this")
)

# Chain with segment filters
view = dataset.filter_segments(["recording_0"]).filter_contents("/points/**")
filter_segments
def filter_segments(
    segment_ids: str | Sequence[str] | DataFrame,
) -> DatasetView

Return a new DatasetView filtered to the given segment IDs.

PARAMETER DESCRIPTION
segment_ids

A segment ID string, a list of segment ID strings, or a DataFusion DataFrame with a column named 'rerun_segment_id'. When passing a DataFrame, if there are additional columns, they will be ignored.

TYPE: str | Sequence[str] | DataFrame

RETURNS DESCRIPTION
DatasetView

A new view filtered to the given segments.

Examples:

# Filter to a single segment
view = dataset.filter_segments("recording_0")

# Filter to specific segments
view = dataset.filter_segments(["recording_0", "recording_1"])

# Filter using a DataFrame
good_segments = segment_table.filter(col("success"))
view = dataset.filter_segments(good_segments)

# Read data from the filtered view
df = view.reader(index="timeline")
get_index_ranges
def get_index_ranges() -> DataFrame

Returns the range bounds of all indexes per segment.

reader
def reader(
    index: str | None,
    *,
    include_semantically_empty_columns: bool = False,
    include_tombstone_columns: bool = False,
    using_index_values: dict[str, IndexValuesLike]
    | DataFrame
    | IndexValuesLike
    | None = None,
    fill_latest_at: bool = False,
) -> DataFrame

Create a reader over this DatasetView.

Returns a DataFusion DataFrame.

Server side filters

The returned DataFrame supports server side filtering for both rerun_segment_id and the index (timeline) column, which can greatly improve performance. For example, the following filters will effectively be handled by the Rerun server.

dataset.reader(index="real_time").filter(col("rerun_segment_id") == "aabbccddee")
dataset.reader(index="real_time").filter(col("real_time") == "1234567890")
dataset.reader(index="real_time").filter(
    (col("rerun_segment_id") == "aabbccddee") & (col("real_time") == "1234567890")
)
PARAMETER DESCRIPTION
index

The index (timeline) to use for the view. Pass None to read only static data.

TYPE: str | None

include_semantically_empty_columns

Whether to include columns that are semantically empty.

TYPE: bool DEFAULT: False

include_tombstone_columns

Whether to include tombstone columns.

TYPE: bool DEFAULT: False

using_index_values

Index values at which to resample data.

When specified, this argument changes the way rows are returned. Instead of returning the rows that exist in the data, one row is returned per (segment, index_value) pair you provide. If the segment has no row at that index value, nulls are returned — or the latest prior value if fill_latest_at=True (which is typically what you want for resampling).

Don't use this argument for plain index slicing — use a DataFusion filter on the index column instead. For example:

from datafusion import col, lit

# All rows in a time window.
ds.reader(index="real_time").filter(
    (col("real_time") >= lit(t0)) & (col("real_time") <= lit(t1))
)

This argument accepts the following shapes: - plain array: values are applied only to segments whose index range covers them (segments outside the range are excluded). - dict: keys are segment IDs, values are per-segment index values to sample at. - DataFrame: must have rerun_segment_id and index columns; treated as a per-segment value list.

Note

The plain array form requires a scan of the segment table to map values to the segments whose index range covers them. On datasets with many segments this can be expensive. Prefer the dict or DataFrame form when the per-segment values are already known on the client side.

Note

Unknown segment IDs are silently ignored — they contribute no rows to the result. Validate client-side if you need to catch unknown segment IDs.

TYPE: dict[str, IndexValuesLike] | DataFrame | IndexValuesLike | None DEFAULT: None

fill_latest_at

Whether to fill null values with the latest valid data.

TYPE: bool DEFAULT: False

RETURNS DESCRIPTION
A DataFusion DataFrame.
schema
def schema() -> Schema

Return the filtered schema for this view.

The schema reflects any content filters applied to the view.

RETURNS DESCRIPTION
Schema

The filtered schema.

segment_ids
def segment_ids() -> list[str]

Return the segment IDs for this view.

If segment filters have been applied, only matching segments are returned.

RETURNS DESCRIPTION
list[str]

The list of segment IDs.

segment_table
def segment_table(
    join_meta: TableEntry | DataFrame | None = None,
    join_key: str = "rerun_segment_id",
) -> DataFrame

Return the segment table as a DataFusion DataFrame.

The segment table contains metadata about each segment in the dataset, including segment IDs, layer names, storage URLs, and size information.

Only segments matching this view's filters are included.

PARAMETER DESCRIPTION
join_meta

Optional metadata table or DataFrame to join with the segment table. If a TableEntry is provided, it will be converted to a DataFrame using reader().

TYPE: TableEntry | DataFrame | None DEFAULT: None

join_key

The column name to use for joining, defaults to "rerun_segment_id". Both the segment table and join_meta must contain this column.

TYPE: str DEFAULT: 'rerun_segment_id'

RETURNS DESCRIPTION
DataFrame

The segment metadata table, optionally joined with join_meta.

Entry

Bases: ABC, Generic[InternalEntryT]

An entry in the catalog.

catalog property
catalog: CatalogClient

The catalog client that this entry belongs to.

created_at property
created_at: datetime

The entry's creation date and time.

id property
id: EntryId

The entry's id.

kind property
kind: EntryKind

The entry's kind.

name property
name: str

The entry's name.

updated_at property
updated_at: datetime

The entry's last updated date and time.

__eq__
def __eq__(other: object) -> bool

Compare this entry to another object.

Supports comparison with str and EntryId to enable the following patterns:

"entry_name" in client.entries()
entry_id in client.entries()

delete
def delete() -> None

Delete this entry from the catalog.

set_name
def set_name(name: str) -> None

Change the name of this entry.

Note: entry names must be unique within the catalog. If the new name is not unique, an error will be raised.

Entry names may only contain ASCII alphanumeric characters, underscores, hyphens, dots, colons and spaces, and must be at most 180 characters long.

PARAMETER DESCRIPTION
name

New name for the entry

TYPE: str

update deprecated
def update(*, name: str | None = None) -> None
Deprecated

Entry.update() is deprecated. Use Entry.set_name() instead.

Update this entry's properties.

PARAMETER DESCRIPTION
name

New name for the entry

TYPE: str | None DEFAULT: None

EntryId

A unique identifier for an entry in the catalog.

__init__
def __init__(id: str) -> None

Create a new EntryId from a string.

__str__
def __str__() -> str

Return str(self).

as_bytes
def as_bytes() -> bytes

Return the raw 16-byte representation.

EntryKind

The kinds of entries that can be stored in the catalog.

__int__
def __int__() -> int

int(self)

__str__
def __str__() -> str

Return str(self).

IndexColumnDescriptor

The descriptor of an index column.

Index columns contain the index values for when the data was updated. They generally correspond to Rerun timelines.

Column descriptors are used to describe the columns in a Schema. They are read-only. To select an index column, use IndexColumnSelector.

is_static property
is_static: bool

Part of generic ColumnDescriptor interface: always False for Index.

name property
name: str

The name of the index.

This property is read-only.

IndexColumnSelector

A selector for an index column.

Index columns contain the index values for when the data was updated. They generally correspond to Rerun timelines.

name property
name: str

The name of the index.

This property is read-only.

__init__
def __init__(index: str) -> None

Create a new IndexColumnSelector.

PARAMETER DESCRIPTION
index

The name of the index to select. Usually the name of a timeline.

TYPE: str

NotFoundError

Bases: Exception

Raised when the requested resource is not found.

OnDuplicateSegmentLayer

Bases: str, Enum

How to handle duplicate segment layers when registering recordings to a dataset.

Attributes: ERROR: Raise an error if a segment layer with the same name already exists. SKIP: Skip the duplicate segment layer. REPLACE: Replace the existing segment layer with the new one.

RegistrationHandle

Handle to track and wait on segment registration tasks.

cancel
def cancel() -> None

Cancel dataset registration. If the registration is already done, this is a noop.

RAISES DESCRIPTION
ValueError

If the cancellation fails.

iter_results
def iter_results(
    timeout_secs: int | None = None,
) -> Iterator[SegmentRegistrationResult]

Stream completed registrations as they finish.

Uses the server's streaming API to yield results as tasks complete. Each result is yielded exactly once when its task completes (success or error).

PARAMETER DESCRIPTION
timeout_secs

Timeout in seconds. None for blocking. Note that using None doesn't guarantee that a TimeoutError will never be eventually raised for long-running tasks. Setting a timeout and polling is recommended for monitoring very large registration batches.

TYPE: int | None DEFAULT: None

YIELDS DESCRIPTION
SegmentRegistrationResult

The result of each completed registration.

RAISES DESCRIPTION
TimeoutError

If the timeout is reached before all tasks complete.

wait
def wait(timeout_secs: int | None = None) -> RegistrationResult

Block until all registrations complete.

PARAMETER DESCRIPTION
timeout_secs

Timeout in seconds. None for blocking. Note that using None doesn't guarantee that a TimeoutError will never be eventually raised for long-running tasks. Setting a timeout and polling is recommended for monitoring very large registration batches.

TYPE: int | None DEFAULT: None

RETURNS DESCRIPTION
RegistrationResult

The result containing the list of segment IDs in registration order.

RAISES DESCRIPTION
ValueError

If any registration fails.

TimeoutError

If the timeout is reached before all tasks complete.

RegistrationResult dataclass

Result of a completed registration batch.

segment_ids instance-attribute
segment_ids: list[str]

The ids of the registered segments.

Schema

The schema representing a set of available columns for a dataset.

A schema contains both index columns (timelines) and component columns (entity/component data).

__eq__
def __eq__(other: object) -> bool

Check equality with another Schema.

__init__
def __init__(inner: SchemaInternal) -> None

Create a new Schema wrapper.

PARAMETER DESCRIPTION
inner

The internal schema object from the bindings.

TYPE: SchemaInternal

__iter__

Iterate over all column descriptors in the schema (index columns first, then component columns).

__repr__
def __repr__() -> str

Return a string representation of the schema.

archetypes
def archetypes(
    *, include_properties: bool = False
) -> list[str]

Return a list of all the archetypes in the schema.

PARAMETER DESCRIPTION
include_properties

If True, archetypes used in properties are included.

TYPE: bool DEFAULT: False

column_for
def column_for(
    entity_path: str, component: str
) -> ComponentColumnDescriptor | None

Look up the column descriptor for a specific entity path and component.

PARAMETER DESCRIPTION
entity_path

The entity path to look up.

TYPE: str

component

The component to look up. Example: Points3D:positions.

TYPE: str

RETURNS DESCRIPTION
ComponentColumnDescriptor | None

The column descriptor, if it exists.

column_for_selector

Look up the column descriptor for a specific selector.

PARAMETER DESCRIPTION
selector

The selector to look up.

String arguments are expected to follow the format: "<entity_path>:<component_type>"

TYPE: str | ComponentColumnSelector | ComponentColumnDescriptor

RETURNS DESCRIPTION
ComponentColumnDescriptor

The column descriptor.

RAISES DESCRIPTION
LookupError

If the column is not found.

ValueError

If the string selector format is invalid or the input type is unsupported.

Note: if the input is already a `ComponentColumnDescriptor`, it is
returned directly without checking for existence.
column_names
def column_names() -> list[str]

Return a list of all column names in the schema.

RETURNS DESCRIPTION
The names of all columns (index columns first, then component columns).
column_names_for
def column_names_for(
    *,
    entity_path: str | None = None,
    archetype: str | type[Archetype] | None = None,
    component_type: str | None = None,
    include_properties: bool = False,
) -> list[str]

Return a list of column names matching the given criteria.

PARAMETER DESCRIPTION
entity_path

If set, only return columns with this entity path.

TYPE: str | None DEFAULT: None

archetype

If set, only return columns with this archetype. Accepts a fully-qualified archetype name (e.g., "rerun.archetypes.Points3D"), a short name (e.g., "Points3D"), or an Archetype class (e.g., rr.Points3D).

TYPE: str | type[Archetype] | None DEFAULT: None

component_type

If set, only return columns with this component type.

TYPE: str | None DEFAULT: None

include_properties

If True, include property columns (/__properties/*).

TYPE: bool DEFAULT: False

columns_for
def columns_for(
    *,
    entity_path: str | None = None,
    archetype: str | type[Archetype] | None = None,
    component_type: str | None = None,
    include_properties: bool = False,
) -> list[ComponentColumnDescriptor]

Return a filtered list of component columns matching the given criteria.

PARAMETER DESCRIPTION
entity_path

If set, only return columns with this entity path.

TYPE: str | None DEFAULT: None

archetype

If set, only return columns with this archetype. Accepts a fully-qualified archetype name (e.g., "rerun.archetypes.Points3D"), a short name (e.g., "Points3D"), or an Archetype class (e.g., rr.Points3D).

TYPE: str | type[Archetype] | None DEFAULT: None

component_type

If set, only return columns with this component type.

TYPE: str | None DEFAULT: None

include_properties

If True, include property columns (/__properties/*).

TYPE: bool DEFAULT: False

component_columns
def component_columns() -> Sequence[ComponentColumnDescriptor]

Return a list of all the component columns in the schema.

Component columns contain the data for a specific component of an entity.

component_types
def component_types(
    *, include_properties: bool = False
) -> list[str]

Return a list of all the component types in the schema.

PARAMETER DESCRIPTION
include_properties

If True, component types used in properties are included.

TYPE: bool DEFAULT: False

entity_paths
def entity_paths(
    *, include_properties: bool = False
) -> list[str]

Return a sorted list of all unique entity paths in the schema. By default, the properties are not included.

PARAMETER DESCRIPTION
include_properties

If True, include property entities (/__properties/*)

TYPE: bool DEFAULT: False

index_columns
def index_columns() -> Sequence[IndexColumnDescriptor]

Return a list of all the index columns in the schema.

Index columns contain the index values for when the data was updated. They generally correspond to Rerun timelines.

SegmentRegistrationResult dataclass

Result of a completed segment registration.

error instance-attribute
error: str | None

Error message if registration failed, or None if successful.

is_error property
is_error: bool

Returns True if the registration failed.

is_success property
is_success: bool

Returns True if the registration was successful.

segment_id instance-attribute
segment_id: str

The resulting segment ID.

uri instance-attribute
uri: str

The source URI that was registered.

TableEntry

Bases: Entry[TableEntryInternal]

A table entry in the catalog.

Note: this object acts as a table provider for DataFusion.

catalog property
catalog: CatalogClient

The catalog client that this entry belongs to.

created_at property
created_at: datetime

The entry's creation date and time.

id property
id: EntryId

The entry's id.

kind property
kind: EntryKind

The entry's kind.

name property
name: str

The entry's name.

storage_url property
storage_url: str

The table's storage URL.

updated_at property
updated_at: datetime

The entry's last updated date and time.

__datafusion_table_provider__
def __datafusion_table_provider__(session: Any) -> Any

Returns a DataFusion table provider capsule.

__eq__
def __eq__(other: object) -> bool

Compare this entry to another object.

Supports comparison with str and EntryId to enable the following patterns:

"entry_name" in client.entries()
entry_id in client.entries()

append
def append(
    batches: _BatchesType | None = None, **named_params: Any
) -> None

Append to the Table.

PARAMETER DESCRIPTION
batches

Arrow data to append to the table. Can be a RecordBatchReader, a single RecordBatch, a list of RecordBatches, or a list of lists of RecordBatches (as returned by datafusion.DataFrame.collect()).

TYPE: _BatchesType | None DEFAULT: None

**named_params

Each named parameter corresponds to a column in the table.

TYPE: Any DEFAULT: {}

arrow_schema
def arrow_schema() -> Schema

Returns the Arrow schema of the table.

blueprint_dataset
def blueprint_dataset() -> DatasetEntry

The associated blueprint dataset.

Tables get a blueprint dataset automatically when they are created. For tables created by older servers, this creates the missing blueprint dataset before returning.

The associated blueprint dataset is owned by this table for lifecycle purposes. Deleting this table also deletes the associated blueprint dataset and its storage.

Note

⚠️ This API is experimental and may change or be removed in future versions! ⚠️ TODO(#12746): Stabilize table blueprint APIs.

blueprints
def blueprints() -> list[str]

Lists all blueprints currently registered with this table.

Note

⚠️ This API is experimental and may change or be removed in future versions! ⚠️ TODO(#12746): Stabilize table blueprint APIs.

default_blueprint
def default_blueprint() -> str | None

Return the name currently set blueprint.

Note

⚠️ This API is experimental and may change or be removed in future versions! ⚠️ TODO(#12746): Stabilize table blueprint APIs.

delete
def delete() -> None

Delete this entry from the catalog.

overwrite
def overwrite(
    batches: _BatchesType | None = None, **named_params: Any
) -> None

Overwrite the Table with new data.

PARAMETER DESCRIPTION
batches

Arrow data to overwrite the table with. Can be a RecordBatchReader, a single RecordBatch, a list of RecordBatches, or a list of lists of RecordBatches (as returned by datafusion.DataFrame.collect()).

TYPE: _BatchesType | None DEFAULT: None

**named_params

Each named parameter corresponds to a column in the table.

TYPE: Any DEFAULT: {}

reader
def reader() -> DataFrame

Registers the table with the DataFusion context and return a DataFrame.

register_blueprint
def register_blueprint(
    uri: str, set_default: bool = True
) -> None

Register an existing .rbl visible to the server as this table's blueprint.

By default, also set this blueprint as default.

The associated blueprint dataset is owned by this table for lifecycle purposes. Deleting this table also deletes the associated blueprint dataset and its storage.

Note

⚠️ This API is experimental and may change or be removed in future versions! ⚠️ TODO(#12746): Stabilize table blueprint APIs.

set_default_blueprint
def set_default_blueprint(blueprint_name: str | None) -> None

Set an already-registered blueprint as default for this table.

Note

⚠️ This API is experimental and may change or be removed in future versions! ⚠️ TODO(#12746): Stabilize table blueprint APIs.

set_name
def set_name(name: str) -> None

Change the name of this entry.

Note: entry names must be unique within the catalog. If the new name is not unique, an error will be raised.

Entry names may only contain ASCII alphanumeric characters, underscores, hyphens, dots, colons and spaces, and must be at most 180 characters long.

PARAMETER DESCRIPTION
name

New name for the entry

TYPE: str

to_arrow_reader
def to_arrow_reader() -> RecordBatchReader

Convert this table to a pyarrow.RecordBatchReader.

update deprecated
def update(*, name: str | None = None) -> None
Deprecated

Entry.update() is deprecated. Use Entry.set_name() instead.

Update this entry's properties.

PARAMETER DESCRIPTION
name

New name for the entry

TYPE: str | None DEFAULT: None

upsert
def upsert(
    batches: _BatchesType | None = None, **named_params: Any
) -> None

Upsert data into the Table.

To use upsert, the table must contain a column with the metadata:

    {"rerun:is_table_index" = "true"}

Any row with a matching index value will have the new data inserted. Any row without a matching index value will be appended as a new row.

PARAMETER DESCRIPTION
batches

Arrow data to upsert into the table. Can be a RecordBatchReader, a single RecordBatch, a list of RecordBatches, or a list of lists of RecordBatches (as returned by datafusion.DataFrame.collect()).

TYPE: _BatchesType | None DEFAULT: None

**named_params

Each named parameter corresponds to a column in the table

TYPE: Any DEFAULT: {}

UnregistrationHandle

Handle to track and wait on segment unregistration tasks.

cancel
def cancel() -> None

Cancel unrregistration. If the unregistration is already done, this is a noop.

RAISES DESCRIPTION
ValueError

If the cancellation fails.

wait
def wait(timeout_secs: int | None = None) -> None

Block until the unregistriation completes.

PARAMETER DESCRIPTION
timeout_secs

Timeout in seconds. None for blocking. Note that using None doesn't guarantee that a TimeoutError will never be eventually raised for long-running tasks.

TYPE: int | None DEFAULT: None

RAISES DESCRIPTION
ValueError

If the uregistration fails.

TimeoutError

If the timeout is reached before all tasks complete.

VersionInfo dataclass

Version and deployment information from a Rerun server.

cloud_provider instance-attribute
cloud_provider: str | None

The cloud provider name (e.g. "aws", "azure"). None if not deployed on cloud.

cloud_region instance-attribute
cloud_region: str | None

The cloud region (e.g. "us-west-2", "eastus"). None if not deployed on cloud.

version instance-attribute
version: str

The version string of the server.