Utilities

DataFusion utilities.

def partition_url(dataset, *, partition_id_col=None, timestamp_col=None, timeline_name=None)

Compute the URL for a partition within a dataset.

This is a Rerun focused DataFusion function that will create a DataFusion expression for the partition URL.

To manually invoke the underlying UDF, see partition_url_udf or partition_url_with_timeref_udf.

PARAMETER	DESCRIPTION
`dataset`	The input Rerun Dataset. TYPE: `DatasetEntry`
`partition_id_col`	The column containing the partition ID. If not provided, it will assume a default value of `rerun_partition_id`. You may pass either a DataFusion expression or a string column name. TYPE: `str \| Expr \| None` DEFAULT: `None`
`timestamp_col`	If this parameter is passed in, generate a URL that will jump to a specific timestamp within the partition. TYPE: `str \| Expr \| None` DEFAULT: `None`
`timeline_name`	When used in combination with `timestamp_col`, this specifies which timeline to seek along. By default this will use the same string as timestamp_col. TYPE: `str \| None` DEFAULT: `None`

def partition_url_udf(dataset)

Create a UDF to the URL for a partition within a Dataset.

This function will generate a UDF that expects one column of input, a string containing the Partition ID.

def partition_url_with_timeref_udf(dataset, timeline_name)

Create a UDF to the URL for a partition within a Dataset with timestamp.

This function will generate a UDF that expects two columns of input, a string containing the Partition ID and the timestamp in nanoseconds.