MODULE REFERENCE
dataset
anomx.data.datasetLightweight datasets for time series, generic sequences, and independent samples.
Public exports
Import these names directly from anomx.data.dataset.
DatasetClientResolve versioned datasets and load channels without a specific host.
Dataset
Dataset(reference: str | None = None, *, anomx_reference: str | None = None, data: pd.DataFrame | None = None, kind: DatasetKind = 'independent_samples', mode: DatasetMode = 'fixed', name: str = '', metadata: Mapping[str, Any] | None = None, client: DatasetClient | PlatformClient | None = None, cache: bool | str | Path = True, cache_ttl: float | None = 0, progress: bool = True)Local observations or a lazy, versioned platform dataset reference.
Dataset("data_dataset-...") resolves through the active work context or an explicit PlatformClient (falling back to ANOMX_URL/ANOMX_TOKEN). Dataset(anomx_reference="data_dataset-...", client=client) makes the remote identity explicit. HTTP endpoint details live in the data source. iter_batches bounds memory; to_dataframe deliberately materializes the whole dataset. Fixed external sources are references, never copied.
| Parameter | Type | Default |
|---|---|---|
reference | str | None | None |
anomx_referencekeyword only | str | None | None |
datakeyword only | pd.DataFrame | None | None |
kindkeyword only | DatasetKind | 'independent_samples' |
modekeyword only | DatasetMode | 'fixed' |
namekeyword only | str | '' |
metadatakeyword only | Mapping[str, Any] | None | None |
clientkeyword only | DatasetClient | PlatformClient | None | None |
cachekeyword only | bool | str | Path | True |
cache_ttlkeyword only | float | None | 0 |
progresskeyword only | bool | True |
Members
anomx_referenceproperty
anomx_reference: str | NoneThe remote dataset identity, or None for local observations.
Returns str | None
selectmethod
select(*, start: TimeBound | None = None, end: TimeBound | None = None) -> DatasetReturn a lazy time window, including start and excluding end.
Strings and datetimes are accepted; timezone-free bounds mean UTC. Explicit bounds replace those on this view. Remote selection happens on the server and requires its dataset selection capability.
| Parameter | Type | Default |
|---|---|---|
startkeyword only | TimeBound | None | None |
endkeyword only | TimeBound | None | None |
Returns Dataset
downsamplemethod
downsample(frequency: str, *, aggregation: str = 'mean') -> DatasetReturn a lazy view aggregated into fixed UTC-epoch bins.
Supported reducers are mean, first, last, min, max, sum, and count. Time filtering precedes aggregation; missing bins are omitted and no values are forward-filled. Bin labels are their left edges, which can precede a selection's start. Remote aggregation reduces transferred rows and requires a platform with dataset selection support.
| Parameter | Type | Default |
|---|---|---|
frequency | str | required |
aggregationkeyword only | str | 'mean' |
Returns Dataset
select_filesmethod
select_files(*, variables: Sequence[str] | None = None, columns: Sequence[str] | None = None, path_prefix: str | None = None, path_start: str | None = None, path_end: str | None = None, row_start: int | None = None, row_end: int | None = None, row_step: int | None = None, sample_start: int | None = None, sample_end: int | None = None, sample_step: int | None = None) -> DatasetSelect indexed files and bounded observations without materialization.
Variables are filename stems reported by the source catalog. Path bounds are lexical integration paths; row bounds are per file. Sample bounds slice the first array dimension inside each observation, preserving all remaining dimensions. End bounds are exclusive. Steps subsample by stride; they do not average, filter or invent timestamps.
| Parameter | Type | Default |
|---|---|---|
variableskeyword only | Sequence[str] | None | None |
columnskeyword only | Sequence[str] | None | None |
path_prefixkeyword only | str | None | None |
path_startkeyword only | str | None | None |
path_endkeyword only | str | None | None |
row_startkeyword only | int | None | None |
row_endkeyword only | int | None | None |
row_stepkeyword only | int | None | None |
sample_startkeyword only | int | None | None |
sample_endkeyword only | int | None | None |
sample_stepkeyword only | int | None | None |
Returns Dataset
source_entriesmethod
source_entries(*, page_size: int = 100) -> Iterator[dict[str, Any]]Iterate immutable source fingerprints without loading their contents.
| Parameter | Type | Default |
|---|---|---|
page_sizekeyword only | int | 100 |
Returns Iterator[dict[str, Any]]
describemethod
describe(*, refresh: bool = False) -> dict[str, Any]Resolve source identity and schema without loading all observations.
| Parameter | Type | Default |
|---|---|---|
refreshkeyword only | bool | False |
Returns dict[str, Any]
iter_batchesmethod
iter_batches(batch_size: int = 5000, *, progress: bool | None = None, refresh: bool = False) -> Iterator[pd.DataFrame]Yield bounded, ordered batches pinned to one resolved source version.
Each remote page is checkpointed before yielding. A new iterator replays saved pages, then resumes at the saved cursor. Close the iterator when stopping early to release the download lock. progress=False silences tqdm in applications and automated jobs.
| Parameter | Type | Default |
|---|---|---|
batch_size | int | 5000 |
progresskeyword only | bool | None | None |
refreshkeyword only | bool | False |
Returns Iterator[pd.DataFrame]
clear_cachemethod
clear_cache(*, max_rows: int | None = None) -> NoneClear this selection's saved pages without contacting the host.
Pages are shared by all prefix lengths. max_rows also selects an old-format materialized prefix entry to remove, when one exists.
| Parameter | Type | Default |
|---|---|---|
max_rowskeyword only | int | None | None |
Returns None
to_dataframemethod
to_dataframe(*, batch_size: int = 5000, progress: bool | None = None, refresh: bool = False, max_rows: int | None = None) -> pd.DataFrameMaterialize a dataset or view, resuming saved pages automatically.
Every load resolves current metadata and permissions. Interrupted downloads resume at a pinned source version, even beyond cache_ttl. Without a content validator this is a resumable local snapshot, not a guarantee against upstream corrections. Completed downloads require a matching content token or an explicit TTL; cache_ttl=None opts into snapshot reuse until refresh or a source version change.
refresh=True restarts the download. max_rows caps output rows after selection/aggregation; whole downloaded pages are retained so a later larger request can reuse their tails. iter_batches uses the same checkpoints without materializing the full dataset in memory.
| Parameter | Type | Default |
|---|---|---|
batch_sizekeyword only | int | 5000 |
progresskeyword only | bool | None | None |
refreshkeyword only | bool | False |
max_rowskeyword only | int | None | None |
Returns pd.DataFrame
to_timeseriesmethod
to_timeseries(*, frequency: str | None = None, aggregation: str = 'mean', fill: Literal['ffill', 'interpolate'] | None = None, value_columns: Sequence[str] | None = None, progress: bool | None = None) -> TimeSeriesCreate an Anomx TimeSeries, optionally resampling irregular observations.
The native forecasting container uses regular, timezone-free indices; aware times are normalized to UTC and their original timezone is stored in metadata. Resampling leaves empty bins as NaN unless fill is explicitly selected. Without a frequency, irregular data is rejected by TimeSeries; the original observations remain available as a frame.
| Parameter | Type | Default |
|---|---|---|
frequencykeyword only | str | None | None |
aggregationkeyword only | str | 'mean' |
fillkeyword only | Literal['ffill', 'interpolate'] | None | None |
value_columnskeyword only | Sequence[str] | None | None |
progresskeyword only | bool | None | None |
Returns TimeSeries
plotmethod
plot(*, ax: Any = None, **kwargs: Any) -> AnyPlot observations on a Matplotlib axes and return that axes.
| Parameter | Type | Default |
|---|---|---|
axkeyword only | Any | None |
**kwargs | Any | — |
Returns Any
dataproperty
data: pd.DataFrameLoad and return the observations as a DataFrame.
Returns pd.DataFrame
valuesmethod
values() -> NDArray[Any]Return observations as a NumPy array.
Returns NDArray[Any]
from_dataframeclassmethod
from_dataframe(frame: pd.DataFrame, *, time_column: str | None = None, kind: DatasetKind | None = None, mode: DatasetMode = 'fixed', name: str = '', metadata: Mapping[str, Any] | None = None) -> DatasetCreate a local dataset, preserving a supplied temporal index.
| Parameter | Type | Default |
|---|---|---|
frame | pd.DataFrame | required |
time_columnkeyword only | str | None | None |
kindkeyword only | DatasetKind | None | None |
modekeyword only | DatasetMode | 'fixed' |
namekeyword only | str | '' |
metadatakeyword only | Mapping[str, Any] | None | None |
Returns Dataset
from_valuesclassmethod
from_values(values: ArrayLike, *, columns: Sequence[str] | None = None, kind: DatasetKind = 'time_series', name: str = '', metadata: Mapping[str, Any] | None = None) -> DatasetCreate a regularly indexed time series from a 1D or 2D array.
| Parameter | Type | Default |
|---|---|---|
values | ArrayLike | required |
columnskeyword only | Sequence[str] | None | None |
kindkeyword only | DatasetKind | 'time_series' |
namekeyword only | str | '' |
metadatakeyword only | Mapping[str, Any] | None | None |
Returns Dataset
from_channelsclassmethod
from_channels(channels: Sequence[str], *, client: DatasetClient | PlatformClient | None = None, loader: Callable[..., object] | None = None, start: str | None = None, end: str | None = None, frequency: str | None = None) -> DatasetResolve recorded channels through an API connection or host loader.
| Parameter | Type | Default |
|---|---|---|
channels | Sequence[str] | required |
clientkeyword only | DatasetClient | PlatformClient | None | None |
loaderkeyword only | Callable[..., object] | None | None |
startkeyword only | str | None | None |
endkeyword only | str | None | None |
frequencykeyword only | str | None | None |
Returns Dataset
start_timemethod
start_time() -> objectReturn the first temporal index value; materializes a remote selection.
Returns object
end_timemethod
end_time() -> objectReturn the last temporal index value; materializes a remote selection.
Returns object
to_dartsmethod
to_darts() -> DartsDatasetConvert temporal data for optional Darts models without a core dependency.
Returns DartsDataset
DatasetKind
DatasetKind = Literal['time_series', 'sequence', 'independent_samples']DatasetMode
DatasetMode = Literal['fixed', 'automatic']