MODULE REFERENCE

base_data_transformer

anomx.data.processing.transformers.base_data_transformer

Data Transformer Base Class

BaseDataTransformer

BaseDataTransformer(name: str = 'BaseDataTransformer', n_jobs: int = 1, verbose: bool = False, parallel_params: bool | Sequence[str] = False, mask_components: bool = True, columns: str | list[str] | None = None, uses_insample: bool = False)

Bases: ABC

Signature parameters
Parameter	Type	Default
`name`	`str`	`'BaseDataTransformer'`
`n_jobs`	`int`	`1`
`verbose`	`bool`	`False`
`parallel_params`	`bool \| Sequence[str]`	`False`
`mask_components`	`bool`	`True`
`columns`	`str \| list[str] \| None`	`None`
`uses_insample`	`bool`	`False`

Abstract class for data transformers.

All the deriving classes have to implement the static method ts_transform; this implemented method can then be applied to TimeSeries or Sequence[TimeSeries] inputs by calling the transform() method. Internally, transform() parallelizes func:ts_transform over all of the TimeSeries inputs passed to it. See the func:ts_transform method documentation for further details on how to implement this method in a user-defined class.

Data transformers requiring to be fit first before calling transform should derive from anomx.data.processing.transformers.FittableDataTransformer instead. Data transformers that are invertible should derive from anomx.data.processing.transformers.InvertibleDataTransformer instead. Transformers which are both fittable and invertible should inherit from both anomx.data.processing.transformers.FittableDataTransformer and anomx.data.processing.transformers.InvertibleDataTransformer.

All Data Transformers can store *fixed parameters* that are automatically passed to func:ts_transform; the fixed parameters of a data transformer object are taken to be all those attributes defined in the __init__ method of the child-most class *before* super().__init__ is called. The fixed parameter values can then be accessed within the func:ts_transform method through the params dictionary argument. More specifically, params['fixed'] stores a dictionary with all of the fixed parameter values, where the keys are simply the attribute names of each fixed parameter (e.g. the self._my_param fixed parameter attribute is accessed through params['fixed']['_my_param']).

Data Transformers which inherit from anomx.data.processing.transformers.FittableDataTransformer can also store fitted parameters alongside fixed parameters; please refer to the anomx.data.processing.transformers.FittableDataTransformer documentation for further details.

Note: the ts_transform method is designed to be a static method instead of a instance method to allow an efficient parallelisation also when the scaler instance is storing a non-negligible amount of data. Using an instance method would imply copying the instance's data through multiple processes, which can easily introduce a bottleneck and nullify parallelisation benefits.

Parameters

name: The data transformer's name
n_jobs: The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input to a method, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors. Note: for a small amount of data, the parallelisation overhead could end up increasing the total required amount of time.
verbose: Optionally, whether to print operations progress
parallel_params: Optionally, specifies which fixed parameters (i.e. the attributes initialized in the child-most class's __init__) take on different values for different parallel jobs. Fixed parameters specified by parallel_params are assumed to be a Sequence of values that should be used for that parameter in each parallel job; the length of this Sequence should equal the number of parallel jobs. If parallel_params=True, every fixed parameter will take on a different value for each parallel job. If parallel_params=False, every fixed parameter will take on the same value for each parallel job. If parallel_params is a Sequence of fixed attribute names, only those attribute names specified will take on different values between different parallel jobs.
mask_components: Optionally, whether to automatically apply any provided component_masks to the TimeSeries inputs passed to transform, fit, inverse_transform, or fit_transform. If True, any specified component_mask will be applied to each input timeseries before passing them to the called method; the masked components will also be automatically 'unmasked' in the returned TimeSeries. If False, then component_mask (if provided) will be passed as a keyword argument, but won't automatically be applied to the input timeseries. See apply_component_mask for further details.
columns: Optionally, a string or list of strings specifying the names of the components (columns) to transform. If specified, only these components will be transformed, and the remaining components will be kept untouched. If None, all components are transformed. Note that if columns is provided, the mask_components attribute must be set to True.
uses_insample: Whether the transformer requires the in-sample (historic) series during inverse transformation. If True, inverse_transform will use the insample argument to pass the transformed historic series to ts_inverse_transform. This is needed when inverse transforming a partial series (e.g. a forecast) requires information from earlier times (e.g. for anomx.data.processing.transformers.diff.Diff).

Example

>>> from anomx.data.processing.transformers import BaseDataTransformer
>>> from anomx._series_utils.timeseries_generation import linear_timeseries
>>>
>>> class SimpleTransform(BaseDataTransformer):
>>>     def __init__(self, a):
>>>         self._a = a
>>>         super().__init__()
>>>
>>>     @staticmethod
>>>     def ts_transform(series, params, **kwargs):
>>>         a = params['fixed']['_a']
>>>         b = kwargs.pop('b')
>>>         return a*series + b
>>>
>>> series = linear_timeseries(length=5)
>>> print(series.values())
[[0.  ]
 [0.25]
 [0.5 ]
 [0.75]
 [1.  ]]
>>> series = SimpleTransform(a=2).transform(series, b=3)
>>> print(series.values())
[[3. ]
 [3.5]
 [4. ]
 [4.5]
 [5. ]]

Members

set_verbosemethod

# BaseDataTransformer.set_verbose Source ↗

set_verbose(value: bool)

Set the verbosity status.

True for enabling the detailed report about scaler's operation progress, False for no additional information.

Parameters

value: New verbosity status

Signature parameters
Parameter	Type	Default
`value`	`bool`	`required`

set_n_jobsmethod

# BaseDataTransformer.set_n_jobs Source ↗

set_n_jobs(value: int)

Set the number of processors to be used by the transformer while processing multiple TimeSeries.

Parameters

value: New n_jobs value. Set to -1 for using all the available cores.

Signature parameters
Parameter	Type	Default
`value`	`int`	`required`

ts_transformstaticmethod

# BaseDataTransformer.ts_transform Source ↗

ts_transform(series: TimeSeries, params: Mapping[str, Any]) -> TimeSeries

The function that will be applied to each series when transform() is called.

This method is not implemented in the base class and must be implemented in the deriving classes.

The function must take as first argument a TimeSeries object and, as a second argument, a dictionary containing the fixed and/or fitted parameters of the transformation; this function should then return a transformed TimeSeries object.

The params dictionary *can* contain up to two keys:

params['fixed'] stores the fixed parameters of the transformation (i.e. attributed defined in the __init__ method of the child-most class *before* super().__init__ is called); params['fixed'] is a dictionary itself, whose keys are the names of the fixed parameter attributes. For example, if _my_fixed_param is defined as an attribute in the child-most class, then this fixed parameter value can be accessed through params['fixed']['_my_fixed_param'].
If the transform inherits from the anomx.data.processing.transformers.FittableDataTransformer class, then params['fitted'] will store the fitted parameters of the transformation; the fitted parameters are simply the output(s) returned by the ts_fit function, whatever those output(s) may be. See anomx.data.processing.transformers.FittableDataTransformer for further details about fitted parameters.

Any positional/keyword argument supplied to the transform method are passed as positional/keyword arguments to ts_transform; hence, ts_transform should also accept *args and/or **kwargs if positional/keyword arguments are passed to transform. Note that if the mask_components attribute of BaseDataTransformer is set to False, then the component_mask provided to transform will be passed as an additional keyword argument to ts_transform.

The BaseDataTransformer class includes some helper methods which may prove useful when implementing a ts_transform function:

The apply_component_mask and unapply_component_mask methods, which apply and 'unapply' component_masks to a TimeSeries respectively; these methods are automatically called in transform if the mask_component attribute of BaseDataTransformer is set to True, but you may want to manually call them if you set mask_components to False and wish to manually specify how component_masks are applied to a TimeSeries.
The stack_samples method, which stacks all the samples in a TimeSeries along the component axis, so that the TimeSeries goes from shape (n_timesteps, n_components, n_samples) to shape (n_timesteps, n_components * n_samples). This stacking is useful if a pointwise transform is being implemented (i.e. transforming the value at time t depends only on the value of the series at that time t). Once transformed, the stacked TimeSeries can be 'unstacked' using the unstack_samples method.

Parameters

series: series to be transformed.
params: Dictionary containing the parameters of the transformation function. Fixed parameters (i.e. attributes defined in the child-most class of the transformation prior to calling super.__init__()) are stored under the 'fixed' key. If the transformation inherits from the FittableDataTransformer class, then the fitted parameters of the transformation (i.e. the values returned by ts_fit) are stored under the 'fitted' key.
args: Any poisitional arguments provided in addition to series when
kwargs: Any additional keyword arguments provided to transform. Note that if the mask_component attribute of BaseDataTransformer is set to False, then component_mask will be passed as a keyword argument.

Notes

This method is designed to be a static method instead of instance method to allow an efficient parallelisation also when the scaler instance is storing a non-negligible amount of data. Using instance methods would imply copying the instance's data through multiple processes, which can easily introduce a bottleneck and nullify parallelisation benefits.

Signature parameters
Parameter	Type	Default
`series`	`TimeSeries`	`required`
`params`	`Mapping[str, Any]`	`required`

Returns TimeSeries

transformmethod

# BaseDataTransformer.transform Source ↗

transform(series: TimeSeriesLike, *args, component_mask: np.ndarray | None = None, series_idx: int | Sequence[int] | None = None, **kwargs) -> TimeSeries | list[TimeSeries]

Transforms a (sequence of) of series by calling the user-implemeneted ts_transform method.

In case a Sequence[TimeSeries] is passed as input data, this function takes care of parallelising the transformation of multiple series in the sequence at the same time. Additionally, if the mask_components attribute was set to True when instantiating BaseDataTransformer, then any provided component_masks will be automatically applied to each input TimeSeries; please refer to 'Notes' for further details on component masking.

Any additionally specified *args and **kwargs are automatically passed to ts_transform.

Parameters

series: (sequence of) series to be transformed.
args: Additional positional arguments for each ts_transform() method call
component_mask: Optionally, a 1-D boolean np.ndarray of length series.n_components that specifies which components of the underlying series the transform should consider. If the mask_components attribute was set to True when instantiating BaseDataTransformer, then the component mask will be automatically applied to each TimeSeries input. Otherwise, component_mask will be provided as an addition keyword argument to ts_transform. See 'Notes' for further details.
series_idx: Optionally, the index(es) of each series corresponding to their positions within the series used to fit the transformer (to retrieve the appropriate transformer parameters).
kwargs: Additional keyword arguments for each ts_transform() method call

Returns

TimeSeriesLike: Transformed data.

Notes

If the mask_components attribute was set to True when instantiating BaseDataTransformer, then any provided component_masks will be automatically applied to each TimeSeries input to transform; component_masks are simply boolean arrays of shape (series.n_components,) that specify which components of each series should be transformed using ts_transform and which components should not. If component_mask[i] is True, then the ith component of each series will be transformed by ts_transform. Conversely, if component_mask[i] is False, the ith component will be removed from each series before being passed to ts_transform; after transforming this masked series, the untransformed ith component will be 'added back' to the output. Note that automatic component_masking can only be performed if the ts_transform does *not* change the number of timesteps in each series; if this were to happen, then the transformed and untransformed components are unable to be concatenated back together along the component axis.

If mask_components was set to False when instantiating BaseDataTransformer, then any provided component_masks will be passed as a keyword argument ts_transform; the user can then manually specify how the component_mask should be applied to each series.

Alternatively, if the columns parameter was specified when instantiating the transformer, the component_mask is automatically generated to target only the specified column names.

Signature parameters
Parameter	Type	Default
`series`	`TimeSeriesLike`	`required`
`*args`	`—`	`—`
`component_mask`keyword only	`np.ndarray \| None`	`None`
`series_idx`keyword only	`int \| Sequence[int] \| None`	`None`
`**kwargs`	`—`	`—`

Returns TimeSeries | list[TimeSeries]

apply_component_maskstaticmethod

# BaseDataTransformer.apply_component_mask Source ↗

apply_component_mask(series: TimeSeries, component_mask: np.ndarray | None = None, return_ts: bool = False) -> TimeSeriesLike | np.ndarray | Sequence[np.ndarray]

Extracts components specified by component_mask from series

Parameters

series: input TimeSeries to be fed into transformer.
component_mask: Optionally, np.ndarray boolean mask of shape (n_components, 1) specifying which components to extract from series. The ith component of series is kept only if component_mask[i] = True. If not specified, no masking is performed.
return_ts: Optionally, specifies that a TimeSeries should be returned, rather than an np.ndarray.

Returns

masked: TimeSeries (if return_ts = True) or np.ndarray (if return_ts = False) with only those components specified by component_mask remaining.

Signature parameters
Parameter	Type	Default
`series`	`TimeSeries`	`required`
`component_mask`	`np.ndarray \| None`	`None`
`return_ts`	`bool`	`False`

Returns TimeSeriesLike | np.ndarray | Sequence[np.ndarray]

unapply_component_maskstaticmethod

# BaseDataTransformer.unapply_component_mask Source ↗

unapply_component_mask(series: TimeSeriesLike, vals: np.ndarray | Sequence[np.ndarray] | TimeSeriesLike, component_mask: np.ndarray | None = None) -> np.ndarray | Sequence[np.ndarray] | TimeSeriesLike

Adds back components previously removed by component_mask in apply_component_mask method.

Parameters

series: input TimeSeries that was fed into transformer.
vals:: np.ndarray or TimeSeries to 'unmask'
component_mask: Optionally, np.ndarray boolean mask of shape (n_components, 1) specifying which components were extracted from series. If given, insert vals back into the columns of the original array. If not specified, nothing is 'unmasked'.

Returns

unmasked: TimeSeries (if vals is a TimeSeries) or np.ndarray (if vals is an np.ndarray) with those components previously removed by component_mask now 'added back'.

Signature parameters
Parameter	Type	Default
`series`	`TimeSeriesLike`	`required`
`vals`	`np.ndarray \| Sequence[np.ndarray] \| TimeSeriesLike`	`required`
`component_mask`	`np.ndarray \| None`	`None`

Returns np.ndarray | Sequence[np.ndarray] | TimeSeriesLike

stack_samplesstaticmethod

# BaseDataTransformer.stack_samples Source ↗

stack_samples(vals: np.ndarray | TimeSeries) -> np.ndarray

Creates an array of shape (n_timesteps * n_samples, n_components) from either a TimeSeries or the array_values of a TimeSeries.

Each column of the returned array corresponds to a component (dimension) of the series and is formed by concatenating all of the samples associated with that component together. More specifically, the ith column is formed by concatenating [component_i_sample_1, component_i_sample_2, ..., component_i_sample_n].

Stacking is useful when implementing a transformation that applies the exact same change to every timestep in the timeseries. In such cases, the samples of each component can be stacked together into a single column, and the transformation can then be applied to each column, thereby 'vectorising' the transformation over all samples of that component; the unstack_samples method can then be used to reshape the output. For transformations that depend on the time_index or the temporal ordering of the observations, stacking should *not* be employed.

Parameters

vals: Timeseries or np.ndarray of shape (n_timesteps, n_components, n_samples) to be 'stacked'.

Returns

stacked: np.ndarray of shape (n_timesteps * n_samples, n_components), where the ith column is formed by concatenating all of the samples of the ith component in vals.

Signature parameters
Parameter	Type	Default
`vals`	`np.ndarray \| TimeSeries`	`required`

Returns np.ndarray

unstack_samplesstaticmethod

# BaseDataTransformer.unstack_samples Source ↗

unstack_samples(vals: np.ndarray, n_timesteps: int | None = None, n_samples: int | None = None, series: TimeSeries | None = None) -> np.ndarray

Reshapes the 2D array returned by stack_samples back into an array of shape (n_timesteps, n_components, n_samples); this 'undoes' the reshaping of stack_samples. Either n_components, n_samples, or series must be specified.

Parameters

vals: np.ndarray of shape (n_timesteps * n_samples, n_components) to be 'unstacked'.
n_timesteps: Optionally, the number of timesteps in the array originally passed to stack_samples. Does *not* need to be provided if series is specified.
n_samples: Optionally, the number of samples in the array originally passed to stack_samples. Does *not* need to be provided if series is specified.
series: Optionally, the TimeSeries object used to create vals; n_samples is inferred from this.

Returns

unstacked: np.ndarray of shape (n_timesteps, n_components, n_samples).

Signature parameters
Parameter	Type	Default
`vals`	`np.ndarray`	`required`
`n_timesteps`	`int \| None`	`None`
`n_samples`	`int \| None`	`None`
`series`	`TimeSeries \| None`	`None`

Returns np.ndarray

nameproperty

# BaseDataTransformer.name Source ↗

name

Name of the data transformer.

functionView source ↗

component_masking

component_masking(transformer_method)

Applies component masking to the series fed to any transform method, and then reverts the masking for the final output series.

Signature parameters
Parameter	Type	Default
`transformer_method`	`—`	`required`