This page contains the auto-generated API reference documentation.

Preprocessing Config

class sequifier.config.preprocess_config.PreprocessorModel(*, project_root: str, data_path: str, read_format: str = 'csv', write_format: str = 'parquet', merge_output: bool = True, selected_columns: list[str] | None = None, split_ratios: list[float], seq_length: int, stride_by_split: list[int] | None = None, max_rows: int | None = None, seed: int, n_cores: int | None = None, batches_per_file: int = 1024, process_by_file: bool = True, continue_preprocessing: bool = False, subsequence_start_mode: str = 'distribute', use_precomputed_maps: list[str] | None = None)

Pydantic model for preprocessor configuration.

project_root

The path to the sequifier project directory.

Type:

str

data_path

The path to the input data file.

Type:

str

read_format

The file type of the input data. Can be ‘csv’ or ‘parquet’.

Type:

str

write_format

The file type for the preprocessed output data.

Type:

str

merge_output

If True, combines all preprocessed data into a single file.

Type:

bool

selected_columns

A list of columns to be included in the preprocessing. If None, all columns are used.

Type:

list[str] | None

split_ratios

A list of floats that define the relative sizes of data splits (e.g., for train, validation, test). The sum of proportions must be 1.0.

Type:

list[float]

seq_length

The sequence length for the model inputs.

Type:

int

stride_by_split

A list of step sizes for creating subsequences within each data split.

Type:

list[int] | None

max_rows

The maximum number of input rows to process. If None, all rows are processed.

Type:

int | None

seed

A random seed for reproducibility.

Type:

int

n_cores

The number of CPU cores to use for parallel processing. If None, it uses the available CPU cores.

Type:

int | None

batches_per_file

The number of batches to process per file.

Type:

int

process_by_file

A flag to indicate if processing should be done file by file.

Type:

bool

continue_preprocessing

Continue preprocessing job that was interrupted while writing to temp folder.

Type:

bool

subsequence_start_mode

“distribute” to minimize max subsequence overlap, or “exact”.

Type:

str

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

Training Config

class sequifier.config.train_config.ModelSpecModel(*, initial_embedding_dim: int, feature_embedding_dims: dict[str, int] | None = None, joint_embedding_dim: int | None = None, dim_model: int, n_head: int, dim_feedforward: int, num_layers: int, activation_fn: str = 'swiglu', normalization: str = 'rmsnorm', positional_encoding: str = 'learned', attention_type: str = 'mha', norm_first: bool = True, n_kv_heads: int | None = None, rope_theta: float = 10000.0, prediction_length: int)

Pydantic model for model specifications.

initial_embedding_dim

The size of the input embedding. Must be equal to dim_model if joint_embedding_dim is None.

Type:

int

feature_embedding_dims

The embedding dimensions for each input column. Must sum to initial_embedding_dim.

Type:

dict[str, int] | None

joint_embedding_dim

Joint embedding layer after initial embedding. Must be equal to dim_model if specified.

Type:

int | None

n_head

The number of heads in the multi-head attention models.

Type:

int

dim_feedforward

The dimension of the feedforward network model.

Type:

int

num_layers

The number of layers in the transformer model.

Type:

int

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sequifier.config.train_config.TrainModel(*, project_root: str, metadata_config_path: str, model_name: str, training_data_path: str, validation_data_path: str, read_format: str = 'parquet', input_columns: list[str], column_types: dict[str, str], categorical_columns: list[str], real_columns: list[str], target_columns: list[str], target_column_types: dict[str, str], id_maps: dict[str, dict[str | int, int]], seq_length: int, n_classes: dict[str, int], inference_batch_size: int, seed: int, export_generative_model: bool, export_embedding_model: bool, export_onnx: bool = True, export_pt: bool = False, export_with_dropout: bool = False, model_spec: ModelSpecModel, training_spec: TrainingSpecModel)

Pydantic model for training configuration.

project_root

The path to the sequifier project directory.

Type:

str

metadata_config_path

The path to the data-driven configuration file.

Type:

str

model_name

The name of the model being trained.

Type:

str

training_data_path

The path to the training data.

Type:

str

validation_data_path

The path to the validation data.

Type:

str

read_format

The file format of the input data (e.g., ‘csv’, ‘parquet’).

Type:

str

input_columns

The list of input columns to be used for training.

Type:

list[str]

column_types

A dictionary mapping each column to its numeric type (‘int64’ or ‘float64’).

Type:

dict[str, str]

categorical_columns

A list of columns that are categorical.

Type:

list[str]

real_columns

A list of columns that are real-valued.

Type:

list[str]

target_columns

The list of target columns for model training.

Type:

list[str]

target_column_types

A dictionary mapping target columns to their types (‘categorical’ or ‘real’).

Type:

dict[str, str]

id_maps

For each categorical column, a map from distinct values to their indexed representation.

Type:

dict[str, dict[str | int, int]]

seq_length

The sequence length of the model’s input.

Type:

int

n_classes

The number of classes for each categorical column.

Type:

dict[str, int]

inference_batch_size

The batch size to be used for inference after model export.

Type:

int

seed

The random seed for numpy and PyTorch.

Type:

int

export_generative_model

If True, exports the generative model.

Type:

bool

export_embedding_model

If True, exports the embedding model.

Type:

bool

export_onnx

If True, exports the model in ONNX format.

Type:

bool

export_pt

If True, exports the model using torch.save.

Type:

bool

export_with_dropout

If True, exports the model with dropout enabled.

Type:

bool

model_spec

The specification of the transformer model architecture.

Type:

sequifier.config.train_config.ModelSpecModel

training_spec

The specification of the training run configuration.

Type:

sequifier.config.train_config.TrainingSpecModel

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class sequifier.config.train_config.TrainingSpecModel(*, device: str, device_max_concat_length: int = 12, epochs: int, log_interval: int = 10, class_share_log_columns: list[str] = <factory>, early_stopping_epochs: int | None = None, save_interval_epochs: int, batch_size: int, learning_rate: float, criterion: dict[str, str], class_weights: dict[str, list[float]] | None = None, accumulation_steps: int | None = None, dropout: float = 0.0, loss_weights: dict[str, float] | None = None, optimizer: ~sequifier.config.train_config.DotDict = <factory>, scheduler: ~sequifier.config.train_config.DotDict = <factory>, scheduler_step_on: str = 'epoch', continue_training: bool = True, enforce_determinism: bool = False, distributed: bool = False, load_full_data_to_ram: bool = True, max_ram_gb: int | float = 16, world_size: int = 1, num_workers: int = 0, backend: str = 'nccl', layer_type_dtypes: dict[str, str] | None = None, layer_autocast: bool | None = True, sampling_strategy: str = 'exact')

Pydantic model for training specifications.

device

The torch.device to train the model on (e.g., ‘cuda’, ‘cpu’, ‘mps’).

Type:

str

device_max_concat_length

Maximum sequence length for concatenation on device.

Type:

int

epochs

The total number of epochs to train for.

Type:

int

log_interval

The interval in batches for logging.

Type:

int

class_share_log_columns

A list of column names for which to log the class share of predictions.

Type:

list[str]

early_stopping_epochs

Number of epochs to wait for validation loss improvement before stopping.

Type:

int | None

save_interval_epochs

The interval in epochs for checkpointing the model.

Type:

int

batch_size

The training batch size.

Type:

int

learning_rate

The learning rate.

Type:

float

criterion

A dictionary mapping each target column to a loss function.

Type:

dict[str, str]

class_weights

A dictionary mapping categorical target columns to a list of class weights.

Type:

dict[str, list[float]] | None

accumulation_steps

The number of gradient accumulation steps.

Type:

int | None

dropout

The dropout value for the transformer model.

Type:

float

loss_weights

A dictionary mapping columns to specific loss weights.

Type:

dict[str, float] | None

optimizer

The optimizer configuration.

Type:

sequifier.config.train_config.DotDict

scheduler

The learning rate scheduler configuration.

Type:

sequifier.config.train_config.DotDict

scheduler_step_on

The time of the .step() call on the scheduler, either ‘epoch’ or ‘batch’

Type:

str

continue_training

If True, continue training from the latest checkpoint.

Type:

bool

distributed

If True, enables distributed training.

Type:

bool

load_full_data_to_ram

If True, loads the entire dataset into RAM.

Type:

bool

world_size

The number of processes for distributed training.

Type:

int

num_workers

The number of worker threads for data loading.

Type:

int

backend

The distributed training backend (e.g., ‘nccl’).

Type:

str

layer_type_dtypes

Dictionary mapping layer types (linear, embedding, norm) to dtypes (bfloat16, float8_e4m3fn).

Type:

dict[str, str] | None

layer_autocast

Whether to use autocast

Type:

bool | None

sampling_strategy

how to equalize data between GPUs

Type:

str

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

Inference Config

class sequifier.config.infer_config.InfererModel(*, project_root: str, metadata_config_path: str, model_path: str | list[str], model_type: str, data_path: str, training_config_path: str = 'configs/train.yaml', read_format: str = 'parquet', write_format: str = 'csv', input_columns: list[str], categorical_columns: list[str], real_columns: list[str], target_columns: list[str], column_types: dict[str, str], target_column_types: dict[str, str], enforce_determinism: bool = False, output_probabilities: bool = False, map_to_id: bool = True, seed: int, device: str, seq_length: int, prediction_length: int = 1, inference_batch_size: int, distributed: bool = False, load_full_data_to_ram: bool = True, world_size: int = 1, num_workers: int = 0, sample_from_distribution_columns: list[str] | None = None, infer_with_dropout: bool = False, autoregression: bool = False, autoregression_extra_steps: int | None = None)

Pydantic model for inference configuration.

project_root

The path to the sequifier project directory.

Type:

str

metadata_config_path

The path to the data-driven configuration file.

Type:

str

model_path

The path to the trained model file(s).

Type:

str | list[str]

model_type

The type of model, either ‘embedding’ or ‘generative’.

Type:

str

data_path

The path to the data to be used for inference.

Type:

str

training_config_path

The path to the training configuration file.

Type:

str

read_format

The file format of the input data (e.g., ‘csv’, ‘parquet’).

Type:

str

write_format

The file format for the inference output.

Type:

str

input_columns

The list of input columns used for inference.

Type:

list[str]

categorical_columns

A list of columns that are categorical.

Type:

list[str]

real_columns

A list of columns that are real-valued.

Type:

list[str]

target_columns

The list of target columns for inference.

Type:

list[str]

column_types

A dictionary mapping each column to its numeric type (‘int64’ or ‘float64’).

Type:

dict[str, str]

target_column_types

A dictionary mapping target columns to their types (‘categorical’ or ‘real’).

Type:

dict[str, str]

output_probabilities

If True, outputs the probability distributions for categorical target columns.

Type:

bool

map_to_id

If True, maps categorical output values back to their original IDs.

Type:

bool

seed

The random seed for reproducibility.

Type:

int

device

The device to run inference on (e.g., ‘cuda’, ‘cpu’, ‘mps’).

Type:

str

seq_length

The sequence length of the model’s input.

Type:

int

inference_batch_size

The batch size for inference.

Type:

int

distributed

If True, enables distributed inference.

Type:

bool

load_full_data_to_ram

If True, loads the entire dataset into RAM.

Type:

bool

world_size

The number of processes for distributed inference.

Type:

int

num_workers

The number of worker threads for data loading.

Type:

int

sample_from_distribution_columns

A list of columns from which to sample from the distribution.

Type:

list[str] | None

infer_with_dropout

If True, applies dropout during inference.

Type:

bool

autoregression

If True, performs autoregressive inference.

Type:

bool

autoregression_extra_steps

The number of additional steps for autoregressive inference.

Type:

int | None

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

Hyperparameter Search Config

class sequifier.config.hyperparameter_search_config.HyperparameterSearch(*, project_root: str, metadata_config_path: str, hp_search_name: str, search_strategy: str = 'sample', n_samples: int | None, model_config_write_path: str, training_data_path: str, validation_data_path: str, read_format: str = 'parquet', input_columns: list[list[str]], column_types: list[dict[str, str]], categorical_columns: list[list[str]], real_columns: list[list[str]], target_columns: list[str], target_column_types: dict[str, str], id_maps: dict[str, dict[str | int, int]], seq_length: list[int], n_classes: dict[str, int], inference_batch_size: int, export_generative_model: bool, export_embedding_model: bool, export_onnx: bool = True, export_pt: bool = False, export_with_dropout: bool = False, model_hyperparameter_sampling: ModelSpecHyperparameterSampling, training_hyperparameter_sampling: TrainingSpecHyperparameterSampling, override_input: bool = False)

Pydantic model for hyperparameter search configuration.

project_root

The path to the sequifier project directory.

Type:

str

metadata_config_path

The path to the data-driven configuration file.

Type:

str

hp_search_name

The name for the hyperparameter search.

Type:

str

search_strategy

The search strategy, either “sample” or “grid”.

Type:

str

n_samples

The number of samples to draw for the search.

Type:

int | None

model_config_write_path

The path to write the model configurations to.

Type:

str

training_data_path

The path to the training data.

Type:

str

validation_data_path

The path to the validation data.

Type:

str

read_format

The file format of the input data.

Type:

str

input_columns

A list of lists of columns to be used for training.

Type:

list[list[str]]

column_types

A list of dictionaries mapping columns to their types.

Type:

list[dict[str, str]]

categorical_columns

A list of lists of categorical columns.

Type:

list[list[str]]

real_columns

A list of lists of real-valued columns.

Type:

list[list[str]]

target_columns

The list of target columns for model training.

Type:

list[str]

target_column_types

A dictionary mapping target columns to their types.

Type:

dict[str, str]

id_maps

A dictionary mapping categorical values to their indexed representation.

Type:

dict[str, dict[str | int, int]]

seq_length

A list of possible sequence lengths.

Type:

list[int]

n_classes

The number of classes for each categorical column.

Type:

dict[str, int]

inference_batch_size

The batch size for inference.

Type:

int

export_onnx

If True, exports the model in ONNX format.

Type:

bool

export_pt

If True, exports the model using torch.save.

Type:

bool

export_with_dropout

If True, exports the model with dropout enabled.

Type:

bool

model_hyperparameter_sampling

The sampling configuration for model hyperparameters.

Type:

sequifier.config.hyperparameter_search_config.ModelSpecHyperparameterSampling

training_hyperparameter_sampling

The sampling configuration for training hyperparameters.

Type:

sequifier.config.hyperparameter_search_config.TrainingSpecHyperparameterSampling

grid_sample(i)

Select a full training configuration based on a grid search index.

This method generates a grid of all possible configurations and selects the configuration at the given index.

Parameters:

i – The index of the configuration to select from the grid.

Returns:

A TrainModel instance populated with the selected configuration.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

n_combinations()

Calculate the total number of possible configurations.

This method computes the total number of unique configurations that can be generated by a grid search over all defined hyperparameters.

Returns:

The total number of possible hyperparameter configurations.

random_sample(i)

Randomly sample a full training configuration.

This method generates a complete training configuration by randomly sampling model and training hyperparameters, as well as selecting a column set and sequence length.

Parameters:

i – The index of the sample, used to create a unique model name.

Returns:

A TrainModel instance populated with a randomly sampled configuration.

sample(i)

Sample a configuration based on the specified search strategy.

This method delegates to either random_sample or grid_sample based on the search_strategy attribute.

Parameters:

i – The index of the sample or grid combination to generate.

Returns:

A TrainModel instance with a generated configuration.

Raises:

Exception – If the search_strategy is not ‘sample’ or ‘grid’.

class sequifier.config.hyperparameter_search_config.ModelSpecHyperparameterSampling(*, initial_embedding_dim: list[int], joint_embedding_dim: list[Optional[int]], dim_model: list[int], feature_embedding_dims: list[dict[str, int]] | None, n_head: list[int], dim_feedforward: list[int], num_layers: list[int], prediction_length: int, activation_fn: list[str], normalization: list[str], positional_encoding: list[str], attention_type: list[str], norm_first: list[bool], n_kv_heads: list[Optional[int]], rope_theta: list[float])

Pydantic model for model specification hyperparameter sampling.

initial_embedding_dim

A list of possible sizes for the initial input embedding.

Type:

list[int]

feature_embedding_dims

A list of possible dictionaries defining embedding dimensions for each input column.

Type:

list[dict[str, int]] | None

joint_embedding_dim

A list of possible sizes for the joint embedding layer projection.

Type:

list[Optional[int]]

dim_model

A list of possible numbers of expected features in the input (d_model).

Type:

list[int]

n_head

A list of possible numbers of heads in the multi-head attention models.

Type:

list[int]

dim_feedforward

A list of possible dimensions of the feedforward network model.

Type:

list[int]

num_layers

A list of possible numbers of layers in the transformer model.

Type:

list[int]

grid_sample(i)

Select a set of model hyperparameters based on a grid search index.

This method generates a grid of all possible model hyperparameter combinations and selects the combination at the given index. Includes sanitation logic to prevent invalid n_kv_heads combinations.

Parameters:

i – The index of the hyperparameter combination to select from the grid.

Returns:

A ModelSpecModel instance populated with the selected set of hyperparameters.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

n_combinations()

Calculate the total number of model hyperparameter combinations.

This method computes the total number of unique model hyperparameter sets that can be generated by the grid search.

Returns:

The total number of possible model hyperparameter combinations.

random_sample()

Randomly sample a set of model hyperparameters.

This method selects a random combination of model hyperparameters from the defined lists of possibilities. It ensures that dim_model, feature_embedding_dims, and n_head are paired correctly, and that n_kv_heads is a valid divisor of n_head.

Returns:

A ModelSpecModel instance populated with a randomly sampled set of hyperparameters.

class sequifier.config.hyperparameter_search_config.TrainingSpecHyperparameterSampling(*, device: str, epochs: list[int], log_interval: int = 10, class_share_log_columns: list[str] = <factory>, early_stopping_epochs: int | None = None, save_interval_epochs: int, batch_size: list[int], learning_rate: list[float], criterion: dict[str, str], class_weights: dict[str, list[float]] | None = None, accumulation_steps: list[int], dropout: list[float] = [0.0], loss_weights: dict[str, float] | None = None, optimizer: list[sequifier.config.train_config.DotDict] = <factory>, scheduler: list[sequifier.config.train_config.DotDict] = <factory>, continue_training: bool, scheduler_step_on: str = 'epoch', distributed: bool = False, load_full_data_to_ram: bool = True, max_ram_gb: int | float = 16, device_max_concat_length: int = 12, world_size: int = 1, num_workers: int = 0, backend: str = 'nccl', layer_type_dtypes: dict[str, str] | None = None, layer_autocast: bool | None = True, sampling_strategy: str = 'exact')

Pydantic model for training specification hyperparameter sampling.

device

The device to train on (e.g., ‘cuda’, ‘cpu’).

Type:

str

epochs

A list of possible numbers of epochs to train for.

Type:

list[int]

log_interval

The interval in batches for logging.

Type:

int

class_share_log_columns

Columns for which to log class share.

Type:

list[str]

early_stopping_epochs

Number of epochs for early stopping.

Type:

int | None

save_interval_epochs

Interval in epochs for saving model checkpoints.

Type:

int

batch_size

A list of possible batch sizes.

Type:

list[int]

learning_rate

A list of possible learning rates.

Type:

list[float]

criterion

A dictionary mapping target columns to loss functions.

Type:

dict[str, str]

class_weights

Optional dictionary mapping columns to class weights.

Type:

dict[str, list[float]] | None

accumulation_steps

A list of possible gradient accumulation steps.

Type:

list[int]

dropout

A list of possible dropout rates.

Type:

list[float]

loss_weights

Optional dictionary mapping columns to loss weights.

Type:

dict[str, float] | None

optimizer

A list of possible optimizer configurations.

Type:

list[sequifier.config.train_config.DotDict]

scheduler

A list of possible scheduler configurations.

Type:

list[sequifier.config.train_config.DotDict]

continue_training

Flag to continue training from a checkpoint.

Type:

bool

layer_type_dtypes

Dictionary mapping layer types (linear, embedding, norm) to dtypes (bfloat16, float8_e4m3fn).

Type:

dict[str, str] | None

layer_autocast

Whether to use autocast

Type:

bool | None

__init__(**kwargs)

Initialize the TrainingSpecHyperparameterSampling instance.

This method initializes the Pydantic BaseModel and then processes the optimizer and scheduler configurations from the provided keyword arguments, converting them into DotDict objects.

Parameters:

**kwargs – Keyword arguments that correspond to the attributes of this class. The ‘optimizer’ and ‘scheduler’ arguments are expected to be lists of dictionaries.

grid_sample(i)

Select a set of training hyperparameters based on a grid search index.

This method generates a grid of all possible hyperparameter combinations and selects the combination at the given index.

Parameters:

i – The index of the hyperparameter combination to select from the grid.

Returns:

A TrainingSpecModel instance populated with the selected set of hyperparameters.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

n_combinations()

Calculate the total number of hyperparameter combinations.

This method computes the total number of unique hyperparameter sets that can be generated by the grid search.

Returns:

The total number of possible hyperparameter combinations.

random_sample()

Randomly sample a set of training hyperparameters.

This method selects a random combination of hyperparameters from the defined lists of possibilities. It ensures that learning rates and schedulers are paired correctly.

Returns:

A TrainingSpecModel instance populated with a randomly sampled set of hyperparameters.

Non-standard Optimizers

class sequifier.optimizers.ademamix.AdEMAMix(params={}, lr=0.001, betas=(0.9, 0.999, 0.9999), eps=1e-08, weight_decay=0, alpha=5.0, T_alpha_beta3=None)

Implements the AdEMAMix optimizer.

This optimizer is based on the paper “AdEMAMix: A Novel Adaptive Optimizer for Deep Learning”. It combines the advantages of Adam and EMA, and introduces a mixing term to further improve performance.

Parameters:

• params (iterable) – Iterable of parameters to optimize or dicts defining parameter groups.

• learning_rate (float, optional) – Learning rate (default: 1e-3).

• betas (Tuple[float, float, float], optional) – Coefficients used for computing running averages of gradient and its square (default: (0.9, 0.999, 0.9999)).

• eps (float, optional) – Term added to the denominator to improve numerical stability (default: 1e-8).

• weight_decay (float, optional) – Weight decay (L2 penalty) (default: 0).

• alpha (float, optional) – Mixing coefficient (default: 5.0).

• T_alpha_beta3 (int, optional) – Time period for alpha and beta3 scheduling (default: None).

__setstate__(state)

Set the state of the optimizer.

Parameters:

state (dict) – The state of the optimizer.

step(closure=None)

Perform a single optimization step.

Parameters:

closure (callable, optional) – A closure that reevaluates the model and returns the loss. (default: None)

Returns:

The loss, if the closure is provided. Otherwise, returns None.

Internals

class sequifier.preprocess.Preprocessor(project_root: str, continue_preprocessing: bool, data_path: str, read_format: str, write_format: str, merge_output: bool, selected_columns: list[str] | None, split_ratios: list[float], seq_length: int, stride_by_split: list[int], max_rows: int | None, seed: int, n_cores: int | None, batches_per_file: int, process_by_file: bool, subsequence_start_mode: str, use_precomputed_maps: list[str] | None)

A class for preprocessing data for the sequifier model.

This class handles loading, preprocessing, and saving data. It supports single-file and multi-file processing, and can handle large datasets by processing them in batches.

project_root

The path to the sequifier project directory.

Type:

str

batches_per_file

The number of batches to process per file.

Type:

int

data_name_root

The root name of the data file.

Type:

str

merge_output

Whether to combine the output into a single file.

Type:

bool

target_dir

The target directory for temporary files.

Type:

str

seed

The random seed for reproducibility.

Type:

int

n_cores

The number of cores to use for parallel processing.

Type:

int

split_paths

The paths to the output split files.

Type:

list[str]

__init__(project_root: str, continue_preprocessing: bool, data_path: str, read_format: str, write_format: str, merge_output: bool, selected_columns: list[str] | None, split_ratios: list[float], seq_length: int, stride_by_split: list[int], max_rows: int | None, seed: int, n_cores: int | None, batches_per_file: int, process_by_file: bool, subsequence_start_mode: str, use_precomputed_maps: list[str] | None)

Initializes the Preprocessor with the given parameters.

Parameters:

• project_root – The path to the sequifier project directory.

• data_path – The path to the input data file.

• read_format – The file type of the input data.

• write_format – The file type for the preprocessed output data.

• merge_output – Whether to combine the output into a single file.

• selected_columns – A list of columns to be included in the preprocessing.

• split_ratios – A list of floats that define the relative sizes of data splits.

• seq_length – The sequence length for the model inputs.

• stride_by_split – A list of step sizes for creating subsequences.

• max_rows – The maximum number of input rows to process.

• seed – A random seed for reproducibility.

• n_cores – The number of CPU cores to use for parallel processing.

• batches_per_file – The number of batches to process per file.

• process_by_file – A flag to indicate if processing should be done file by file.

• use_precomputed_maps – An optional list of columns for which to enforce precomputed maps

sequifier.preprocess.cast_columns_to_string(data: DataFrame) → DataFrame

Casts the column names of a Polars DataFrame to strings.

This is often necessary because Polars schemas may use integers as column names (e.g., ‘0’, ‘1’, ‘2’…) which need to be strings for some operations.

Parameters:

data – The Polars DataFrame.

Returns:

The same DataFrame with its columns attribute modified.

sequifier.preprocess.combine_maps(map1: dict[Union[str, int], int], map2: dict[Union[str, int], int]) → dict[Union[str, int], int]

Combines two ID maps into a new, consolidated map.

Takes all unique keys from both map1 and map2, sorts them, and creates a new, single map where keys are mapped to 1-based indices based on the sorted order. This ensures a consistent mapping across different data chunks.

Parameters:

• map1 – The first ID map.

• map2 – The second ID map.

Returns:

A new, combined, and re-indexed ID map.

sequifier.preprocess.combine_multiprocessing_outputs(project_root: str, target_dir: str, n_splits: int, input_files: dict[int, list[str]], dataset_name: str, write_format: str, in_target_dir: bool = False, pre_split_str: str | None = None, post_split_str: str | None = None) → None

Combines multiple intermediate batch files into final split files.

This function iterates through each split and combines all the intermediate files listed in input_files[split] into a single final output file for that split.

• For “csv” format, it uses the csvstack command-line utility.

• For “parquet” format, it uses pyarrow.parquet.ParquetWriter to concatenate the files efficiently.

Parameters:

• project_root – The path to the sequifier project directory.

• target_dir – The temporary directory containing intermediate files.

• n_splits – The number of data splits.

• input_files – A dictionary mapping split index (int) to a list of input file paths (str) for that split.

• dataset_name – The root name for the final output files.

• write_format – The file format (“csv” or “parquet”).

• in_target_dir – If True, the final combined file is written inside target_dir. If False, it’s written to data/.

• pre_split_str – An optional string to insert into the filename before the “-split{i}” part.

• post_split_str – An optional string to insert into the filename after the “-split{i}” part.

sequifier.preprocess.combine_parquet_files(files: list[str], out_path: str) → None

Combines multiple Parquet files into a single Parquet file.

This function reads the schema from the first file and uses it to initialize a ParquetWriter. It then iterates through all files in the list, reading each one as a table and writing it to the new combined file. This is more memory-efficient than reading all files into one large table first.

Parameters:

• files – A list of paths to the Parquet files to combine.

• out_path – The path for the combined output Parquet file.

sequifier.preprocess.create_file_paths_for_multiple_files1(project_root: str, target_dir: str, n_splits: int, n_batches: int, process_id: int, file_index_str: str, dataset_name: str, write_format: str) → dict[int, list[str]]

Creates a dictionary of temporary file paths for a specific data file.

This is used in the multi-file, merge_output=True workflow. It generates file path names for intermediate batches before they are combined.

The naming pattern is: {dataset_name}-{process_id}-{file_index_str}-split{split}-{batch_id}.{write_format}

Parameters:

• project_root – The path to the sequifier project directory.

• target_dir – The temporary directory to place files in.

• n_splits – The number of data splits.

• n_batches – The number of batches created by the process.

• process_id – The ID of the multiprocessing worker.

• file_index_str – The index of the file being processed by this worker.

• dataset_name – The root name of the dataset.

• write_format – The file extension.

Returns:

A dictionary mapping a split index (int) to a list of file paths (str) for all batches in that split.

sequifier.preprocess.create_file_paths_for_multiple_files2(project_root: str, target_dir: str, n_splits: int, n_processes: int, n_files: dict[int, int], dataset_name: str, write_format: str) → dict[int, list[str]]

Creates a dictionary of intermediate file paths for a multi-file run.

This is used in the multi-file, merge_output=True workflow. It generates the file paths for the combined files from each process, which are the inputs to the final combination step.

The naming pattern is: {dataset_name}-{process_id}-{file_index}-split{split}.{write_format}

Parameters:

• project_root – The path to the sequifier project directory.

• target_dir – The temporary directory where files are located.

• n_splits – The number of data splits.

• n_processes – The total number of multiprocessing workers.

• n_files – A dictionary mapping process_id to the number of files that process handled.

• dataset_name – The root name of the dataset.

• write_format – The file extension.

Returns:

A dictionary mapping a split index (int) to a list of all intermediate combined file paths (str) for that split.

sequifier.preprocess.create_file_paths_for_single_file(project_root: str, target_dir: str, n_splits: int, n_batches: int, dataset_name: str, write_format: str) → dict[int, list[str]]

Creates a dictionary of temporary file paths for a single-file run.

This is used in the single-file, merge_output=True workflow. It generates file path names for intermediate batches created by different processes before they are combined.

The naming pattern is: {dataset_name}-split{split}-{core_id}.{write_format}

Parameters:

• project_root – The path to the sequifier project directory.

• target_dir – The temporary directory to place files in.

• n_splits – The number of data splits.

• n_batches – The number of processes (batches) running in parallel.

• dataset_name – The root name of the dataset.

• write_format – The file extension.

Returns:

A dictionary mapping a split index (int) to a list of file paths (str) for all batches in that split.

sequifier.preprocess.create_id_map(data: DataFrame, column: str) → dict[Union[str, int], int]

Creates a map from unique values in a column to integer indices.

Finds all unique values in the specified column of the data DataFrame, sorts them, and creates a dictionary mapping each unique value to a 1-based integer index.

Parameters:

• data – The Polars DataFrame containing the column.

• column – The name of the column to map.

Returns:

A dictionary mapping unique values (str or int) to an integer index (starting from 1).

sequifier.preprocess.delete_files(files: list[str] | dict[int, list[str]]) → None

Deletes a list of files from the filesystem.

Parameters:

files – A list of file paths to delete, or a dictionary whose values are lists of file paths to delete.

sequifier.preprocess.extract_sequences(data: DataFrame, schema: Any, seq_length: int, stride_for_split: int, columns: list[str], subsequence_start_mode: str) → DataFrame

Extracts subsequences from a DataFrame of full sequences.

This function takes a DataFrame where each row contains all items for a single sequenceId. It iterates through each sequenceId, extracts all possible subsequences of seq_length using the specified stride_for_split, calculates the starting position of each subsequence within the original sequence, and formats them into a new, long-format DataFrame that conforms to the provided schema.

Parameters:

• data – The input Polars DataFrame, grouped by “sequenceId”.

• schema – The schema for the output long-format DataFrame.

• seq_length – The length of the subsequences to extract.

• stride_for_split – The step size to use when sliding the window to create subsequences.

• columns – A list of the data column names (features) to extract.

• subsequence_start_mode – “distribute” to minimize max subsequence overlap, or “exact”.

Returns:

A new, long-format Polars DataFrame containing the extracted subsequences, matching the provided schema. Includes columns for sequenceId, subsequenceId, startItemPosition, inputCol, and the sequence items (‘0’, ‘1’, …).

sequifier.preprocess.extract_subsequences(in_seq: dict[str, list], seq_length: int, stride_for_split: int, columns: list[str], subsequence_start_mode: str) → dict[str, list[list[Union[float, int]]]]

Extracts subsequences from a dictionary of sequence lists.

This function takes a dictionary in_seq where keys are column names and values are lists of items for a single full sequence. It first pads the sequences with 0s at the beginning if they are shorter than seq_length. Then, it calculates the subsequence start indices using get_subsequence_starts and extracts all subsequences.

Parameters:

• in_seq – A dictionary mapping column names to lists of items (e.g., {‘col_A’: [1, 2, 3, 4, 5], ‘col_B’: [6, 7, 8, 9, 10]}).

• seq_length – The length of the subsequences to extract.

• stride_for_split – The desired step size between subsequences.

• columns – A list of the column names (keys in in_seq) to process.

• subsequence_start_mode – “distribute” to minimize max subsequence overlap, or “exact”.

Returns:

A dictionary mapping column names to a list of lists, where each inner list is a subsequence.

sequifier.preprocess.get_batch_limits(data: DataFrame, n_batches: int) → list[tuple[int, int]]

Calculates row indices to split a DataFrame into batches.

This function divides the DataFrame into n_batches roughly equal chunks. Crucially, it ensures that no sequenceId is split across two different batches. It does this by finding the ideal split points and then adjusting them to the nearest sequenceId boundary.

Parameters:

• data – The DataFrame to split. Must be sorted by “sequenceId”.

• n_batches – The desired number of batches.

Returns:

A list of (start_index, end_index) tuples, where each tuple defines the row indices for a batch.

sequifier.preprocess.get_combined_statistics(n1: int, mean1: float, std1: float, n2: int, mean2: float, std2: float) → tuple[float, float]

Calculates the combined mean and standard deviation of two data subsets.

Uses a stable parallel algorithm (related to Welford’s algorithm) to combine statistics from two subsets without needing the original data.

Parameters:

• n1 – Number of samples in subset 1.

• mean1 – Mean of subset 1.

• std1 – Standard deviation of subset 1.

• n2 – Number of samples in subset 2.

• mean2 – Mean of subset 2.

• std2 – Standard deviation of subset 2.

Returns:

A tuple (combined_mean, combined_std) containing the combined mean and standard deviation of the two subsets.

sequifier.preprocess.get_group_bounds(data_subset: DataFrame, split_ratios: list[float])

Calculates row indices for splitting a sequence into groups.

This function takes a DataFrame data_subset (which typically contains all items for a single sequenceId) and calculates the row indices to split it into multiple groups (e.g., train, val, test) based on the provided split_ratios.

Parameters:

• data_subset – The DataFrame (for a single sequence) to split.

• split_ratios – A list of floats (e.g., [0.8, 0.1, 0.1]) that sum to 1.0, defining the relative sizes of the splits.

Returns:

A list of (start_index, end_index) tuples, one for each proportion, defining the row slices for each group.

sequifier.preprocess.get_subsequence_starts(in_seq_length: int, seq_length: int, stride_for_split: int, subsequence_start_mode: str) → ndarray

Calculates the start indices for extracting subsequences.

This function determines the starting indices for sliding a window of seq_length over an input sequence of in_seq_length. It aims to use stride_for_split, but adjusts the step size slightly to ensure that the windows are distributed as evenly as possible and cover the full sequence from the beginning to the end.

Parameters:

• in_seq_length – The length of the original input sequence.

• seq_length – The length of the subsequences to extract.

• stride_for_split – The desired step size between subsequences.

• subsequence_start_mode – “distribute” to minimize max subsequence overlap, or “exact”.

Returns:

A numpy array of integer start indices for each subsequence.

sequifier.preprocess.insert_top_folder(path: str, folder_name: str) → str

Inserts a directory name into a file path, just before the filename.

Example

insert_top_folder(“a/b/c.txt”, “temp”) returns “a/b/temp/c.txt”

Parameters:

• path – The original file path.

• folder_name – The name of the folder to insert.

Returns:

The new path string with the folder inserted.

sequifier.preprocess.load_precomputed_id_maps(project_root: str, data_columns: list[str] | None, required_maps: list[str] | None = None) → dict[str, dict[Union[str, int], int]]

Loads custom ID maps from configs/id_maps if the folder exists.

Parameters:

• project_root – The path to the project root directory.

• data_columns – Optional list of columns present in the data to validate against the found map files.

• required_maps – Optional list of columns for which a precomputed id_map is required

Returns:

A dictionary mapping column names to their ID maps.

sequifier.preprocess.preprocess(args: Any, args_config: dict[str, Any]) → None

Runs the main data preprocessing pipeline.

This function loads the preprocessing configuration, initializes the Preprocessor class, and executes the preprocessing steps based on the loaded configuration.

Parameters:

• args – An object containing command-line arguments. Expected to have a config_path attribute specifying the path to the YAML configuration file.

• args_config – A dictionary containing additional configuration parameters that may override or supplement the settings loaded from the config file.

sequifier.preprocess.preprocess_batch(project_root: str, data_name_root: str, process_id: int, batch: DataFrame, schema: Any, split_paths: list[str], seq_length: int, stride_by_split: list[int], data_columns: list[str], col_types: dict[str, str], split_ratios: list[float], target_dir: str, write_format: str, batches_per_file: int, subsequence_start_mode: str) → None

Processes a batch of data.

Parameters:

• project_root – The path to the sequifier project directory.

• data_name_root – The root name of the data file.

• process_id – The id of the process.

• batch – The batch of data to process.

• schema – The schema for the preprocessed data.

• split_paths – The paths to the output split files.

• seq_length – The sequence length for the model inputs.

• stride_by_split – A list of step sizes for creating subsequences.

• data_columns – A list of data columns.

• col_types – A dictionary containing the column types.

• split_ratios – A list of floats that define the relative sizes of data splits.

• target_dir – The target directory for temporary files.

• write_format – The file format for the output files.

• batches_per_file – The number of batches to process per file.

• subsequence_start_mode – “distribute” to minimize max subsequence overlap, or “exact”.

sequifier.preprocess.process_and_write_data_pt(data: DataFrame, seq_length: int, path: str, column_types: dict[str, str])

Processes the sequence DataFrame and writes it to a .pt file.

This function takes the long-format sequence DataFrame (data), aggregates it by sequenceId and subsequenceId, and pivots it so that each inputCol becomes its own column containing a list of sequence items. It also extracts the startItemPosition.

It then converts these lists into NumPy arrays, splits them into sequences (all but last item) and targets (all but first item), and converts them to PyTorch tensors along with sequence/subsequence IDs and start positions. The final data tuple (sequences_dict, targets_dict, sequence_ids_tensor, subsequence_ids_tensor, start_item_positions_tensor) is saved to a .pt file using torch.save.

Parameters:

• data – The long-format Polars DataFrame of extracted sequences.

• seq_length – The total sequence length (N). The resulting tensors will have sequence length N-1.

• path – The output file path (e.g., “data/batch_0.pt”).

• column_types – A dictionary mapping column names to their string data types, used to determine the correct torch dtype.

sequifier.make.make(args)

Creates a new sequifier project.

Parameters:

args – The command-line arguments.

class sequifier.hyperparameter_search.HyperparameterSearcher(hyperparameter_search_config)

A class for performing hyperparameter search.

Manages the hyperparameter search process based on a given configuration. This class handles sampling hyperparameters, creating training configurations, launching training subprocesses, and logging results.

__init__(hyperparameter_search_config)

Initializes the HyperparameterSearcher instance.

Parameters:

hyperparameter_search_config (HyperparameterSearchConfig) – An object containing the configuration for the hyperparameter search, loaded via load_hyperparameter_search_config.

hyperparameter_search() → None

Performs the hyperparameter search loop.

It iterates from the start_run number up to the total n_samples, generating a unique seed for each run and calling _create_config_and_run to execute it.

Returns:

None

sequifier.hyperparameter_search.hyperparameter_search(config_path, skip_metadata) → None

Main function for initiating a hyperparameter search process.

This function loads the hyperparameter search configuration, initializes the searcher, and starts the search.

Parameters:

• config_path (str) – Path to the hyperparameter search YAML configuration file.

• skip_metadata (bool) – Flag indicating whether to run the search on unprocessed data.

Returns:

None

sequifier.helpers.configure_determinism(seed: int, strict: bool = False) → None

Enforces deterministic execution for reproducibility.

sequifier.helpers.configure_logger(project_root: str, model_name: str, rank: int | None = 0)

Configures Loguru to replicate the legacy LogFile behavior.

Legacy Behavior Mapping: 1. Console: Only Rank 0 prints high-level info. 2. File 2 (Detailed): Captures ALL logs (equivalent to old level 2). 3. File 3 (Summary): Captures only HIGH importance logs (equivalent to old level 3). 4. Formatting: Files contain raw messages only (no timestamp prefix).

sequifier.helpers.construct_index_maps(id_maps: dict[str, dict[Union[str, int], int]] | None, target_columns_index_map: list[str], map_to_id: bool | None) → dict[str, dict[int, Union[str, int]]]

Constructs reverse index maps (int index to original ID).

This function creates reverse mappings from the integer indices back to the original string or integer identifiers. It only performs this operation if map_to_id is True and id_maps is provided.

A special mapping for index 0 is added: - If original IDs are strings, 0 maps to “unknown”. - If original IDs are strings, 1 maps to “other”. - If original IDs are integers, 0 maps to (minimum original ID) - 2. - If original IDs are integers, 1 maps to (minimum original ID) - 1.

Parameters:

• id_maps – A nested dictionary mapping column names to their respective ID-to-index maps (e.g., {‘col_name’: {‘original_id’: 1, …}}). Expected to be provided if map_to_id is True.

• target_columns_index_map – A list of column names for which to construct the reverse maps.

• map_to_id – A boolean flag. If True, the reverse maps are constructed. If False or None, an empty dictionary is returned.

Returns:

A dictionary where keys are column names from target_columns_index_map and values are the reverse maps (index-to-original-ID). Returns an empty dict if map_to_id is not True.

Raises:

• AssertionError – If map_to_id is True but id_maps is None.

• AssertionError – If the values of a map are not consistently string or integer (excluding the added ‘0’ key).

sequifier.helpers.get_torch_dtype(dtype_str: str) → dtype

Converts a string to a torch dtype, supporting bfloat16 and fp8.

sequifier.helpers.normalize_path(path: str, project_root: str) → str

Normalizes a path to be relative to a project path, then joins them.

This function ensures that a given path is correctly expressed as an absolute path rooted at project_root. It does this by first removing the project_root prefix from path (if it exists) and then joining the result back to project_root.

This is useful for handling paths that might be provided as either relative (e.g., “data/file.txt”) or absolute (e.g., “/abs/path/to/project/data/file.txt”).

Parameters:

• path – The path to normalize.

• project_root – The absolute path to the project’s root directory.

Returns:

A normalized, absolute path.

sequifier.helpers.numpy_to_pytorch(data: DataFrame, column_types: dict[str, torch.dtype], all_columns: list[str], seq_length: int) → dict[str, torch.Tensor]

Converts a long-format Polars DataFrame to a dict of sequence tensors.

This function assumes the input DataFrame data is in a long format where each row represents a sequence for a specific feature. It expects a column named “inputCol” that contains the feature name (e.g., ‘price’, ‘volume’) and other columns representing time steps (e.g., “0”, “1”, …, “L”).

It generates two tensors for each column in all_columns: 1. An “input” tensor (from time steps L down to 1). 2. A “target” tensor (from time steps L-1 down to 0).

Example

For seq_length = 3 and all_columns = [‘price’], it will create: - ‘price’: Tensor from columns [“3”, “2”, “1”] - ‘price_target’: Tensor from columns [“2”, “1”, “0”]

Parameters:

• data – The long-format Polars DataFrame. Must contain “inputCol” and columns named as strings of integers for time steps.

• column_types – A dictionary mapping feature names (from “inputCol”) to their desired torch.dtype.

• all_columns – A list of all feature names (from “inputCol”) to be processed and converted into tensors.

• seq_length – The total sequence length (L). This determines the column names for time steps (e.g., “0” to “L”).

Returns:

A dictionary mapping feature names to their corresponding PyTorch tensors. Target tensors are stored with a _target suffix (e.g., {‘price’: <tensor>, ‘price_target’: <tensor>}).

sequifier.helpers.read_data(path: str, read_format: str, columns: list[str] | None = None) → DataFrame

Reads data from a CSV or Parquet file into a Polars DataFrame.

Parameters:

• path – The file path to read from.

• read_format – The format of the file. Supported formats are “csv” and “parquet”.

• columns – An optional list of column names to read. This argument is only used when read_format is “parquet”.

Returns:

A Polars DataFrame containing the data from the file.

Raises:

ValueError – If read_format is not “csv” or “parquet”.

sequifier.helpers.subset_to_input_columns(data: DataFrame | LazyFrame, input_columns: list[str]) → DataFrame | LazyFrame

Filters a DataFrame to rows where ‘inputCol’ is in a list of column_names.

This function supports both Polars (DataFrame, LazyFrame) and Pandas DataFrames, dispatching to the appropriate filtering method.

• For Polars objects, it uses data.filter(pl.col(“inputCol”).is_in(…)).

• For other objects (presumably Pandas), it builds a numpy boolean mask and filters using data.loc[…].

Note: The type hint only specifies Polars objects, but the implementation includes a fallback path for Pandas-like objects.

Parameters:

• data – The Polars (DataFrame, LazyFrame) or Pandas DataFrame to filter. It must contain a column named “inputCol”.

• input_columns – A list of values. Rows will be kept if their value in “inputCol” is present in this list.

Returns:

A filtered DataFrame or LazyFrame of the same type as the input.

sequifier.helpers.write_data(data: DataFrame, path: str, write_format: str, **kwargs) → None

Writes a Polars (or Pandas) DataFrame to a CSV or Parquet file.

This function detects the type of the input DataFrame. - For Polars DataFrames, it uses .write_csv() or .write_parquet(). - For other DataFrame types (presumably Pandas), it uses .to_csv()

or .to_parquet().

Note: The type hint specifies pl.DataFrame, but the implementation includes a fallback path that suggests compatibility with Pandas DataFrames.

Parameters:

• data – The Polars (or Pandas) DataFrame to write.

• path – The destination file path.

• write_format – The format to write. Supported formats are “csv” and “parquet”.

• **kwargs – Additional keyword arguments passed to the underlying write function (e.g., write_csv for Polars, to_csv for Pandas).

Returns:

None.

Raises:

ValueError – If write_format is not “csv” or “parquet”.

class sequifier.io.yaml.TrainModelDumper(stream, default_style=None, default_flow_style=False, canonical=None, indent=None, width=None, allow_unicode=None, line_break=None, encoding=None, explicit_start=None, explicit_end=None, version=None, tags=None, sort_keys=True)

A custom YAML dumper for TrainModel objects.

This dumper extends the base yaml.Dumper to provide custom serialization for TrainModel and related objects, ensuring a clean and readable YAML output. It also modifies the indentation behavior for better formatting.

increase_indent(flow=False, indentless=False)

Increase the indentation level for the YAML output.

This method overrides the default behavior to force indentation for all block-style collections, improving the readability of the output YAML.

Parameters:

• flow – Whether the context is a flow-style collection.

• indentless – Whether the context is an indentless sequence.

Returns:

The result of the parent class’s increase_indent method, with flow forced to False.

sequifier.io.yaml.represent_dot_dict(dumper, data)

Represents DotDict objects as a simple YAML mapping. The original output showed a ‘dictitems’ attribute. If your DotDict is essentially a dictionary, this will work.

sequifier.io.yaml.represent_numpy_float(dumper, data)

Represents numpy.float64 (and similar numpy floats) as standard YAML floats.

sequifier.io.yaml.represent_numpy_int(dumper, data)

Represents numpy.int64 (and similar numpy integers) as standard YAML integers.

sequifier.io.yaml.represent_sequifier_object(dumper, data)

Represents objects from ‘sequifier.config.train_config’ (like TrainModel, ModelSpecModel, TrainingSpecModel) as a simple YAML mapping, using the object’s __dict__. This effectively removes the !!python/object tag and the explicit ‘__dict__:’, ‘__fields_set__:’ keys.

class sequifier.io.sequifier_dataset_from_folder.SequifierDatasetFromFolder(data_path: str, config: TrainModel)

An efficient PyTorch Dataset that pre-loads all data into RAM.

This is the ideal strategy when the entire dataset split can fit into the system’s memory. It pays a one-time I/O cost at initialization, after which all data access during training is extremely fast (RAM access).

__getitem__(idx: int) → Tuple[Dict[str, Tensor], Dict[str, Tensor], int, int, int]

Retrieves a single sample from the pre-loaded data.

Parameters:

idx – The index of the sample to retrieve.

Returns:

• sequence (dict): Dictionary of feature tensors for the sample.

• targets (dict): Dictionary of target tensors for the sample.

• sequence_id (int): The sequence ID of the sample.

• subsequence_id (int): The subsequence ID within the sequence.

• start_position (int): The starting item position of the subsequence

  within the original full sequence.

Return type:

A tuple containing

__init__(data_path: str, config: TrainModel)

Initializes the dataset by loading all .pt files from the data directory into memory. Each .pt file is expected to contain a tuple: (sequences_dict, targets_dict, sequence_ids_tensor, subsequence_ids_tensor, start_item_positions_tensor).

class sequifier.io.sequifier_dataset_from_file.SequifierDatasetFromFile(data_path: str, config: TrainModel, shuffle: bool = True)

An iterable-style dataset that pre-loads all data into CPU RAM and yields pre-collated batches.

This is the idiomatic PyTorch solution for implementing custom ‘en block’ batching. The __iter__ method handles shuffling and batch slicing, ensuring maximum performance.

__iter__() → Iterator[Tuple[Dict[str, Tensor], Dict[str, Tensor], None, None, None]]

Yields batches of data.

Handles shuffling (if enabled) and slicing data based on distributed rank and worker ID.

Yields:

Iterator[Tuple[Dict[str, torch.Tensor], Dict[str, torch.Tensor], None, None, None]] –

An iterator where each item is a tuple containing:

• data_batch (dict): Dictionary of feature tensors for the batch.

• targets_batch (dict): Dictionary of target tensors for the batch.

• None: Placeholder for sequence_id (not used in this dataset type).

• None: Placeholder for subsequence_id (not used in this dataset type).

• None: Placeholder for start_position (not used in this dataset type).

__len__() → int

Returns the total number of samples in the dataset.

set_epoch(epoch: int)

Allows the training loop to set the epoch for deterministic shuffling.

sequifier.optimizers.optimizers.get_optimizer_class(optimizer_name: str) → Optimizer

Gets the optimizer class from a string.

Parameters:

optimizer_name – The name of the optimizer.

Returns:

The optimizer class.

```