organize_calculation_files#

Workflow for organizing calculation files.

Calculation files for each specified tick are merged into a single csv. The individual tick calculation files are also compressed into a tar.xz archive. After verifying that the file exists in the archive, the individual tick calculation file is removed.

Flows

`run_flow`	Main organize calculation files flow.
`run_flow_compress_files`	Organize calculation files subflow for compressing files.
`run_flow_merge_files`	Organize calculation files subflow for merging files.
`run_flow_remove_files`	Organize calculation files subflow for removing files.

run_flow(context: ContextConfig, series: SeriesConfig, parameters: ParametersConfig) → None[source]#

Main organize calculation files flow.

Calls the following subflows, in order:

run_flow_merge_files()
run_flow_compress_files()
run_flow_remove_files()

run_flow_compress_files(context: ContextConfig, series: SeriesConfig, parameters: ParametersConfig) → None[source]#

Organize calculation files subflow for compressing files.

Iterate through conditions and seeds to combine and compress individual ticks into a .tar.xz archive. If the archive exists and the specified tick is not in the archive, the tick is appended. If the archive exists and specified tick exists in the archive, the tick is skipped.

run_flow_merge_files(context: ContextConfig, series: SeriesConfig, parameters: ParametersConfig) → None[source]#

Organize calculation files subflow for merging files.

Iterate through conditions and seeds to merge contents of individual ticks into a single csv. If merged csv exists and the specified tick does not exist in the csv, the tick is appended. If the merged csv exists and specified tick exists in the csv, the tick is skipped.

run_flow_remove_files(context: ContextConfig, series: SeriesConfig, parameters: ParametersConfig) → None[source]#

Organize calculation files subflow for removing files.

Iterate through conditions and seeds to remove individual ticks if the tick exists in the corresponding .tar.xz archive.

Configs

`ContextConfig`	Context configuration for organize calculation files flow.
`ParametersConfig`	Parameter configuration for organize calculation files flow.
`SeriesConfig`	Series configuration for organize calculation files flow.

class ContextConfig[source]#

Context configuration for organize calculation files flow.

working_location: str#: Location for input and output files (local path or S3 bucket).

class ParametersConfig[source]#

Parameter configuration for organize calculation files flow.

suffix: str#: Calculation type suffix.

ticks: list[int]#: List of ticks to run flow on.

region: str | None = None#: Subcellular region name.

class SeriesConfig[source]#

Series configuration for organize calculation files flow.

name: str#: Name of the simulation series.

seeds: list[int]#: List of series random seeds.

conditions: list[dict]#: List of series condition dictionaries (must include unique condition “key”).