deode.suites package

Empty __init__.py.

Submodules

deode.suites.base module

Ecflow suites base class.

class EcflowNode(name, node_type, parent, ecf_files, variables=None, trigger: EcflowSuiteTriggers | List[EcflowNode] | EcflowNode | None = None, def_status=None, ecf_files_remotely=None, cron=None, limit=None)[source]

Bases: object

A Node class is the abstract base class for Suite, Family and Task.

Every Node instance has a name, and a path relative to a suite. The trigger of a node can either be another EcflowNode, a list of EcflowNode objects or an EcflowSuiteTriggers object.

class EcflowNodeContainer(name, node_type, parent, ecf_files, variables=None, trigger=None, def_status=None, ecf_files_remotely=None, cron=None, limit=None)[source]

Bases: EcflowNode

Ecflow node container.

class EcflowSuite(name, ecf_files, variables=None, dry_run=False, def_status=None, ecf_files_remotely=None)[source]

Bases: EcflowNodeContainer

EcflowSuite.

save_as_defs(def_file)[source]

Save defintion file.

Parameters:: def_file (str) – Name of the definition file.

class EcflowSuiteCron(days_of_week, time)[source]

Bases: object

EcFlow Cron in a suite.

class EcflowSuiteFamily(name, parent, ecf_files, variables=None, trigger=None, def_status=None, ecf_files_remotely=None, cron=None, limit=None)[source]

Bases: EcflowNodeContainer

A family in ecflow.

class EcflowSuiteLimit(limit_name, max_jobs)[source]

Bases: object

Ecflow limit for active jobs in family.

class EcflowSuiteTask(name, parent, config, task_settings, ecf_files, input_template=None, parse=True, variables=None, ecf_micro='%', trigger=None, def_status=None, ecf_files_remotely=None, cron=None)[source]

Bases: EcflowNode

A task in an ecflow suite/family.

class EcflowSuiteTrigger(node, mode='complete')[source]

Bases: object

EcFlow Trigger in a suite.

class EcflowSuiteTriggers(triggers, mode='AND')[source]

Bases: object

Triggers to an ecflow suite.

add_triggers(triggers, mode='AND')[source]

Add triggers.

Parameters:

triggers (EcflowSuiteTriggers) – The triggers
mode (str, optional) – Cat mode. Defaults to “AND”.

static create_string(triggers, mode)[source]

Create the trigger string.

Parameters:

triggers (list) – List of trigger objects
mode (str) – Concatenation type.

Raises:

ValueError – If there are no triggers to be processed
TypeError – If trigger is not an EcflowSuiteTrigger object

Returns:

The trigger string based on trigger objects.

Return type:

str

class SuiteDefinition(config, dry_run=False)[source]

Bases: object

Definition of suite.

save_as_defs(def_file)[source]

Save definition file.

Parameters:: def_file (str) – Name of definition file

deode.suites.clean_old_data module

Ecflow suites.

class DeodeCleaningSuiteDefinition(config, dry_run=False)[source]

Bases: SuiteDefinition

Definition of suite.

deode.suites.deode module

Ecflow suites.

class DeodeSuiteDefinition(config, dry_run=False)[source]

Bases: SuiteDefinition

Definition of suite for the Deode Workflow.

deode.suites.deode_suite_components module

Module to create the different parts of the DEODE ecFlow suite.

class CycleFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the Cycle ecFlow family.

class E923MonthlyFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the E923Monthly ecFlow family.

class ForecastFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the Forecast ecFlow family.

class InitializationFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the Initialization ecFlow family.

class InputDataFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None, external_marsprep_trigger_node=None)[source]

Bases: EcflowSuiteFamily

Class for creating the InputDataFamily ecFlow family.

class InterpolationFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, cycles: Cycles, trigger=None, ecf_files_remotely=None, do_prep: bool = True, dry_run: bool = False)[source]

Bases: EcflowSuiteFamily

Class for creating the Interpolation ecFlow family.

class LBCFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, cycle: Cycle, trigger=None, lbc_family_trigger=None, ecf_files_remotely=None, do_prep: bool = True, dry_run: bool = False)[source]

Bases: EcflowSuiteFamily

Class for creating the ecFlow LBCFamily.

class LBCSubFamilyGenerator(parent, config, task_settings: TaskSettings, input_template, ecf_files, bdint: timedelta, lbc_time_generator: Generator[datetime, None, None], trigger=None, ecf_files_remotely=None, do_prep: bool = True, limit: EcflowSuiteLimit | None = None)[source]

Bases: EcflowSuiteFamily

Class for creating the ecFlow LBCSubFamilyGenerator.

When iterating over this class, it will create a new LBC family for each time in the lbc_time_generator.

class MergeIOFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, n_io_merge: int, nproc_io: int, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the MergeIO ecFlow family.

class PgdInputFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the PGD input ecFlow family.

class PgdNode(node_name, parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily, EcflowSuiteTask

Class for creating the PGD ecFlow family and task.

class PostCycleFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, cycle: Cycle, ecf_out, suite_name, trigger=None, external_cycle_cleaning_trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the PostCycle ecFlow family.

class PrepFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the PrepFamily ecFlow family.

class StaticDataFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, trigger=None, ecf_files_remotely=None)[source]

Bases: EcflowSuiteFamily

Class for creating the StaticData ecFlow family.

class TimeDependentFamily(parent, config, task_settings: TaskSettings, input_template, ecf_files, ecf_out, suite_name, trigger=None, ecf_files_remotely=None, do_prep: bool = True, dry_run: bool = False)[source]

Bases: EcflowSuiteFamily

Class for creating the time dependent part of a DW suite.

property last_node: Return the last family node of self.

deode.suites.discover_suite module

Discover suites.

available_suites(reg)[source]

Create a list of available tasks.

Parameters:: reg (DeodePluginRegistry) – Deode plugin registry
Returns:: Suite objects
Return type:: known_types (list)

discover(package, base)[source]

Discover SuiteDefinition classes.

Plugin classes are discovered in a given namespace package, deriving from a given base class. The base class itself is ignored, as are classes imported from another module (based on cls.__module__). Each discovered class is identified by the class name by changing it to lowercase and stripping the name of the base class, if it appears as a suffix.

Parameters:

package (types.ModuleType) – Namespace package containing the plugins
base (type) – Base class for the plugins

Returns:

type): Discovered plugin classes

Return type:

(dict of str

discover_modules(package, what='plugin')[source]

Discover plugin modules.

Parameters:

package (types.ModuleType) – Namespace package containing the plugins
what (str, optional) – String describing what is supposed to be discovered. Defaults to “plugin”.

Yields:

tuple – str: Name of the imported module types.ModuleType: The imported module

get_suite(name, config)[source]

Create a deode.suites.SuiteDefinition object from configuration.

Parameters:

name (_type_) – _description_
config (_type_) – _description_

Raises:

NotImplementedError – If SuiteDefinition name is not amongst the known SuiteDefinition names.

Returns:

_description_

Return type:

_type_

deode.suites.suite_utils module

Module for utility functions used in the suite definition scripts.

class Cycle(*, day: str, time: str, validtime: str, basetime: str)[source]

Bases: object

Class for representing a cycle.

basetime: str

day: str

time: str

validtime: str

class Cycles(first_cycle: str, last_cycle: str, cycle_length: str)[source]

Bases: object

Class for generating and iterating over Cycle objects.

property current_cycle: Cycle: Return the current Cycle object.

property current_index: int: Return the current cycle index.

cycle_length: str

property end_of_month: bool

Return True if the next cycle is in a different month.

Returns:: True if the next cycle is in a different month
Return type:: bool

first_cycle: str

last_cycle: str

property next_cycle: Cycle

Return the next Cycle object.

Raises:: StopIteration – If there are no more cycles.

bd_generator(bdint: timedelta, mode: str = 'start', do_prep: bool = True) → Generator[Tuple[int, int, int, int], None, None][source]

Generate batch numbers.

Parameters:

bdint – The batch interval.
mode – The mode of the workflow.
do_prep – Whether to do prep.

Yields:

Tuple[int, int, int, int]

lbc_times_generator(basetime: datetime, endtime: datetime, step: timedelta, mode: str = 'start', do_prep: bool = True) → Generator[int | datetime, None, None][source]

Generate lbc times.

For each of them there will be LBC[NN] family.

Parameters:

basetime – The base time.
endtime – The end time.
step – The step size.
mode – The mode of the workflow.
do_prep – Whether to do prep.

Yields:

datetime – The time period for which the next LBC will be computed.

Returns:

The time period for which the last LBC will be computed.

Return type:

datetime