Skip to main content

modeller_runner

Utility functions for running modellers from configs.

Module​

Functions​

run_modeller​

def run_modeller(    modeller: _Modeller,    pod_identifiers: Iterable[str],    require_all_pods: bool = False,    project_id: Optional[str] = None,    run_on_new_data_only: bool = False,    batched_execution: Optional[bool] = None,    test_run: bool = False,    force_rerun_failed_files: bool = True,) ‑> Optional[Any]:

Runs the modeller.

Run the modeller, submitting tasks to the pods and waiting for the results.

Arguments

  • modeller: The Modeller instance being used to manage the task.
  • pod_identifiers: The group of pod identifiers to run the task against.
  • require_all_pods: Require all pod identifiers specified to accept the task request to complete task execution.
  • project_id: Project ID the task belongs to. Defaults to None.
  • run_on_new_data_only: Whether to run the task on new datapoints only. Defaults to False.
  • batched_execution: Whether to run the task in batched mode. Defaults to False.
  • test_run: Whether to run the modeller in test mode. False if batched_execution is False. Defaults to False.
  • force_rerun_failed_files: If True, forces a rerun on files that the task previously failed on. If False, the task will skip files that have previously failed. Note: This option can only be enabled if both enable_batch_resilience and individual_file_retry_enabled are True. Defaults to True.

Raises

  • PodResponseError: If require_all_pods is true and at least one pod identifier specified rejects or fails to respond to a task request.

run_modeller_async​

async def run_modeller_async(    modeller: _Modeller,    pod_identifiers: Iterable[str],    require_all_pods: bool = False,    project_id: Optional[str] = None,    run_on_new_data_only: bool = False,    batched_execution: Optional[bool] = None,    test_run: bool = False,    force_rerun_failed_files: bool = True,) ‑> Optional[Any]:

Runs the modeller.

Run the modeller, submitting tasks to the pods and waiting for the results.

Arguments

  • modeller: The Modeller instance being used to manage the task.
  • pod_identifiers: The group of pod identifiers to run the task against.
  • require_all_pods: Require all pod identifiers specified to accept the task request to complete task execution.
  • project_id: The project ID the task belongs to.
  • run_on_new_data_only: Whether to run the task on new datapoints only. Defaults to False.
  • batched_execution: Whether to run the task in batched mode. Defaults to False.
  • test_run: Whether to run the modeller in test mode. False if batched_execution is False. Defaults to False.
  • force_rerun_failed_files: If True, forces a rerun on files that the task previously failed on. If False, the task will skip files that have previously failed. Note: This option can only be enabled if both enable_batch_resilience and individual_file_retry_enabled are True. Defaults to True.

Raises

  • PodResponseError: If require_all_pods is true and at least one pod identifier specified rejects or fails to respond to a task request.

setup_modeller​

def setup_modeller(    pod_identifiers: list[str],    task_details: TaskConfig,    bitfount_hub: BitfountHub,    ms_config: MessageServiceConfig,    identity_verification_method: Union[str, IdentityVerificationMethod] = IdentityVerificationMethod.OIDC_DEVICE_CODE,    private_key_file: Optional[Path] = None,    idp_url: Optional[str] = None,    project_id: Optional[str] = None,) ‑> _Modeller:

Creates a modeller.

Arguments

  • pod_identifiers: The pod identifiers of the pods to be used in the task.
  • task_details: The task details as a TaskConfig instance.
  • bitfount_hub: The BitfountHub instance.
  • ms_config: The message service settings as a MessageServiceConfig instance.
  • identity_verification_method: The identity verification method to use.
  • private_key_file: The path to the private key used by this modeller.
  • idp_url: URL of the modeller's identity provider.
  • project_id: The project ID the task belongs to.

Returns The created Modeller.

setup_modeller_from_config​

def setup_modeller_from_config(modeller_config: ModellerConfig) ‑> tuple:

Creates a modeller from a loaded config mapping.

Arguments

  • modeller_config: The modeller configuration.

Returns 7-tuple of:

  • the created Modeller
  • the list of pod identifiers to run the task against
  • the project_id (if within a project)
  • the run_on_new_data_only boolean flag
  • the batched_execution boolean flag
  • the test_run boolean flag
  • the force_rerun_failed_files boolean flag

setup_modeller_from_config_file​

def setup_modeller_from_config_file(    path_to_config_yaml: Union[str, PathLike],    pod_identifier: Optional[str] = None,    project_id: Optional[str] = None,    template_params: Optional[dict[str, Any]] = None,) ‑> tuple:

Creates a modeller from a YAML config file.

Arguments

  • path_to_config_yaml: The path to the config file
  • pod_identifier: Optional pod identifier to use instead of the one in the config
  • project_id: Optional project ID to use instead of the one in the config
  • template_params: Optional dictionary mapping template parameter names to values

Returns A tuple of the created Modeller and the list of pod identifiers to run

Raises

  • BitfountVersionError: If the task configuration is incompatible with the current SDK