upload_task_templates
This script takes a config of task templates and uploads them to bitfount hub.
The config also contains paths to any custom models and weights that are used in the task templates. If the models or weights have changed since the last upload, the models and weights are uploaded to bitfount hub and the task templates are updated to use the new versions of the models and weights. Finally, there is also the option to archive task templates that are no longer used.
Example dev usage:
```bash
BITFOUNT_ENVIRONMENT=dev python -m bitfount.runners.upload_task_templates \\
task_templates/config-staging-public.yaml -u <username>
Example production usage:
python -m bitfount.runners.upload_task_templates \\
task_templates/config-production-public.yaml -u <username>
The above command can also be run with the --password
flag to provide the password
for the user. If the password is not provided, an authentication flow will be started.
Module
Functions
get_username_to_use
def get_username_to_use(config_file: os.PathLike) ‑> str:
Gets the username to use for the task template owner.
main
def main( config_file: os.PathLike, password: Optional[str] = None, username: Optional[str] = None,) ‑> None:
Uploads models, weights and task templates to bitfount.
Iterates over the models in the config file and uploads them to bitfount if the code hash or weights are different from the latest version on bitfount. Once the models are updated, the task templates are updated to use the latest versions of the models and the task templates are themselves uploaded to bitfount.
Arguments
config_file
: The path to the YAML config.username
: The username for the task template owner. Defaults to None and gets set based on config used.password
: The password for the task template owner.
update_and_upload_task_templates
def update_and_upload_task_templates( config_yaml: dict, username: str, model_names_and_versions: dict, jwt: Optional[ExternallyManagedJWT] = None, keep_slug: bool = False, testing: bool = False,) ‑> Union[list[dict[str, Any]], dict[str, dict[str, Any]]]:
Updates the task templates and uploads them to bitfount hub if they have changed.
Arguments
config_yaml
: The config yaml.username
: The username for the model owner.model_names_and_versions
: A dictionary containing the model names and new version numbers.jwt
: The JWT for the model owner.keep_slug
: Whether to return the task templates with the slug as the key.testing
: Used to indicate whether this function is used in a testing context. If True, the function will not make any requests to the hub.
Raises
ValueError
: If the task template config is invalid.
Classes
BasicTaskTemplateConfig
class BasicTaskTemplateConfig(**data: Any):
Base task template config model for static and runtime type checking.
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError
][pydantic_core.ValidationError] if the input data cannot be
validated to form a valid model.
self
is explicitly positional-only to allow self
as a field name.
Ancestors
- pydantic.main.BaseModel
Subclasses
ModelConfig
class ModelConfig(**data: Any):
Model config model for static and runtime type checking.
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError
][pydantic_core.ValidationError] if the input data cannot be
validated to form a valid model.
self
is explicitly positional-only to allow self
as a field name.
Ancestors
- pydantic.main.BaseModel
Variables
- static
model_computed_fields
- static
model_config
- static
model_fields
- static
model_file : str
- static
private : bool
- static
weights_file : Optional[str]
TaskTemplateConfig
class TaskTemplateConfig(**data: Any):
Model config model for static and runtime type checking.
Create a new model by parsing and validating input data from keyword arguments.
Raises [ValidationError
][pydantic_core.ValidationError] if the input data cannot be
validated to form a valid model.
self
is explicitly positional-only to allow self
as a field name.
Ancestors
- BasicTaskTemplateConfig
- pydantic.main.BaseModel
Variables
- static
description : str
- static
model_computed_fields
- static
model_config
- static
model_fields
- static
sampleDatasetDownloadUrl : Optional[str]
- static
tags : list
- static
template : str
- static
title : str
- static
type : Union[Literal['image-segmentation'], Literal['image-classification'], Literal['object-detection'], Literal['text-classification'], Literal['text-generation'], Literal['tabular-classification'], Literal['tabular-regression'], Literal['tabular-analytics']]