API reference¶

Reference of the app's Python code, generated from its docstrings. For the HTTP API, see API and the Swagger UI of a deployment.

App¶

app ¶

API server entry point.

init_app ¶

init_app() -> FlaskApp

Initialize FOCA application.

Checks that only owners can access tasks, refuses service clients (FOCA's require_no_service_clients()), as tasks are owned by users only, logs the groups of the role user and checks the credentials of the TES backends, removes credentials of callers from tasks stored by earlier versions, and drops the index of earlier versions on the tasks. FOCA validates the config, e.g., that groups are only mapped with a group claim, refuses to start without admins (security.access_control.require_admin), and warns of placeholders in the service info. FOCA loads the plugins of the config (plugins) and checks the backend list once, so that problems stop the app on startup. FOCA's plugin listing is served with the TES API (cf. :py:func:pro_tes.gateway.add_plugin_listing).

Returns:

Type	Description
`FlaskApp`	FOCA application.

Raises:

Type	Description
`RuntimeError`	Access control is not enabled and enforced, service clients are configured, credentials of TES backends are missing, or stored tasks share identifiers (cf. :py:func:`pro_tes.utils.db.check_unique_task_ids`).
`ValidationError`	The config is invalid, e.g., access control is enabled without trusted token issuers, or issuers are trusted without audience.
`ValueError`	The config is invalid (raised by FOCA), e.g., groups are mapped without a group claim, or there are no admins (unless admins are optional, `security.access_control.require_admin`), or a configured index conflicts with an existing index, or plugins can't be loaded, or the config has keys of earlier versions (e.g., `custom.middlewares`; with a hint).

run_app ¶

run_app(app: FlaskApp) -> None

Run FOCA application.

Configuration¶

config_models ¶

Custom app config models.

DEFAULT_API_PATH `module-attribute` ¶

DEFAULT_API_PATH = '/ga4gh/tes/v1'

Path of the TES API, relative to the URL of a backend, as the TES specification has it; that of the API of proTES too.

ConfigModel ¶

Bases: BaseModel

Base of the models of the custom configuration: unknown keys are errors.

So that misspelled or removed keys (e.g., in config overlays) stop the app and the Celery worker on startup, instead of being ignored.

DB ¶

Bases: ConfigModel

DB config for post_task.

Parameters:

Name	Type	Description	Default
`insert_attempts`		Number of attempts to insert a new task in DB.	required

Attributes:

Name	Type	Description
`insert_attempts`	`int`	Number of attempts to insert a new task in DB.

TaskID ¶

Bases: ConfigModel

Task ID config.

Parameters:

Name	Type	Description	Default
`charset`		Characters to use when generating task IDs.	required
`length`		Length of the generated task ID.	required

Attributes:

Name	Type	Description
`charset`	`str`	Characters to use when generating task IDs.
`length`	`int`	Length of the generated task ID.

Polling ¶

Bases: ConfigModel

How tasks are tracked at their TES backends.

Parameters:

Name	Description	Default
`wait`	Seconds between polls of the state of a task.	required
`attempts`	Consecutive failed polls (the count is reset by a successful one) after which the state of the task is set to `UNKNOWN`, and polls back off.	required
`max_wait`	Maximum number of seconds between polls while they back off (values below `wait` count as `wait`); below RabbitMQ's default acknowledgement timeout (`consumer_timeout`, 30 minutes), as are those of `wait`.	required
`max_cancel_time`	Seconds after a cancel at which a task that is still being canceled (`CANCELING`: its backend has neither reported a final state nor that it no longer has the task) is set to `SYSTEM_ERROR`, and no longer tracked; at most a year.	required

Attributes:

Name	Type	Description
`wait`	`Annotated[int, Field(gt=0, lt=CONSUMER_TIMEOUT)]`	Seconds between polls of the state of a task.
`attempts`	`PositiveInt`	Consecutive failed polls (the count is reset by a successful one) after which the state of the task is set to `UNKNOWN`, and polls back off.
`max_wait`	`Annotated[int, Field(gt=0, lt=CONSUMER_TIMEOUT)]`	Maximum number of seconds between polls while they back off (values below `wait` count as `wait`); below RabbitMQ's default acknowledgement timeout (`consumer_timeout`, 30 minutes), as are those of `wait`.
`max_cancel_time`	`Annotated[int, Field(gt=0, le=MAX_SECONDS)]`	Seconds after a cancel at which a task that is still being canceled is set to `SYSTEM_ERROR`, and no longer tracked.

PostTask ¶

Bases: ConfigModel

Configuration for POST /task.

Parameters:

Name	Description	Default
`db`	DB insert behavior.	required
`task_id`	Task ID generation settings.	required
`polling`	Polling behavior.	required
`url_credentials`	What happens to tasks with credentials in the URLs of their inputs or outputs (`https://user:password@host/...`): they are rejected (`reject`), or the credentials are removed (`strip`); credentials are never stored or forwarded.	required
`max_inline_content`	Maximum size of the inline contents of the inputs of a task (`inputs[].content`), together, in bytes (UTF-8); tasks with larger contents are rejected.	required
`max_create_time`	Seconds after which tasks still being created (in state `UNKNOWN`, without identifier at a TES instance; e.g., if proTES stopped while forwarding them) are set to `SYSTEM_ERROR`; longer than creating a task may take, and at most a year.	required
`max_log_size`	Maximum size of the logs of a TES instance that are stored per task (standard output and error of executors and system logs together), in bytes (UTF-8); longer ones are cut, keeping their end. `0` stores none, without getting them. At most 8 MiB, as documents in MongoDB have at most 16 MB.	required
`limits`	Limits of the tasks per user (cf. :py:mod:`pro_tes.limits`); off by default.	required

Attributes:

Name	Type	Description
`db`	`DB`	DB insert behavior.
`task_id`	`TaskID`	Task ID generation settings.
`polling`	`Polling`	Polling behavior.
`url_credentials`	`Literal['reject', 'strip']`	What happens to tasks with credentials in the URLs of their inputs or outputs (`https://user:password@host/...`): they are rejected (`reject`), or the credentials are removed (`strip`); credentials are never stored or forwarded.
`max_inline_content`	`PositiveInt`	Maximum size of the inline contents of the inputs of a task, together, in bytes (UTF-8).
`max_create_time`	`Annotated[int, Field(gt=0, le=MAX_SECONDS)]`	Seconds after which tasks still being created are set to `SYSTEM_ERROR`.
`max_log_size`	`Annotated[int, Field(ge=0, le=8388608)]`	Maximum size of the logs of a TES instance that are stored per task, in bytes (UTF-8).
`limits`	`TaskLimits`	Limits of the tasks per user.

ListTasks ¶

Bases: ConfigModel

Configuration for GET /tasks.

Defaults are those of the TES specification (pages of fewer than 2048 tasks).

Parameters:

Name	Description	Default
`default_page_size`	Number of tasks per page if the request sets no page size.	required
`max_page_size`	Maximum number of tasks per page, at most 2047; larger requested page sizes are reduced to it.	required
`max_page_size_full`	Maximum number of tasks per page of the view `FULL` (with the logs and the contents of inputs); larger page sizes are reduced to it (and to `max_page_size`).	required
`max_page_bytes_full`	Maximum size of the tasks of a page of the view `FULL`, in bytes of the response body; a page ends before the task that would exceed it, but has at least one task.	required

Attributes:

Name	Type	Description
`default_page_size`	`PositiveInt`	Number of tasks per page if the request sets no page size.
`max_page_size`	`PositiveInt`	Maximum number of tasks per page; larger requested page sizes are reduced to it.
`max_page_size_full`	`PositiveInt`	Maximum number of tasks per page of the view `FULL`.
`max_page_bytes_full`	`PositiveInt`	Maximum size of the tasks of a page of the view `FULL`, in bytes of the response body.

check_default_page_size ¶

check_default_page_size() -> ListTasks

Check that the default page size is at most the maximum.

Returns:

Type	Description
`ListTasks`	The validated configuration.

Raises:

Type	Description
`ValueError`	The default page size exceeds the maximum.

Controllers ¶

Bases: ConfigModel

Controller configurations.

Parameters:

Name	Type	Description	Default
`post_task`		Settings for POST /task.	required
`list_tasks`		Settings for GET /tasks.	required

Attributes:

Name	Type	Description
`post_task`	`PostTask`	Settings for POST /task.
`list_tasks`	`ListTasks`	Settings for GET /tasks.

ClientCredentials ¶

Bases: ConfigModel

Client of proTES at its identity provider (OAuth 2.0 client credentials).

proTES gets its own access tokens with the client credentials grant (RFC 6749, section 4.4) for the TES backends with auth: client_credentials. The client ID and secret are read from the environment variables PROTES_CLIENT_ID and PROTES_CLIENT_SECRET, never from the config.

Parameters:

Name	Description	Default
`issuer`	Issuer of the identity provider; its token endpoint is taken from its OpenID Connect discovery document unless `token_endpoint` is set.	required
`token_endpoint`	Token endpoint of the identity provider.	required
`scope`	Scope of the requested tokens (space-separated), if any.	required
`audience_parameter`	Parameter of token requests that names the TES backend that a token is for, so that each backend gets a token of its own, which the others don't accept: `resource` (RFC 8707, resource indicators) or `audience` (as some identity providers name it); the value is the `audience` of the backend's entry, or else its URL. By default, no backend is named, and all backends get the same token.	required
`auth_method`	How the client authenticates at the token endpoint: with HTTP Basic (`client_secret_basic`) or in the request body (`client_secret_post`).	required
`refresh_before`	Seconds before the expiry of a token at which a new one is fetched (at most half its lifetime).	required
`timeout`	Timeout of requests to the identity provider, in seconds.	required

Attributes:

Name	Type	Description
`issuer`	`str \| None`	Issuer of the identity provider; its token endpoint is taken from its OpenID Connect discovery document unless `token_endpoint` is set.
`token_endpoint`	`str \| None`	Token endpoint of the identity provider.
`scope`	`str \| None`	Scope of the requested tokens (space-separated), if any.
`audience_parameter`	`Literal['resource', 'audience'] \| None`	Parameter of token requests that names the TES backend that a token is for (`resource` or `audience`), if any.
`auth_method`	`Literal['client_secret_basic', 'client_secret_post']`	How the client authenticates at the token endpoint: with HTTP Basic (`client_secret_basic`) or in the request body (`client_secret_post`).
`refresh_before`	`PositiveInt`	Seconds before the expiry of a token at which a new one is fetched (at most half its lifetime).
`timeout`	`PositiveInt`	Timeout of requests to the identity provider, in seconds.

TesServiceEntry ¶

Bases: ServiceEntry

TES instance of custom.tes.backends, with the settings of proTES.

Parameters:

Name	Description	Default
`api_path`	Path of the TES API, relative to `url`, for TES instances that serve it elsewhere than the TES specification says; `/` if it is at `url` itself.	required
`audience`	Audience of the access tokens of proTES for the TES instance, requested with `client_credentials.audience_parameter` (e.g., the client ID of the TES instance); by default, its URL.	required
`**data`	Fields of :py:class:`foca.plugins.backends.ServiceEntry`.	required

Attributes:

Name	Type	Description
`api_path`	`str`	Path of the TES API, relative to `url`, without trailing slash (empty if it is at `url` itself).
`audience`	`str \| None`	Audience of the access tokens of proTES for the TES instance, requested with `client_credentials.audience_parameter` (e.g., the client ID of the TES instance); by default, its URL.

check_api_path `classmethod` ¶

check_api_path(value: str) -> str

Check the API path, and remove trailing slashes.

Parameters:

Name	Type	Description	Default
`value`	`str`	API path, as given.	required

Returns:

Type	Description
`str`	The API path.

Raises:

Type	Description
`ValueError`	The path is not absolute, has characters other than unreserved ones (e.g., a query), or segments `.` or `..`.

Tes ¶

Bases: ConfigModel

TES backend configuration.

Parameters:

Name	Description	Default
`backends`	TES backends that tasks may be forwarded to, in order: configured ones (`type: ga4gh-tes`) and service registries whose verified TES services are used (`type: ga4gh-service-registry`; cf. :py:mod:`foca.plugins.backends`; TES instances also with `api_path`, cf. :py:class:`TesServiceEntry`).	required
`url_schemes`	URL schemes allowed for the identity provider of `client_credentials` (those of backends and registries are set per entry, `url_schemes`).	required
`client_credentials`	Client of proTES at its identity provider, for the backends with `auth: client_credentials`.	required
`timeout`	Timeout of requests to TES backends, in seconds (of connecting, at most 5 seconds); requests to create or cancel tasks are not sent again after timeouts, so it should be long enough for slow backends, but short enough that requests to create tasks (which may try several backends) stay below the read timeout of a reverse proxy in front of proTES.	required
`version_cache_ttl`	Seconds that the TES version of a backend, as read from its service info, is cached (per process); also the API path of each discovered backend.	required
`probe_with_token`	Whether the service info of a backend is read for its TES version (and API path, if discovered) with the credentials of the backend (e.g., proTES' access token); by default, it is read without credentials, as service infos are public, so that backends that tasks don't go to don't get them.	required
`max_response_size`	Maximum size of each response of a backend that proTES reads, in bytes (1 MiB to 256 MiB); larger responses fail, with an error in the logs. Bounds the memory of each request (e.g., tasks in the view `FULL`, whose logs can be of any size, which take a Celery worker process about 7 times their size).	required

Attributes:

Name	Type	Description
`backends`	`BackendListConfig`	TES backends that tasks may be forwarded to, in order.
`url_schemes`	`list[Literal['https', 'http']]`	URL schemes allowed for the identity provider of `client_credentials`.
`client_credentials`	`ClientCredentials`	Client of proTES at its identity provider, for the backends with `auth: client_credentials`.
`timeout`	`PositiveInt`	Timeout of requests to TES backends, in seconds.
`version_cache_ttl`	`PositiveInt`	Seconds that the TES version of a backend, as read from its service info, is cached (per process); also the API path of each discovered backend.
`probe_with_token`	`bool`	Whether the service info of a backend is read for its TES version (and API path, if discovered) with the credentials of the backend.
`max_response_size`	`Annotated[int, Field(ge=1048576, le=268435456)]`	Maximum size of each response of a backend that proTES reads, in bytes.

uses_tokens `property` ¶

uses_tokens: bool

Whether a backend has, or discovered ones get, proTES' token.

parse_tes_instances `classmethod` ¶

parse_tes_instances(value: object) -> object

Parse TES instances with the settings of proTES.

FOCA's backend list keeps entries that are instances of its models as they are (also :py:class:foca.plugins.backends.BackendList), so the settings are available from the entries of its candidates. As the field has FOCA's type, model_dump() leaves them out: don't load the config from such a dump.

Parameters:

Name	Type	Description	Default
`value`	`object`	Backend list, as given.	required

Returns:

Type	Description
`object`	The backend list, with entries of TES instances parsed.

check_version_labels ¶

check_version_labels() -> Self

Refuse labels tes.version of versions that proTES doesn't support.

Returns:

Type	Description
`Self`	The configuration.

Raises:

Type	Description
`ValueError`	An entry has such a label.

check_audiences ¶

check_audiences() -> Self

Check the audiences of the entries of TES instances.

Returns:

Type	Description
`Self`	The configuration.

Raises:

Type	Description
`ValueError`	An entry has an audience, but no `auth: client_credentials`, or audiences are not requested (`client_credentials.audience_parameter`); or an audience has surrounding whitespace; or one requested as `resource` is not an absolute URI without query and fragment (RFC 8707, section 2).

check_removed_keys `classmethod` ¶

check_removed_keys(data: object) -> object

Refuse the keys replaced by backends, with the mapping.

Parameters:

Name	Type	Description	Default
`data`	`object`	TES backend configuration, as given.	required

Returns:

Type	Description
`object`	The configuration, unchanged.

Raises:

Type	Description
`ValueError`	The configuration has `service_list` or `discovery`.

check_urls ¶

check_urls() -> Tes

Check the URLs of the identity provider if it is needed.

Returns:

Type	Description
`Tes`	The validated configuration.

Raises:

Type	Description
`ValueError`	A URL of the identity provider is invalid or has a scheme that is not allowed; or a backend has (or discovered backends get) `auth: client_credentials`, but neither the issuer nor the token endpoint is set.

ServiceInfo ¶

Bases: ConfigModel

What the service info tells about the TES backends.

Parameters:

Name	Type	Description	Default
`list_backends`		Whether the service info, which anyone can read, lists the TES backends (`backends`); its field `gateway` stays either way.	required

Attributes:

Name	Type	Description
`list_backends`	`bool`	Whether the service info lists the TES backends.

CustomConfig ¶

Bases: ConfigModel

Custom app configuration.

Parameters:

Name	Description	Default
`controllers`	All controller-related config.	required
`tes`	TES backends and how proTES calls them.	required
`service_info`	What the service info tells about the TES backends.	required
`retention`	Retention of finished tasks (cf. :py:mod:`pro_tes.retention`); off by default.	required
`accounting`	Accounting of the resources that tasks used (cf. :py:mod:`pro_tes.accounting`); off by default.	required

Attributes:

Name	Type	Description
`controllers`	`Controllers`	All controller-related config.
`tes`	`Tes`	TES backends and how proTES calls them.
`service_info`	`ServiceInfo`	What the service info tells about the TES backends.
`retention`	`Retention`	Retention of finished tasks.
`accounting`	`Accounting`	Accounting of the resources that tasks used.

check_removed_keys `classmethod` ¶

check_removed_keys(data: object) -> object

Refuse removed keys, naming what replaces them.

Parameters:

Name	Type	Description	Default
`data`	`object`	Custom config, as given.	required

Returns:

Type	Description
`object`	The custom config, unchanged.

Raises:

Type	Description
`ValueError`	The config has `access_control`, whose only setting (`require_admin`) moved to FOCA's `security.access_control.require_admin`, or `storeLogs`, whose only setting (`execution_trace`) set whether forwards were recorded in `logs[].metadata.forwarded_to`, which is no longer served (cf. the gateway tags of tasks).

split_server_url ¶

split_server_url(url: str, name: str) -> SplitResult

Split the URL of a server, checking its scheme, host and port.

Refuses user info, as FOCA does for backends, since URLs are logged. Error messages leave out the URL, as it may contain credentials, and FOCA shows them.

Parameters:

Name	Type	Description	Default
`url`	`str`	URL, e.g., of a TES backend or an identity provider.	required
`name`	`str`	What the URL is of, for error messages.	required

Returns:

Type	Description
`SplitResult`	Parts of the URL.

Raises:

Type	Description
`ValueError`	The URL is malformed, has no scheme or host, has an invalid port, or has user info.

misc ¶

Miscellaneous utilities.

has_credentials ¶

has_credentials(url: str) -> bool

Check whether a URL has credentials (user information).

Parameters:

Name	Type	Description	Default
`url`	`str`	URL.	required

Returns:

Type	Description
`bool`	Whether the URL has a user name or password (also empty ones, as in
`bool`	`https://@host/path`).

Raises:

Type	Description
`ValueError`	The URL is malformed.

strip_auth ¶

strip_auth(url: str) -> str

Remove basic authentication information from URI, if present.

Expected URI format: scheme://user:password@host:port/path?query#fragment After removal: scheme://host:port/path?query#fragment

Parameters:

Name	Type	Description	Default
`url`	`str`	URI.	required

Returns:

Type	Description
`str`	URI without basic auth.

utc_now ¶

utc_now() -> str

Get the current time in UTC, in RFC 3339 format, as TES requires.

Returns:

Type	Description
`str`	The current time, e.g., `2026-10-22T12:34:56.789012+00:00`.

Controllers¶

server ¶

Controllers for GA4GH TES API endpoints.

CancelTask ¶

CancelTask(id: str, *args: Any, **kwargs: Any) -> dict[str, Any]

Cancel unfinished task.

Parameters:

Name	Type	Description	Default
`id`	`str`	Task identifier.	required
`*args`	`Any`	Variable length argument list.	`()`
`**kwargs`	`Any`	Arbitrary keyword arguments.	`{}`

CreateTask ¶

CreateTask(*args: Any, **kwargs: Any) -> tuple[Any, int, dict[str, str]] | Response

Create task.

The request is read here only: the task processing gets the task, the request path, and the inbound request and caller as plain data (cf. :py:mod:foca.plugins.adapter), not the request. Errors of the plugin engine get the problem details of the error (its status, title, detail and problem type).

Parameters:

Name	Type	Description	Default
`*args`	`Any`	Variable length argument list.	`()`
`**kwargs`	`Any`	Arbitrary keyword arguments, with the task as `body`.	`{}`

Returns:

Type	Description
`tuple[Any, int, dict[str, str]] \| Response`	Response to the client (by default, the task identifier), with its
`tuple[Any, int, dict[str, str]] \| Response`	status and headers; or the problem details of an error of the plugin
`tuple[Any, int, dict[str, str]] \| Response`	engine.

GetTask ¶

GetTask(id: str, *args: Any, **kwargs: Any) -> dict[str, Any]

Get info for individual task.

Parameters:

Name	Type	Description	Default
`id`	`str`	Task identifier.	required
`*args`	`Any`	Variable length argument list.	`()`
`**kwargs`	`Any`	Arbitrary keyword arguments.	`{}`

GetTaskProvenance ¶

GetTaskProvenance(id: str, *args: Any, **kwargs: Any) -> dict[str, Any]

Get the provenance of a task (proTES extension).

Parameters:

Name	Type	Description	Default
`id`	`str`	Task identifier.	required
`*args`	`Any`	Variable length argument list.	`()`
`**kwargs`	`Any`	Arbitrary keyword arguments.	`{}`

Returns:

Type	Description
`dict[str, Any]`	The provenance of the task, with its references expanded (cf.
`dict[str, Any]`	py:meth:`pro_tes.ga4gh.tes.task_runs.TaskRuns.get_task_provenance`).

GetUsage ¶

GetUsage(*args: Any, **kwargs: Any) -> dict[str, Any]

Get the resources that tasks used, per user (proTES extension).

For admins only (enforced by the policies).

Parameters:

Name	Type	Description	Default
`*args`	`Any`	Variable length argument list.	`()`
`**kwargs`	`Any`	Arbitrary keyword arguments: the user (`user_id`) and the time range (`since`, `until`) of the tasks, if any.	`{}`

Returns:

Type	Description
`dict[str, Any]`	Whether resources are recorded (`recording`, cf.
`dict[str, Any]`	`custom.accounting.enabled`), and the totals per user (`users`, cf.
`dict[str, Any]`	py:func:`pro_tes.accounting.usage_totals`).

ListTasks ¶

ListTasks(*args: Any, **kwargs: Any) -> dict[str, Any]

List all available tasks.

Parameters:

Name	Type	Description	Default
`*args`	`Any`	Variable length argument list.	`()`
`**kwargs`	`Any`	Arbitrary keyword arguments.	`{}`

task_runs ¶

Class implementing TES API-server-side controller methods.

TaskRuns ¶

TaskRuns()

Class for TES API server-side controller methods.

Attributes:

Name	Type	Description
`custom_config`	`CustomConfig`	Custom configuration of the app.
`db_client`	`Collection[dict[str, Any]]`	Database collection storing task objects.

Construct object instance.

provenance `cached` `property` ¶

provenance: ProvenanceStore

Store of the provenance of pipeline runs, in the database of the tasks.

Created on first use (by POST /tasks only), so that the other endpoints don't depend on its collections.

Returns:

Type	Description
`ProvenanceStore`	The store (cf. :py:class:`foca.plugins.ProvenanceStore`).

create_task ¶

create_task(task: dict[str, Any] | None, collection: str, inbound: InboundRequest, principal: Principal, request_id: str | None = None, **kwargs: Any) -> tuple[Any, dict[str, str]]

Start task.

The user of the request becomes the owner of the task (cf. :py:mod:pro_tes.ga4gh.tes.ownership). The task is checked, then the plugin pipeline of the request runs (cf. :py:mod:pro_tes.gateway): its request phase may change the task, and its route phase chooses the TES backends, in order, from those of custom.tes.backends that can run the task unchanged (cf. :py:func:pro_tes.ga4gh.tes.backends.eligible_backends). Tasks that fail validation, are rejected by a plugin, or have no backend are not stored.

The task is then stored, and relayed to the backends in turn, after the forward phase of each attempt, until one accepts the task. It is relayed to the next one only if a backend certainly did not accept it, never if it may have (cf. :py:class:pro_tes.ga4gh.tes.backends.TesClient). Tasks that no backend accepts are not kept: they are deleted, with the owner's roles on them. Tasks that a backend may have accepted are kept (as SYSTEM_ERROR).

The provenance of the pipeline run (cf. :py:class:foca.plugins.ProvenanceStore) is stored with the task (provenance): the pipeline, plugins and route when the task is stored, with the attempts once a backend accepted the task (or may have). Runs that fail after they started (e.g., tasks rejected by a plugin, or accepted by no backend) are recorded only if plugins.provenance.record_failed is on.

Parameters:

Name	Type	Description	Default
`task`	`dict[str, Any] \| None`	Task of the request (its JSON body); not changed.	required
`collection`	`str`	Request path of the tasks (e.g., `/ga4gh/tes/v1/tasks`), the resource of the access control policies of the task's owner.	required
`inbound`	`InboundRequest`	Inbound request (cf. :py:func:`foca.plugins.adapter.inbound_from_request`); its payload is replaced by the checked task.	required
`principal`	`Principal`	Caller of the request (cf. :py:func:`foca.plugins.adapter.current_principal`).	required
`request_id`	`str \| None`	Identifier of the request, for logs.	`None`
`**kwargs`	`Any`	Additional keyword arguments passed along with request.	`{}`

Returns:

Type	Description
`Any`	Response to the client: by default, the task identifier; and its
`dict[str, str]`	headers (none by default; plugins may change both in the
`tuple[Any, dict[str, str]]`	`response` phase).

Raises:

Type	Description
`GatewayError`	The pipeline run failed (e.g., a plugin rejected the task, a step failed, the tags choose an invalid pipeline or backends, the request passed proTES before), or no backend accepted the task (in time); in the latter case, the task is not kept.
`BackendOutcomeUnknown`	A TES instance may have accepted the task, but did not confirm it; the task is stored with the state `SYSTEM_ERROR`, and with the TES instance, without remote identifier.
`BadRequest`	The request has no JSON body, or the task is invalid.
`InlineContentTooLarge`	The inline contents of the inputs of the task exceed the limit, also after a plugin changed the task.
`Unauthorized`	The user of the request could not be identified.
`TaskLimitReached`	The user has as many tasks as a limit of `custom.controllers.post_task.limits` allows.

get_task_provenance ¶

get_task_provenance(id: str) -> dict[str, Any]

Get the provenance of a task, with its references expanded.

Only owners of the task and admins may get it (enforced by the policies). The builds of the plugins that ran and default and preset pipelines are stored once, by hash (cf. :py:class:foca.plugins.ProvenanceStore); they are added to the record (builds, pipeline.resolved), unless it has them already (e.g., if they could not be stored). References that can't be resolved are left out.

Parameters:

Name	Type	Description	Default
`id`	`str`	Task identifier.	required

Returns:

Type	Description
`dict[str, Any]`	The provenance of the task.

Raises:

Type	Description
`ProvenanceNotFound`	The task has no provenance (e.g., it was created by an earlier version).
`TaskNotFound`	The task is not available.

list_tasks ¶

list_tasks(collection: str, **kwargs: Any) -> dict[str, Any]

Return list of the tasks of the user of the request.

Parameters:

Name	Type	Description	Default
`collection`	`str`	Request path of the tasks (e.g., `/ga4gh/tes/v1/tasks`), the resource of the access control policies of their owners.	required
`**kwargs`	`Any`	Keyword arguments passed along with request.	`{}`

Returns:

Name	Type	Description
	`dict[str, Any]`	Response object according to TES API schema . Cf.
`https`	`dict[str, Any]`	//github.com/ga4gh/task-execution-schemas/blob/117cd927aa119469ee80cedbd8940c968b81c5d6/openapi/task_execution_service.openapi.yaml;
	`dict[str, Any]`	only the tasks that the user owns are listed (also for admins),
	`dict[str, Any]`	newest first, and `next_page_token` is empty on the last page.

Raises:

Type	Description
`BadRequest`	The page size is not positive, or the page token or the tag filters are invalid.

get_task ¶

get_task(id: str, **kwargs: Any) -> dict[str, Any]

Return detailed information about a task.

Only owners of the task may get it (enforced by the policies).

Parameters:

Name	Type	Description	Default
`id`	`str`	Task identifier.	required
`**kwargs`	`Any`	Additional keyword arguments passed along with request.	`{}`

Returns:

Type	Description
`dict[str, Any]`	Response object according to TES API schema . Cf. https://github.com/ga4gh/task-execution-schemas/blob/117cd927aa119469ee80cedbd8940c968b81c5d6/openapi/task_execution_service.openapi.yaml

Raises:

Type	Description
`TaskNotFound`	The requested task is not available.

cancel_task ¶

cancel_task(id: str, **kwargs: Any) -> dict[str, Any]

Cancel task.

Only owners of the task may cancel it (enforced by the policies). Tasks that have not finished (also those in state UNKNOWN, e.g., not tracked yet) are canceled at their TES instance, and are then in state CANCELING, until their tracking stores the final state that their TES instance reports; finished tasks and tasks being canceled are left as they are, also if their state changed while they were canceled (e.g., a final state stored by their tracking).

Parameters:

Name	Type	Description	Default
`id`	`str`	Task identifier.	required
`**kwargs`	`Any`	Additional keyword arguments passed along with request.	`{}`

Returns:

Type	Description
`dict[str, Any]`	Task identifier.

Raises:

Type	Description
`BackendAuthError`	No credentials for the TES instance of the task are available.
`BackendNotConfigured`	The TES instance of the task is no longer configured.
`BackendOutcomeUnknown`	The TES instance may have canceled the task, but did not confirm it.
`BackendRefused`	The TES instance refused to cancel the task, which has not finished.
`TaskNotCancelable`	The task has not finished, but its identifier at its TES instance is unknown (e.g., it was not forwarded yet).
`TaskNotFound`	The requested task is not available.

tags_filter ¶

tags_filter(keys: list[str], values: list[str]) -> dict[str, Any]

Get the database filter of the tag filters of GET /tasks (TES 1.1).

Keys and values are zipped, in order: a missing or empty value matches any value of the tag, also an empty one. Tag keys are taken literally by $getField (MongoDB 5.0 or later): they may contain dots (e.g., gateway.backend) and start with $, unlike the keys of dotted paths.

Parameters:

Name	Type	Description	Default
`keys`	`list[str]`	Keys of the tags (`tag_key`).	required
`values`	`list[str]`	Values of the tags (`tag_value`).	required

Returns:

Type	Description
`dict[str, Any]`	The filter; empty without keys.

Raises:

Type	Description
`BadRequest`	There are more values than keys, or a key or value has null characters.

served_size ¶

served_size(task: dict[str, Any]) -> int

Get the size of a task in a page of tasks, as served.

Pages are served as JSON indented by 2 spaces, with non-ASCII characters escaped (one byte per character); in a page, each line of a task after the first is indented by 4 more spaces than in the task alone.

Parameters:

Name	Type	Description	Default
`task`	`dict[str, Any]`	Task, as served.	required

Returns:

Type	Description
`int`	Size of the task in the response body, in bytes.

models ¶

TES models.

CustomBaseModel ¶

Bases: BaseModel

Base model that all other models derive from.

TesCancelTaskResponse ¶

Bases: CustomBaseModel

Response to a task cancellation (empty).

TesCreateTaskResponse ¶

Bases: CustomBaseModel

Response to a task creation, with the task identifier.

TesExecutor ¶

Bases: CustomBaseModel

Executor: a command run in a container.

TesExecutorLog ¶

Bases: CustomBaseModel

Logs of an executor run.

TesFileType ¶

Bases: Enum

Type of an input or output: file or directory.

TesInput ¶

Bases: CustomBaseModel

Input file or directory of a task.

TesOutput ¶

Bases: CustomBaseModel

Output file or directory of a task.

TesOutputFileLog ¶

Bases: CustomBaseModel

Log of an output file.

TesResources ¶

Bases: CustomBaseModel

Resources requested by a task.

TesState ¶

Bases: Enum

State of a task.

TesTaskLog ¶

Bases: CustomBaseModel

Logs of a task attempt.

drop_non_string_metadata `classmethod` ¶

drop_non_string_metadata(value: Any) -> Any

Drop metadata values that are not strings.

The TES specification allows only strings; tasks stored by proTES before 0.4.0 have their forwards in forwarded_to (a list, or a nested object), which is no longer served, and backends may log other values.

Parameters:

Name	Type	Description	Default
`value`	`Any`	Metadata, as stored or reported by a backend.	required

Returns:

Type	Description
`Any`	The metadata with string values only.

TesTask ¶

Bases: CustomBaseModel

TES task.

TesListTasksResponse ¶

Bases: CustomBaseModel

Response to a task listing, with a page of tasks.

TesEndpoint ¶

Bases: CustomBaseModel

Create model instance for external TES endpoint.

Parameters:

Name	Description	Default
`host`	Host at which the TES API is served that is processing this request; note that this should include the path information but not the base path defined in the TES API specification; e.g., specify https://my.tes.com/api if the actual API is hosted a https://my.tes.com/api/ga4gh/tes/v1.	required
`base_path`	Override the default path suffix defined in the TES API specification, i.e., `/ga4gh/tes/v1`.	required
`task_id`	Identifier of the task at the external TES endpoint; not set for tasks stored by earlier versions.	required
`tes_version`	TES version that the task was relayed in; not set for tasks stored by earlier versions.	required

Attributes:

Name	Type	Description
`host`	`str`	Host at which the TES API is served that is processing this request; note that this should include the path information but not the base path defined in the TES API specification; e.g., specify https://my.tes.com/api if the actual API is hosted at https://my.tes.com/api/ga4gh/tes/v1.
`base_path`	`str`	Override the default path suffix defined in the TES API specification, i.e., `/ga4gh/tes/v1`.
`task_id`	`str \| None`	Identifier of the task at the external TES endpoint; not set for tasks stored by earlier versions.
`tes_version`	`str \| None`	TES version that the task was relayed in; not set for tasks stored by earlier versions.

DbDocument ¶

Bases: CustomBaseModel

Create model instance for task request database document.

Parameters:

Name	Description	Default
`task`	Information about task.	required
`task_original`	Information about original task.	required
`user_id`	Owner of the task (user identifier); `None` for tasks created by earlier versions, which have no owner.	required
`worker_id`	Identifier of worker task.	required
`tes_endpoint`	External TES endpoint.	required
`provenance`	Provenance of the pipeline run that created the task (cf. :py:meth:`foca.plugins.ProvenanceStore.save`); `None` for tasks created by earlier versions.	required

Attributes:

Name	Type	Description
`task`	`TesTask`	Information about task.
`task_original`	`TesTask`	Information about original task.
`user_id`	`str \| None`	Owner of the task (user identifier); `None` for tasks created by earlier versions, which have no owner.
`worker_id`	`str`	Identifier of worker task.
`tes_endpoint`	`TesEndpoint`	External TES endpoint.
`provenance`	`dict[str, Any] \| None`	Provenance of the pipeline run that created the task (cf. :py:meth:`foca.plugins.ProvenanceStore.save`); `None` for tasks created by earlier versions.

states ¶

TES task states.

States ¶

TES task states, in groups.

Each state of :class:~pro_tes.ga4gh.tes.models.TesState is in exactly one of the base groups UNDEFINED, CANCELABLE, CANCELING and FINISHED; the other groups are unions of these. Tasks are tracked at their backend until their state is in FINISHED.

Plugins and TES instances¶

gateway ¶

proTES as a gateway: the plugin engine of FOCA processes new tasks.

proTES declares itself a gateway of the TES API (:py:data:GATEWAY, passed to Foca(gateway=...) by the app and the Celery worker), so that FOCA runs the plugin pipeline of plugins (cf. :py:mod:foca.plugins) for each new task: plugins may change the task (phase request), choose and order the TES backends (route), and change the request to each backend (forward) and the response to the client (response).

The functions of this module are the app's callbacks of the engine; they get plain data, without the web framework:

:py:func:sanitize_task removes credentials from the URLs of inputs and outputs, and checks the size of their inline contents, after each plugin that changed the task;
:py:func:validate_task checks the task against the TES models of proTES and of py-tes, after each such plugin (invalid output counts as a failed plugin);
:py:func:check_outbound checks that the task of an outbound request can be relayed, after the forward phase of each attempt.

The app serves FOCA's plugin listing (GET /plugins, cf. :py:mod:foca.plugins.api) with the TES API (cf. :py:func:add_plugin_listing), so that clients see what they may choose per task (tags gateway.pipeline and gateway.backends).

TASKS_PATH `module-attribute` ¶

TASKS_PATH = f'{DEFAULT_API_PATH}/tasks'

Path of the outbound requests to create tasks, as plugins see them; sent with the API path of the backend (api_path of its entry) instead of the default one (cf. :py:meth:pro_tes.ga4gh.tes.backends.TesClient.forward).

TES_SPEC `module-attribute` ¶

TES_SPEC = 'tes'

Name of the API specification of the TES API (api.specs).

PLUGINS_PATH `module-attribute` ¶

PLUGINS_PATH = '/ga4gh/tes/v1/plugins'

Path of FOCA's plugin listing, below the base path of the TES API.

RESERVED_HEADERS `module-attribute` ¶

RESERVED_HEADERS = frozenset({'connection', 'content-length', 'host', 'proxy-authorization', 'te', 'trailer', 'transfer-encoding', 'upgrade'})

Headers (lower case) that plugins can't set in requests to backends: they would change where or how requests are sent, e.g., to another virtual host with the backend's credentials.

MAX_PATH_SIZE `module-attribute` ¶

MAX_PATH_SIZE = 4096

Maximum size of each path of a task (cf. :py:func:_task_paths), in bytes of UTF-8, the value of PATH_MAX on Linux; backends may allow less. Other fields (e.g., URLs) are limited by the size of request bodies only.

GATEWAY `module-attribute` ¶

GATEWAY = GatewayApp(api='tes', validate=validate_task, sanitize=sanitize_task, validate_outbound=check_outbound, name='proTES', version=version('pro-tes'), backends=backend_entries)

Settings of proTES as a gateway (cf. :py:class:foca.plugins.runtime.GatewayApp).

handle_url_credentials ¶

handle_url_credentials(payload: dict[str, Any], mode: Literal['reject', 'strip']) -> None

Reject or remove credentials in the URLs of inputs and outputs.

Credentials are user names and passwords (user information, e.g., https://user:password@host/path). Inputs and outputs are kept otherwise, also inputs without URL (with content); malformed ones are left to the validation of the task.

Parameters:

Name	Type	Description	Default
`payload`	`dict[str, Any]`	Task (JSON); changed in place.	required
`mode`	`Literal['reject', 'strip']`	Whether tasks with credentials are rejected (`reject`), or the credentials are removed (`strip`).	required

Raises:

Type	Description
`BadRequest`	A URL has credentials, and `mode` is `reject`; or a URL is malformed.

check_inline_content ¶

check_inline_content(payload: dict[str, Any], limit: int) -> None

Check the size of the inline contents of the inputs of a task.

The limit applies to the content of each input separately (the TES specification recommends a minimum size per field); all contents together are limited by the size of request bodies.

Parameters:

Name	Type	Description	Default
`payload`	`dict[str, Any]`	Task (JSON).	required
`limit`	`int`	Maximum size of the content of an input (`inputs[].content`), in bytes (UTF-8).	required

Raises:

Type	Description
`InlineContentTooLarge`	A content is larger.

sanitize_task ¶

sanitize_task(payload: Any) -> Any

Sanitize a task after a plugin changed it.

Credentials in the URLs of inputs and outputs are removed (whatever custom.controllers.post_task.url_credentials says, which applies to the task of the client), and the inline contents of the inputs are checked.

Parameters:

Name	Type	Description	Default
`payload`	`Any`	Task (JSON); changed in place.	required

Returns:

Type	Description
`Any`	The task.

Raises:

Type	Description
`BadRequest`	A URL is malformed.
`InlineContentTooLarge`	An inline content is larger than `custom.controllers.post_task.max_inline_content`.

tes_task ¶

tes_task(payload: Any) -> tes.Task

Convert a task to the model of py-tes, as it is relayed.

Parameters:

Name	Type	Description	Default
`payload`	`Any`	Task (JSON); not changed.	required

Returns:

Type	Description
`Task`	The task, without the fields that TES servers set
`Task`	(`SERVER_FIELDS`).

Raises:

Type	Description
`TypeError`	The task is not a JSON object, or has fields or values that py-tes doesn't accept.
`ValueError`	The task has values that py-tes doesn't accept, or an output with wildcards without the `path_prefix` that TES 1.1 requires.

validate_task ¶

validate_task(payload: Any) -> None

Validate a task, e.g., after a plugin changed it.

Parameters:

Name	Type	Description	Default
`payload`	`Any`	Task (JSON).	required

Raises:

Type	Description
`ValidationError`	The task is invalid as per the TES model of proTES.
`TypeError`	The task can't be relayed (cf. :py:func:`tes_task`).
`ValueError`	The task can't be relayed, or has a path over `MAX_PATH_SIZE`.

check_outbound ¶

check_outbound(outbound: OutboundRequest) -> None

Check that the task of an outbound request can be relayed.

Requests that fail the check are not sent; the next backend is tried. Plugins can't change the method and path of the request: tasks are created with POST /ga4gh/tes/v1/tasks (TASKS_PATH, whatever the API path of the backend, which the client applies), so that other requests (e.g., for tasks of other users) can't be sent with proTES's credentials.

Parameters:

Name	Type	Description	Default
`outbound`	`OutboundRequest`	Outbound request, after the `forward` phase.	required

Raises:

Type	Description
`BadRequest`	A URL is malformed.
`InlineContentTooLarge`	An inline content is larger than `custom.controllers.post_task.max_inline_content`.
`TypeError`	The task can't be relayed (cf. :py:func:`tes_task`).
`ValueError`	The task can't be relayed, the method or path of the request was changed, its query is not a mapping of names to lists of values, or it has reserved headers (`RESERVED_HEADERS`).

relayed_task ¶

relayed_task(payload: Any, prefix: str, version: TesVersion = CANONICAL) -> tes.Task

Get the task of an outbound request as it is relayed to a backend.

Parameters:

Name	Type	Description	Default
`payload`	`Any`	Task of the outbound request; not changed.	required
`prefix`	`str`	Prefix of the tags reserved for the gateway (`plugins.tags.prefix`).	required
`version`	`TesVersion`	TES version of the backend.	`CANONICAL`

Returns:

Type	Description
`Task`	The task in the TES version of the backend (cf.
`Task`	py:mod:`pro_tes.ga4gh.tes.versions`), without the fields that TES
`Task`	servers set, without tags under the prefix (cf.
`Task`	py:func:`foca.plugins.tags.strip_for_forwarding`) and without
`Task`	credentials in the URLs of inputs and outputs (also those that
`Task`	plugins of the `forward` phase added).

Raises:

Type	Description
`BadRequest`	A URL is malformed.
`TaskIncompatible`	The version can't express the task (e.g., a plugin of the `forward` phase changed it).
`TypeError`	The task can't be relayed (cf. :py:func:`tes_task`).
`ValueError`	The task can't be relayed.

backend_entries ¶

backend_entries(config: Any) -> BackendListConfig

Get the entries of the backend list of proTES.

Parameters:

Name	Type	Description	Default
`config`	`Any`	FOCA configuration of proTES (:py:class:`foca.models.config.Config`).	required

Returns:

Type	Description
`BackendListConfig`	The entries of `custom.tes.backends`.

add_plugin_listing ¶

add_plugin_listing(config: Config) -> None

Serve FOCA's plugin listing with the TES API.

Adds the paths of FOCA's specification of the listing (GET /plugins and GET /plugins/{plugin_id}, :py:data:foca.plugins.api.SPEC_PATH) before the files of the TES API (spec tes of api.specs; first, so that the fields of the TES API, e.g., info, win), so that they are served below its base path (:py:data:PLUGINS_PATH). The access control rules users-list-plugins and users-get-plugins allow them.

Parameters:

Name	Type	Description	Default
`config`	`Config`	FOCA configuration of proTES, before the app is created; changed in place. Nothing is added if the spec `tes` is removed or lists the listing already.	required

backends ¶

TES backends of proTES, and their clients, authenticated as configured.

The backends that tasks may be forwarded to are those of the backend list, custom.tes.backends (cf. :py:mod:foca.plugins.backends), in order: configured ones (type: ga4gh-tes), and those that service registries list as verified (type: ga4gh-service-registry), which FOCA discovers and caches per process. Backends at or below the public URL of proTES (server.public_url) are never used, so that proTES does not forward tasks to itself.

proTES authenticates at TES backends with credentials of its own, never with those of its callers (auth of each entry; discovered backends get that of their registry entry):

client_credentials: an access token of proTES, obtained from its identity provider with the OAuth 2.0 client credentials grant (RFC 6749, section 4.4), with the client ID and secret of the environment variables CLIENT_ID_ENV and CLIENT_SECRET_ENV; tokens are cached, and fetched anew shortly before they expire; with custom.tes.client_credentials.audience_parameter, each backend gets a token for its own audience (cf. :py:func:token_audience), otherwise all get the same;
basic: HTTP Basic credentials of the environment variables named in the config (a transitional mode, for backends that don't accept tokens yet);
none: no credentials.

Credentials are looked up for each request, so that long-running Celery tasks get fresh tokens. Requests get no other credentials: none from a .netrc file of the environment, also not after redirects (cf. :py:class:BackendSession). Requests to discovered backends go through FOCA's guarded HTTP client (:py:class:foca.utils.http.GuardedClient), which checks the address of each connection itself, so that hosts whose addresses change between checks (DNS rebinding) can't lead requests to private addresses (unless allowed, allow_private_addresses of the registry entry); their redirects are not followed. Responses of all backends are read up to custom.tes.max_response_size bytes; larger ones fail, with an error in the logs (:py:class:pro_tes.exceptions.BackendResponseTooLarge). Requests to configured backends time out after custom.tes.timeout seconds without data, and after CONNECT_TIMEOUT seconds while connecting; requests to discovered backends must end within custom.tes.timeout seconds as a whole, and connecting within FOCA's MAX_CONNECT_TIMEOUT (10) seconds (cf. :py:meth:TesClient._request).

Tasks are created and canceled with a single POST request each (which doesn't follow redirects), to the API path of the backend's URL (<url>/ga4gh/tes/v1, or api_path of its entry; for discovered backends, <url> itself if their service info is there, cf. :py:class:ApiPaths), so that backends never get a task twice; tasks are got from the same path. If it is unknown whether the backend processed such a request (e.g., because the response timed out), the request is not sent again (cf. :py:class:pro_tes.exceptions.BackendOutcomeUnknown).

The TES version of a backend (:py:func:backend_version) is the label tes.version of its entry; otherwise, what its service info reports (read without credentials, unless custom.tes.probe_with_token is set), read leniently and cached per process. Backends of unknown versions count as the oldest supported version.

BackendConnectTimeout ¶

Bases: BackendRequestNotSent, ConnectTimeout

Connecting to a discovered backend timed out (not sent).

TokenKey ¶

Bases: NamedTuple

Key of a cached access token of proTES.

Attributes:

Name	Type	Description
`endpoint`	`str`	URL of the token endpoint.
`client_id`	`str`	Client ID of proTES.
`scope`	`str \| None`	Requested scope, if any.
`audience_parameter`	`str \| None`	Parameter of the requested audience, if any.
`audience`	`str \| None`	Requested audience, if any.

TokenProvider ¶

TokenProvider()

Access tokens of proTES (client credentials grant), cached.

Thread-safe; each process (e.g., each Gunicorn or Celery worker) has its own cache. Each token (cf. :py:class:TokenKey) is fetched by one thread at a time, so that threads that need the same new token at the same time wait for the same fetch instead of each fetching one; threads that need other tokens (e.g., for other audiences) don't wait for it, nor do threads that can still use the cached token while it is refreshed.

Create a provider with empty caches.

clear ¶

clear() -> None

Remove all cached tokens and token endpoints.

get_token ¶

get_token(config: Tes, audience: str | None = None) -> str

Get an access token of proTES.

A cached token is used until refresh_before seconds before it expires (at most half its lifetime before), then a new one is fetched (meanwhile, other threads use the cached one); if that fails, the cached token is used until it expires. Tokens are cached per token endpoint, client, scope and audience; expired ones are removed whenever a token is fetched.

Parameters:

Name	Type	Description	Default
`config`	`Tes`	TES backend configuration.	required
`audience`	`str \| None`	Audience of the token, requested with `client_credentials.audience_parameter` (ignored if that is not set); by default, none.	`None`

Returns:

Type	Description
`str`	The access token.

Raises:

Type	Description
`BackendAuthError`	The client ID or secret is not set, or no token was obtained.

TesClient ¶

TesClient(url: str, backend: Backend, entry: ServiceEntry | RegistryEntry, timeout: int | None = None)

Bases: HTTPClient

Client of a TES backend, authenticated as configured for the backend.

Credentials are looked up for each request (cf. :py:meth:TesClient._request_params). Tasks are created, got and canceled with a single request each, to the API path of the backend (api_path of its entry, or the one found for discovered backends; unlike py-tes, which tries other paths too, and so may create a task more than once); redirects of POST requests are not followed.

Attributes:

Name	Type	Description
`backend`		The backend.
`entry`		Entry of the backend in `custom.tes.backends` (the registry entry, for discovered backends).
`api_path`	`str`	Path of the TES API, relative to the URL of the backend; read on first use for discovered backends, which may raise :py:class:`pro_tes.exceptions.BackendRequestNotSent`.
`max_response_size`		Maximum size of each response that is read, in bytes (`custom.tes.max_response_size`); larger responses raise :py:class:`pro_tes.exceptions.BackendResponseTooLarge`.

Create a client.

Parameters:

Name	Type	Description	Default
`url`	`str`	URL of the backend, possibly with a base path.	required
`backend`	`Backend`	The backend.	required
`entry`	`ServiceEntry \| RegistryEntry`	Entry of the backend in `custom.tes.backends` (the registry entry, for discovered backends), with its credentials.	required
`timeout`	`int \| None`	Timeout of requests, in seconds; by default `custom.tes.timeout`.	`None`

api_path `property` ¶

api_path: str

Path of the TES API, relative to the URL of the backend.

That of its entry; for discovered backends, read on first use (cf. :py:class:ApiPaths).

Raises:

Type	Description
`BackendRequestNotSent`	The path of a discovered backend can't be read now (e.g., it is unavailable).
`BackendAuthError`	No credentials are available to read it (`custom.tes.probe_with_token`).

forward ¶

forward(outbound: OutboundRequest, task: Task, connect_timeout: float | None = None) -> tuple[str, BackendResponse]

Relay a task, with a single request, as the plugins prepared it.

The request has the method, query and headers of the outbound request (cf. :py:meth:foca.plugins.PipelineRun.attempts; usually POST /ga4gh/tes/v1/tasks), its path with the API path of the backend for the default one (plugins see the path of the API of proTES, cf. :py:func:pro_tes.gateway.check_outbound), the task as body, and the credentials of the backend (none if the plugins set outbound.auth to none; plugins can't choose other credentials), added after the plugins ran.

Parameters:

Name	Type	Description	Default
`outbound`	`OutboundRequest`	Outbound request, after the `forward` phase.	required
`task`	`Task`	Task to relay (cf. :py:func:`pro_tes.gateway.relayed_task`).	required
`connect_timeout`	`float \| None`	Timeout of connecting, in seconds, if shorter than `CONNECT_TIMEOUT` (e.g., the time left to forward the task).	`None`

Returns:

Type	Description
`str`	Identifier of the task at the backend, and the response of the
`BackendResponse`	backend.

Raises:

Type	Description
`RequestException`	The backend did not accept the task: it was not reached (cf. :py:func:`not_sent`), or it rejected the task (e.g., with a client error).
`BackendAuthError`	No credentials are available.
`BackendOutcomeUnknown`	The backend may have accepted the task, but did not confirm it with its identifier.

get_task ¶

get_task(task_id: str, view: str = 'BASIC') -> tes.Task

Get a task (GET /tasks/{id}), with a single request.

Parameters:

Name	Type	Description	Default
`task_id`	`str`	Identifier of the task at the backend.	required
`view`	`str`	View of the task (`MINIMAL`, `BASIC` or `FULL`).	`'BASIC'`

Returns:

Type	Description
`Task`	The task, without the fields that py-tes doesn't know (cf.
`Task`	py:func:`known_fields`).

Raises:

Type	Description
`RequestException`	The request failed, or the backend answered with an error.
`BackendAuthError`	No credentials are available.

service_info ¶

service_info(auth: str | None = None) -> Any

Get the service info (GET /service-info), with a single request.

Parameters:

Name	Type	Description	Default
`auth`	`str \| None`	Name of the credentials to use (e.g., `none`, as service infos are public); `None` for those of the backend.	`None`

Returns:

Type	Description
`Any`	The service info, as the backend serves it.

Raises:

Type	Description
`RequestException`	The request failed, or the backend answered with an error.
`BackendAuthError`	No credentials are available.
`ValueError`	The response is no JSON.

get_service_info ¶

get_service_info(url: str, auth: str | None = None) -> requests.Response

Send a single GET request for a service info, whatever the answer.

Parameters:

Name	Type	Description	Default
`url`	`str`	URL of the service info.	required
`auth`	`str \| None`	Name of the credentials to use; `None` for those of the backend.	`None`

Returns:

Type	Description
`Response`	The response, with any status.

Raises:

Type	Description
`RequestException`	The request failed.
`BackendAuthError`	No credentials are available.

cancel_task ¶

cancel_task(task_id: str) -> None

Cancel a task (POST /tasks/{id}:cancel), with a single request.

Parameters:

Name	Type	Description	Default
`task_id`	`str`	Identifier of the task at the backend.	required

Raises:

Type	Description
`RequestException`	The backend was not reached (cf. :py:func:`not_sent`), or it rejected the request.
`BackendAuthError`	No credentials are available.
`BackendOutcomeUnknown`	The backend may have canceled the task, but did not confirm it.

BackendSession ¶

BackendSession()

Bases: Session

Session of requests to configured backends, without .netrc.

requests takes credentials from a .netrc file of the environment (~/.netrc, or the file of NETRC) for requests without credentials of their own, instead of proTES' access token, and again after each redirect. This session never does: requests get only the credentials of their backend. Other settings of the environment, such as proxies and CA certificates, still apply. Credentials are dropped on redirects to other hosts or ports, and from https to http, as requests does.

Create a session.

rebuild_auth ¶

rebuild_auth(prepared_request: PreparedRequest, response: Response) -> None

Drop the credentials of a redirect to another host; add none.

Parameters:

Name	Type	Description	Default
`prepared_request`	`PreparedRequest`	The request of the redirect.	required
`response`	`Response`	The response with the redirect.	required

BackendVersions ¶

BackendVersions()

TES versions of backends, from their service info, cached.

Thread-safe (a lost update only reads a service info once more, and failures keep what other threads have read meanwhile); each process (e.g., each Gunicorn or Celery worker) has its own cache.

Create an empty cache.

clear ¶

clear() -> None

Remove all cached versions.

get ¶

get(backend: Backend, probe: bool = True) -> TesVersion

Get the TES version of a backend.

Never raises: tasks can be relayed to backends of unknown versions.

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	Backend.	required
`probe`	`bool`	Whether to read the service info of the backend if no version is cached, or the cached one expired.	`True`

Returns:

Type	Description
`TesVersion`	The version of the label `tes.version` of the backend, else that
`TesVersion`	of its service info, else (e.g., if the backend is unavailable)
`TesVersion`	the oldest supported version.

was_read ¶

was_read(backend: Backend) -> bool

Tell whether reading the service info of a backend was tried.

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	Backend.	required

Returns:

Type	Description
`bool`	Whether this process has tried to read it (and still knows).

confirmed ¶

confirmed(backend: Backend) -> bool

Tell whether a backend stated its TES version or answered at all.

Doesn't read its service info.

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	Backend.	required

Returns:

Type	Description
`bool`	Whether the version comes from the label of the backend or an
`bool`	answer of the backend, not from having no answer so far.

parameters ¶

parameters() -> list[str]

Get the backend parameters that backends support, as far as known.

Reads neither service infos nor registries, as clients without credentials get them through the service info of proTES.

Returns:

Type	Description
`list[str]`	The keys that the service infos read so far list, sorted.

ApiPaths ¶

ApiPaths()

API paths of discovered backends, read from their URLs, cached.

Registries list the URL of each TES service, which FOCA lists without the API path of the TES specification (/ga4gh/tes/v1, cf. :py:mod:foca.plugins.backends). The TES API of a discovered backend is at that URL if its service info is there (GET <url>/service-info answers with that of a TES service), else at the default API path below it, if the answer settles that: 404 or 410, or another service info. Refusals (401 or 403) mean the default path too, as before 0.6.0, but are read again after PROBE_RETRY seconds, and don't replace a path settled before. Other answers (e.g., redirects, 429, server errors, pages that aren't JSON) and failed requests leave the path unknown: the path read before is kept for PROBE_RETRY seconds, if any was settled; otherwise, no request is sent (no path is guessed, as requests to a wrong path could take a task for gone, 404). The service info is read as for TES versions (cf. :py:meth:BackendVersions._probe): without credentials, unless custom.tes.probe_with_token is set, within PROBE_TIMEOUT seconds. Settled paths are cached per process for custom.tes.version_cache_ttl seconds. Thread-safe (a lost update only reads a service info once more).

Create an empty cache.

clear ¶

clear() -> None

Remove all cached paths.

get ¶

get(client: TesClient, timeout: float = PROBE_TIMEOUT) -> str

Get the API path of a discovered backend.

Parameters:

Name	Type	Description	Default
`client`	`TesClient`	Client of the backend, at its URL.	required
`timeout`	`float`	Time to read the service info, in seconds, as a whole (e.g., at most the time left to forward a task).	`PROBE_TIMEOUT`

Returns:

Type	Description
`str`	The API path, relative to the URL: empty if the TES API is at
`str`	the URL, else `DEFAULT_API_PATH`.

Raises:

Type	Description
`BackendRequestNotSent`	The path is unknown (e.g., the service info timed out, or the backend answered with a server error), and none was settled before.
`BackendAuthError`	No credentials are available (with `custom.tes.probe_with_token`).

get_env ¶

get_env(name: str) -> str

Get the value of an environment variable with credentials.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the environment variable.	required

Returns:

Type	Description
`str`	The value of the environment variable.

Raises:

Type	Description
`BackendAuthError`	The variable is not set, or empty.

token_audience ¶

token_audience(backend: Backend, entry: ServiceEntry | RegistryEntry, config: Tes) -> str | None

Get the audience of the access tokens of proTES for a backend.

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	The backend.	required
`entry`	`ServiceEntry \| RegistryEntry`	Entry of the backend in `custom.tes.backends` (the registry entry, for discovered backends).	required
`config`	`Tes`	TES backend configuration.	required

Returns:

Type	Description
`str \| None`	The `audience` of the backend's entry, or else its URL (for
`str \| None`	discovered backends, always their URL); `None` if audiences are not
`str \| None`	requested (`client_credentials.audience_parameter`).

get_backend_list ¶

get_backend_list(public_url: str | None = None) -> BackendList

Get the backend list of proTES.

The entries are validated again (a small cost); the backends discovered in registries are cached per process. Must be called in an app context.

Parameters:

Name	Type	Description	Default
`public_url`	`str \| None`	Public URL of proTES; by default, `server.public_url` of the app's config.	`None`

Returns:

Type	Description
`BackendList`	The backends of `custom.tes.backends`.

get_backends ¶

get_backends(refresh: bool = False) -> list[Backend]

Get the TES backends that tasks may be forwarded to, in order.

Must be called in an app context.

Parameters:

Name	Type	Description	Default
`refresh`	`bool`	Whether to read the registries now (rate-limited).	`False`

Returns:

Type	Description
`list[Backend]`	The configured backends, and those discovered in registries, in the
`list[Backend]`	order of `custom.tes.backends` (cf.
`list[Backend]`	py:meth:`foca.plugins.backends.BackendList.expand`).

get_backend ¶

get_backend(url: str) -> tuple[Backend, ServiceEntry | RegistryEntry]

Get a TES backend by its URL, and the entry it comes from.

If no backend has the URL, the registries are read again (rate-limited), as another process (e.g., the one that forwarded a task that the Celery worker tracks) may have discovered the backend already. Must be called in an app context.

Parameters:

Name	Type	Description	Default
`url`	`str`	Base URL of the backend; URLs are compared normalized (e.g., ignoring trailing slashes and the case of the host).	required

Returns:

Type	Description
`Backend`	The backend, and its entry of `custom.tes.backends` (the registry
`ServiceEntry \| RegistryEntry`	entry, for discovered backends), e.g., with its credentials.

Raises:

Type	Description
`BackendNotConfigured`	No backend has the URL.

service_info_fields ¶

service_info_fields() -> dict[str, Any]

Get the service info fields with the TES backends of proTES.

Served with the service info (service_info.runtime_fields). Must be called in an app context.

Returns:

Type	Description
`dict[str, Any]`	The TES backends (`backends`): identifier, URL, source, group and
`dict[str, Any]`	labels of each, without credentials (cf.
`dict[str, Any]`	py:meth:`foca.plugins.backends.BackendList.describe`), unless
`dict[str, Any]`	`custom.service_info.list_backends` is false; the backend parameters
`dict[str, Any]`	they support, as far as known (cf.
`dict[str, Any]`	py:meth:`BackendVersions.parameters`; the config of the service info
`dict[str, Any]`	wins); and what clients may choose per task (`gateway`: the path of
`dict[str, Any]`	the plugin listing, whether clients may choose pipelines and backends,
`dict[str, Any]`	and the capabilities of the plugins, without plugin identifiers; cf.
`dict[str, Any]`	py:func:`foca.plugins.api.service_info_fields`), unless the
`dict[str, Any]`	plugins can't be loaded (then the app doesn't start, with the
`dict[str, Any]`	error of the plugins).

quote_id ¶

quote_id(task_id: str) -> str

Quote the identifier of a task at a backend, as a path segment.

Parameters:

Name	Type	Description	Default
`task_id`	`str`	Identifier of the task, as the backend set it.	required

Returns:

Type	Description
`str`	The identifier, with all characters but unreserved ones
`str`	percent-encoded, also the dots of the identifiers `.` and `..`, so
`str`	that it can't change the path of requests.

not_sent ¶

not_sent(exc: RequestException) -> bool

Check whether a failed request was certainly not received.

Only errors that prevent the connection from being established count: the connection was refused, the host was not resolved, the connection timed out, or the server's certificate was not accepted; and errors of requests that could not be prepared (e.g., invalid headers). For all others (e.g., read timeouts, or connections lost after the request was sent), the server may have received, and processed, the request.

Parameters:

Name	Type	Description	Default
`exc`	`RequestException`	Error of the request.	required

Returns:

Type	Description
`bool`	Whether the server certainly did not receive the request.

known_fields ¶

known_fields(value: Any, model: type = tes.Task) -> Any

Drop the fields of a task that the models of py-tes don't know.

py-tes raises on such fields, but backends may add them to what they serve (e.g., TESP: author). Applies to the task and the objects nested in it (_NESTED_MODELS); free-form maps (e.g., tags) are kept.

Parameters:

Name	Type	Description	Default
`value`	`Any`	Task as served by a backend (JSON), or an object or a list of objects nested in it; other values are returned as they are.	required
`model`	`type`	Model of py-tes of the value.	`Task`

Returns:

Type	Description
`Any`	The value, without the unknown fields; not changed in place.

response_headers ¶

response_headers(response: Response) -> Headers

Get the headers of a response, for the plugins of the response phase.

Headers that FOCA doesn't accept (e.g., with NUL characters) are left out: the backend accepted the task, so they must not fail the attempt.

Parameters:

Name	Type	Description	Default
`response`	`Response`	Response of a backend.	required

Returns:

Type	Description
`Headers`	The headers of the response.

backend_request ¶

backend_request(method: str, url: str, **kwargs: Any) -> requests.Response

Send a request to a configured backend (cf. :py:class:BackendSession).

Parameters:

Name	Type	Description	Default
`method`	`str`	HTTP method.	required
`url`	`str`	URL of the request.	required
`**kwargs`	`Any`	Other arguments of `requests.request()`.	`{}`

Returns:

Type	Description
`Response`	The response.

skip_redirect_body ¶

skip_redirect_body(response: Response, **_: Any) -> None

Leave the body of a redirect unread (hook of requests).

requests reads the body of each redirect that it follows as a whole otherwise, and keeps it in the history of the response; the connection is closed instead.

Parameters:

Name	Type	Description	Default
`response`	`Response`	A response of a backend.	required
`**_`	`Any`	Other arguments of the hook.	`{}`

read_limited ¶

read_limited(response: Response, max_size: int, timeout: float | None = None) -> requests.Response

Read the body of a streamed response, up to a size.

Bodies of answers with other statuses than 2xx are cut off at the size instead, as their status decides how they are handled.

Parameters:

Name	Type	Description	Default
`response`	`Response`	Streamed response.	required
`max_size`	`int`	Maximum size of the body (decoded), in bytes.	required
`timeout`	`float \| None`	Maximum time to read it, in seconds, if any; as a whole, as the timeout of requests only counts time without data.	`None`

Returns:

Type	Description
`Response`	The response, with its body.

Raises:

Type	Description
`BackendResponseTooLarge`	The body of a 2xx answer is larger (logged as an error).
`ConnectionError`	The body can't be read.
`Timeout`	Reading the body takes longer.

too_large ¶

too_large(url: str, max_size: int) -> BackendResponseTooLarge

Log that a response is too large, and get the error to raise.

Parameters:

Name	Type	Description	Default
`url`	`str`	URL of the request.	required
`max_size`	`int`	Maximum size of responses, in bytes.	required

Returns:

Type	Description
`BackendResponseTooLarge`	The error.

as_response ¶

as_response(guarded: HttpResponse, url: str) -> requests.Response

Get a response of the guarded client as a response of requests.

Parameters:

Name	Type	Description	Default
`guarded`	`HttpResponse`	Response of the guarded client, with its whole body.	required
`url`	`str`	URL of the request.	required

Returns:

Type	Description
`Response`	The response; its encoding is that of its media type, as `requests`
`Response`	finds it.

create_client ¶

create_client(host: str, base_path: str = '') -> TesClient

Create a client of a configured TES backend.

Parameters:

Name	Type	Description	Default
`host`	`str`	Base URL of the backend, as configured.	required
`base_path`	`str`	Path to append to `host`, if any.	`''`

Returns:

Type	Description
`TesClient`	Client of the backend.

Raises:

Type	Description
`BackendNotConfigured`	The backend is not configured.
`ValueError`	The URL is invalid.

client_for ¶

client_for(backend: Backend, timeout: int | None = None) -> TesClient

Create a client of a candidate backend, e.g., of a pipeline run.

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	Backend, as expanded from `custom.tes.backends` (cf. :py:func:`get_backends`).	required
`timeout`	`int \| None`	Timeout of requests, in seconds; by default `custom.tes.timeout`.	`None`

Returns:

Type	Description
`TesClient`	Client of the backend, with the credentials of its entry.

Raises:

Type	Description
`BackendNotConfigured`	No entry of `custom.tes.backends` yields the backend (e.g., the list changed meanwhile).

reported_version ¶

reported_version(service_info: Any) -> TesVersion | None

Get the TES version that a service info reports, read leniently.

Backends report it in different ways (e.g., tesk-api 1.1.0: artifact tes, version 1.0, while TESK's chart has it as artifact TES and a number; Poiesis: v1.1.0), or report none (Funnel 0.11.14: an empty type.version).

Parameters:

Name	Type	Description	Default
`service_info`	`Any`	Service info, as served by a backend.	required

Returns:

Type	Description
`TesVersion \| None`	The version; `None` if it reports none that proTES supports.

is_tes ¶

is_tes(service_info: Any) -> bool

Tell whether a service info is that of a TES service.

Parameters:

Name	Type	Description	Default
`service_info`	`Any`	Service info, as served by a backend.	required

Returns:

Type	Description
`bool`	Whether its type is `org.ga4gh` `tes` (ignoring case), of any version.

token_hint ¶

token_hint(backend: Backend) -> str

Get the hint at custom.tes.probe_with_token, for refused reads.

The setting doesn't give backends without credentials any.

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	Backend whose service info was refused without credentials.	required

Returns:

Type	Description
`str`	The hint (to append to others), or nothing for `auth: none`.

backend_version ¶

backend_version(backend: Backend, probe: bool = True) -> TesVersion

Get the TES version of a backend (cf. :py:class:BackendVersions).

Parameters:

Name	Type	Description	Default
`backend`	`Backend`	Backend.	required
`probe`	`bool`	Whether its service info may be read now.	`True`

Returns:

Type	Description
`TesVersion`	The version; the oldest supported one if it is unknown.

eligible_backends ¶

eligible_backends(task: Any, backends: list[Backend]) -> list[Backend]

Get the backends that can run a task unchanged; plugins can use it.

Tasks that every supported version can express can run anywhere, and need no service info. For other tasks, no further service info is read once PROBE_TIMEOUT seconds have passed, so that unavailable backends don't keep the request waiting; the remaining backends count with what is cached (cf. :py:meth:BackendVersions.confirmed). Backends never read come first, so that an unavailable one can't keep them from being read.

Parameters:

Name	Type	Description	Default
`task`	`Any`	Task of the canonical TES version (JSON).	required
`backends`	`list[Backend]`	Backends, in order.	required

Returns:

Type	Description
`list[Backend]`	The backends that can run the task, in order.

forward_reason ¶

forward_reason(exc: Exception) -> str

Get the reason code of a failed attempt to relay a task.

Parameters:

Name	Type	Description	Default
`exc`	`Exception`	Error of the attempt (cf. :py:meth:`TesClient.forward`).	required

Returns:

Type	Description
`str`	Reason code (cf. :py:meth:`foca.plugins.Attempt.failed`), e.g.,
`str`	`no_credentials`, `address_not_allowed`, `connect_timeout`,
`str`	`connect_error`, `http_403`, or that of an unknown outcome (cf.
`str`	py:class:`pro_tes.exceptions.BackendOutcomeUnknown`).

Task tracking¶

celery_worker ¶

Celery worker entry point.

create_celery_app ¶

create_celery_app(foca: Foca) -> Celery

Create the Celery app, reporting stored tasks that share identifiers.

Parameters:

Name	Type	Description	Default
`foca`	`Foca`	FOCA instance.	required

Returns:

Type	Description
`Celery`	Celery app.

Raises:

Type	Description
`RuntimeError`	Stored tasks share identifiers (cf. :py:func:`pro_tes.utils.db.check_unique_task_ids`).
`ValueError`	A configured index conflicts with an existing index.

check_worker_backends ¶

check_worker_backends(**kwargs: Any) -> None

Check the credentials of the TES backends when the worker starts.

As the app does (cf. :py:func:pro_tes.security.check_backends), so that a worker without them doesn't start, instead of failing to track tasks.

Parameters:

Name	Type	Description	Default
`**kwargs`	`Any`	Arguments of the signal; ignored.	`{}`

Raises:

Type	Description
`SystemExit`	Credentials of TES backends are missing (Celery logs other exceptions of signal handlers, and starts anyway).

track_task_progress ¶

Celery tasks that track tasks at their TES backends.

A task is tracked by a chain of Celery tasks: each polls the backend once and, while the task is unfinished, schedules the next poll (countdown), so that no worker process is blocked for the runtime of a task. Messages are acknowledged after the poll (acks_late), so that the broker delivers them again if a worker is lost; polls that fail because MongoDB is unavailable are retried, and those that fail because the broker is go back to it.

Each task is tracked by one chain of polls only: each poll has an identifier, and the stored task holds the identifier of its current poll (tracking.poll) and the time it was claimed (tracking.polled_at). Each poll passes the tracking on to the next one, which gets a new identifier; polls whose identifier is no longer current do nothing, which ends their chain. A poll may also take over from the poll that scheduled it (previous), if that one didn't pass the tracking on (e.g., because its worker was lost). When a worker starts, and then periodically, the tracking of stored unfinished tasks that haven't been polled for long is resumed (:py:func:task__resume_tracking), e.g., of tasks whose polls the broker lost.

backoff ¶

backoff(failures: int) -> int

Get the number of seconds until the next poll.

Polls are polling.wait seconds apart until polling.attempts consecutive polls have failed; then the interval doubles with each failed poll, up to polling.max_wait seconds (values of max_wait below wait count as wait).

Parameters:

Name	Type	Description	Default
`failures`	`int`	Number of consecutive failed polls.	required

Returns:

Type	Description
`int`	Seconds until the next poll.

interval ¶

interval(doublings: int) -> int

Get polling.wait seconds, doubled a number of times.

At most polling.max_wait seconds, and at least polling.wait.

Parameters:

Name	Type	Description	Default
`doublings`	`int`	Number of times the interval is doubled.	required

Returns:

Type	Description
`int`	Seconds.

task__track_task_progress ¶

task__track_task_progress(self: Any, worker_id: str, remote_host: str, remote_base_path: str, remote_task_id: str, poll: str | None = None, previous: str | None = None, failures: int = 0) -> None

Poll a task at its TES backend once, and schedule the next poll.

The state is stored if it changed (a stored cancelation only once the backend has finished the task); a finished task is got once more in the view FULL, unless the poll is delivered again, its final state, logs and outputs are stored together, and tracking ends. Failed polls are retried, also if the backend is not (or no longer) configured or discovered; after polling.attempts consecutive failures, the state is set to UNKNOWN (the task may still run at its backend), and polls back off (see :py:func:backoff) until the backend answers again. A task being canceled stays CANCELING meanwhile, unless its backend answers that it no longer has the task (404): then it is CANCELED, and tracking ends; after polling.max_cancel_time seconds, it is SYSTEM_ERROR, and tracking ends too. Polls that fail because MongoDB is unavailable are retried, with the same arguments, after up to polling.max_wait seconds, until it is available again; polls that fail because the broker is unavailable go back to it.

proTES authenticates at the TES service as configured for it (see :py:mod:pro_tes.ga4gh.tes.backends), with fresh credentials for each request.

Parameters:

Name	Type	Description	Default
`self`	`Any`	Celery task instance (bound task).	required
`worker_id`	`str`	Worker identifier.	required
`remote_host`	`str`	Host at which the TES API is served that is processing this request; note that this should include the path information but not the base path defined in the TES API specification; e.g., specify https://my.tes.com/api if the actual API is hosted at https://my.tes.com/api/ga4gh/tes/v1.	required
`remote_base_path`	`str`	Override the default path suffix defined in the TES API specification, i.e., `/ga4gh/tes/v1`.	required
`remote_task_id`	`str`	task run identifier on remote TES service.	required
`poll`	`str \| None`	Identifier of the poll; `None` for the first poll after the task was created.	`None`
`previous`	`str \| None`	Identifier of the poll that scheduled this one.	`None`
`failures`	`int`	Number of consecutive failed polls before this one.	`0`

Raises:

Type	Description
`ValueError`	The task finished, but has no logs.

task__resume_tracking ¶

task__resume_tracking() -> int

Resume the tracking of stored unfinished tasks not polled for long.

That is, of tasks with an unfinished state and a remote identifier (tes_endpoint.task_id; tasks stored by earlier versions may not have one) that no poll claimed in the last STALE_FACTOR times polling.max_wait seconds, or ever. Each gets a new poll, which takes over from the stored one, so that a chain that still tracks the task ends. Run when a worker starts, and then periodically (cf. :py:func:resume_tracking_on_start); idempotent. Tasks whose creation was cut off are set to SYSTEM_ERROR first (cf. :py:func:_fail_cut_off), as are unfinished tasks whose tracking ended (cf. :py:func:_fail_ended).

Returns:

Type	Description
`int`	Number of tasks whose tracking is resumed.

resume_tracking_periodically ¶

resume_tracking_periodically(stop: Event, interval: float) -> None

Resume the tracking of stale tasks every interval seconds.

Parameters:

Name	Type	Description	Default
`stop`	`Event`	Event that ends the loop.	required
`interval`	`float`	Seconds between runs of :py:func:`task__resume_tracking`.	required

resume_tracking_on_start ¶

resume_tracking_on_start(**kwargs: Any) -> Thread

Resume the tracking of stale tasks when a worker starts, and periodically.

Tasks may have lost their tracking, e.g., if the broker lost its messages. The tracking is resumed by a Celery task (cf. :py:func:task__resume_tracking), which runs in the app context, when the worker starts and then every polling.max_wait seconds, from a daemon thread of the worker's main process.

Parameters:

Name	Type	Description	Default
`**kwargs`	`Any`	Arguments of the signal; ignored.	`{}`

Returns:

Type	Description
`Thread`	Thread that resumes the tracking periodically.

Database¶

db ¶

Utility class for common MongoDB operations.

DbDocumentConnector ¶

DbDocumentConnector(collection: Collection[dict[str, Any]], worker_id: str)

MongoDB connector to a given proTES database document.

Parameters:

Name	Type	Description	Default
`collection`	`Collection[dict[str, Any]]`	Database collection.	required
`worker_id`	`str`	Celery task identifier.	required

Attributes:

Name	Type	Description
`collection`	`Collection[dict[str, Any]]`	Database collection.
`worker_id`	`str`	Celery task identifier.

Construct object instance.

get_document ¶

get_document(projection: Mapping[str, Any] | None = None) -> DbDocument

Get document associated with task.

Parameters:

Name	Type	Description	Default
`projection`	`Mapping[str, Any] \| None`	A projection object indicating which fields of the document to return. By default, all fields except the MongoDB identifier `_id` and the provenance are returned.	`None`

Returns:

Type	Description
`DbDocument`	Instance of `pro_wes.ga4gh.wes.models.DbDocument` associated with the task.

Raise

ValueError: Returned document does not conform to schema.

update_task_state ¶

update_task_state(state: str = 'UNKNOWN', if_state: str | Sequence[str] | None = None, fields: Mapping[str, Any] | None = None) -> bool

Update task status.

Parameters:

Name	Type	Description	Default
`state`	`str`	New task status; one of `pro_tes.ga4gh.tes.models.TesState`.	`'UNKNOWN'`
`if_state`	`str \| Sequence[str] \| None`	Update the status only if it is this one, or one of these (atomically); in any case if `None`.	`None`
`fields`	`Mapping[str, Any] \| None`	Other fields of the document to set with the status (by dotted path), e.g., the time of a cancel.	`None`

Returns:

Type	Description
`bool`	Whether the status was updated.

Raises:

Type	Description
`ValueError`	Invalid state passed.

upsert_fields_in_root_object ¶

upsert_fields_in_root_object(root: str, projection: Mapping[str, Any] | None = None, **kwargs: object) -> DbDocument

Insert or update fields in(to) the same root (object) field.

Parameters:

Name	Type	Description	Default
`root`	`str`	Root field name.	required
`projection`	`Mapping[str, Any] \| None`	A projection object indicating which fields of the document to return. By default, all fields except the MongoDB identifier `_id` and the provenance are returned.	`None`
`**kwargs`	`object`	Key-value pairs of fields to insert/update.	`{}`

Returns:

Type	Description
`DbDocument`	Inserted/updated document, or `None` if database operation failed.

remove_stored_credentials ¶

remove_stored_credentials(collection: Collection[dict[str, Any]]) -> None

Remove credentials of callers from tasks stored by earlier versions.

That is, their HTTP Basic credentials (basic_auth), and credentials in the URLs of inputs and outputs (also of the tasks as received, and of the outputs in their logs). Run on startup; idempotent.

Parameters:

Name	Type	Description	Default
`collection`	`Collection[dict[str, Any]]`	Database collection of the tasks.	required

find_duplicate_task_ids ¶

find_duplicate_task_ids(collection: Collection[dict[str, Any]]) -> dict[str | None, int]

Find identifiers that stored tasks share.

Such tasks prevent the unique index on task identifiers (task.id, cf. db in config.yaml); earlier versions didn't have it.

Parameters:

Name	Type	Description	Default
`collection`	`Collection[dict[str, Any]]`	Database collection of the tasks.	required

Returns:

Type	Description
`dict[str \| None, int]`	Number of tasks per shared identifier, in the order of the
`dict[str \| None, int]`	identifiers; `None` for tasks without identifier (more than one
`dict[str \| None, int]`	prevents the index, too), and the representation of identifiers
`dict[str \| None, int]`	that aren't strings (e.g., lists).

drop_legacy_task_index ¶

drop_legacy_task_index(collection: Collection[dict[str, Any]]) -> None

Drop the index of earlier versions on task_id and worker_id.

The fields of the index didn't identify tasks (task identifiers are stored as task.id), so it never rejected a duplicate identifier; it is replaced by a unique index on task.id and an index on worker_id (cf. db in config.yaml). Run on startup; idempotent, also if several instances start at the same time.

Parameters:

Name	Type	Description	Default
`collection`	`Collection[dict[str, Any]]`	Database collection of the tasks.	required

check_unique_task_ids ¶

check_unique_task_ids(foca: Foca, exc: ValueError) -> None

Report stored tasks that share identifiers, if indexes failed.

FOCA can't create the unique index on task identifiers (task.id, cf. db in config.yaml) while stored tasks share identifiers, e.g., tasks stored by earlier versions, which didn't have the index. Nothing is changed: operators resolve the duplicates (cf. docs/upgrading.md).

Parameters:

Name	Type	Description	Default
`foca`	`Foca`	FOCA instance whose app (or Celery app) could not be created.	required
`exc`	`ValueError`	Error of FOCA while creating the app.	required

Raises:

Type	Description
`RuntimeError`	The error is a duplicate key error, and stored tasks share identifiers; it names them (up to `MAX_REPORTED_DUPLICATES`), and the number of tasks for each.

models ¶

Class to convert py-tes to proTES TES task model.

TaskModelConverter ¶

TaskModelConverter(task: Task)

Convert py-tes to proTES TES task model.

Convert :class:tes.models.Task to :class:pro_tes.ga4gh.tes.models.TesTask

Parameters:

Name	Type	Description	Default
`task`	`Task`	Instance of :class:`tes.models.Task`	required

Attributes:

Name	Type	Description
`task`	`Task`	Instance of :class:`tes.models.Task`

Construct object instance.

convert_task ¶

convert_task() -> TesTask

Convert py-tes to proTES TES task.

Returns:

Type	Description
`TesTask`	Instance of :class:`pro_tes.ga4gh.tes.models.TesTask`

convert_state ¶

convert_state() -> TesState

Convert py-tes to proTES TES task state.

Returns:

Type	Description
`TesState`	Instance of :class:`pro_tes.ga4gh.tes.models.TesState`

convert_inputs ¶

convert_inputs() -> list[TesInput] | None

Convert py-tes to proTES TES task inputs.

Returns:

Type	Description
`list[TesInput] \| None`	List of :class:`pro_tes.ga4gh.tes.models.TesInput`

convert_outputs ¶

convert_outputs() -> list[TesOutput] | None

Convert py-tes to proTES TES task outputs.

Returns:

Type	Description
`list[TesOutput] \| None`	List of :class:`pro_tes.ga4gh.tes.models.TesOutput`

convert_resources ¶

convert_resources() -> TesResources | None

Convert py-tes to proTES TES task resources.

Returns:

Type	Description
`TesResources \| None`	Instance of :class:`pro_tes.ga4gh.tes.models.TesResources`

convert_executors ¶

convert_executors() -> list[TesExecutor]

Convert py-tes to proTES TES task executors.

Returns:

Type	Description
`list[TesExecutor]`	List of :class:`pro_tes.ga4gh.tes.models.TesExecutor`

convert_logs ¶

convert_logs() -> list[TesTaskLog] | None

Convert py-tes to proTES TES task logs.

Returns:

Type	Description
`list[TesTaskLog] \| None`	List of :class:`pro_tes.ga4gh.tes.models.TesTaskLog`

convert_executor_logs `staticmethod` ¶

convert_executor_logs(log: TaskLog) -> list[TesExecutorLog]

Convert py-tes to proTES TES task executor logs.

Parameters:

Name	Type	Description	Default
`log`	`TaskLog`	py-tes task log.	required

Returns:

Type	Description
`list[TesExecutorLog]`	List of :class:`pro_tes.ga4gh.tes.models.TesExecutorLog`

convert_output_file_logs `staticmethod` ¶

convert_output_file_logs(log: TaskLog) -> list[TesOutputFileLog]

Convert py-tes to proTES TES task output file logs.

Parameters:

Name	Type	Description	Default
`log`	`TaskLog`	py-tes task log.	required

Returns:

Type	Description
`list[TesOutputFileLog]`	List of :class:`pro_tes.ga4gh.tes.models.TesOutputFileLog`

convert_time `staticmethod` ¶

convert_time(timestamp: datetime | None) -> str | None

Convert py-tes to proTES TES task time.

Parameters:

Name	Type	Description	Default
`timestamp`	`datetime \| None`	Time to convert.	required

Returns:

Type	Description
`str \| None`	String representation of time in RFC3339 format, or `None` if `timestamp` is `None`.

Errors¶

exceptions ¶

proTES exceptions.

Errors of the plugin engine of FOCA (cf. :py:mod:foca.plugins.errors) get the problem details of the gateway error, with their problem type, from the controller of POST /tasks; the mapping below is their fallback.

BackendAuthError ¶

Bases: RuntimeError

Raised when proTES has no credentials for a TES backend.

E.g., because its identity provider issued no access token.

BackendRequestNotSent ¶

Bases: ConnectionError

Raised when a request to a discovered TES backend was certainly not sent.

E.g., because connecting failed or timed out (cf. :py:class:foca.utils.http.RequestNotSent), or because the API path of the backend is unknown, so that the request was not made (cf. :py:class:pro_tes.ga4gh.tes.backends.ApiPaths; reading the service info for it may have reached the backend). A connection error, so that proTES handles it as it handles unreachable backends.

BackendAddressNotAllowed ¶

Bases: BackendRequestNotSent

Raised when the host of a discovered TES backend is not public.

Or can't be resolved: FOCA's guarded HTTP client checks the address of each connection (cf. :py:class:foca.utils.http.GuardedClient). No request is sent.

BackendResponseTooLarge ¶

Bases: RequestException

Raised when a TES backend answers with more than proTES reads.

That is, with more than custom.tes.max_response_size bytes; the rest of the response is not read. The request was sent, and may have been processed.

BackendOutcomeUnknown ¶

BackendOutcomeUnknown(message: str, reason: str = 'outcome_unknown')

Bases: RuntimeError

Raised when a TES backend may have processed a request, or not.

E.g., because the response timed out, the connection was lost after the request was sent, or the backend answered with a server error. Such requests (to create or cancel tasks) are not sent again, neither to the backend nor to other backends, so that tasks are never created twice.

Attributes:

Name	Type	Description
`reason`		Reason code, e.g., `read_timeout`, `http_502` or `response_too_large` (cf. :py:class:`foca.plugins.OutcomeUnknown`).

Create the error.

Parameters:

Name	Type	Description	Default
`message`	`str`	Description of the error.	required
`reason`	`str`	Reason code.	`'outcome_unknown'`

BackendRefused ¶

Bases: RuntimeError

Raised when a TES backend refused a request, which it didn't process.

BackendNotConfigured ¶

Bases: ValueError

Raised when a TES backend is neither configured nor discovered.

TaskIncompatible ¶

Bases: ValueError

A task can't be expressed in the TES version of a backend.

NoEligibleBackend ¶

NoEligibleBackend(detail: str, run: Any = None)

Bases: GatewayError

No backend can run the task unchanged (e.g., only TES 1.0 backends).

The task is not stored: the client has to change it, or wait for a TES instance that supports it.

Create the error.

Parameters:

Name	Type	Description	Default
`detail`	`str`	Description of the problem, for the client.	required
`run`	`Any`	Run of the request.	`None`

IdsUnavailableProblem ¶

Bases: PyMongoError

Raised when task identifier is unavailable.

InlineContentTooLarge ¶

Bases: RequestEntityTooLarge

Raised when the inline contents of the inputs of a task are too large.

Cf. custom.controllers.post_task.max_inline_content.

TaskNotFound ¶

Bases: NotFound

Raised when task with given task identifier was not found.

ProvenanceNotFound ¶

Bases: NotFound

Raised when a task has no provenance.

E.g., because it was created by an earlier version of proTES.

InvalidUsageTime ¶

Bases: BadRequest

Raised when a time of a request for the usage of tasks is invalid.

Cf. the query parameters since and until of GET /usage.

TaskNotCancelable ¶

Bases: Conflict

Raised when a task has not finished, but can't be canceled.

E.g., because it was not forwarded yet, so that its identifier at its TES instance is unknown.

API reference¶

App¶

app ¶

init_app ¶

run_app ¶

Configuration¶

config_models ¶

DEFAULT_API_PATH module-attribute ¶

ConfigModel ¶

DB ¶

TaskID ¶

Polling ¶

PostTask ¶

ListTasks ¶

check_default_page_size ¶

Controllers ¶

ClientCredentials ¶

TesServiceEntry ¶

check_api_path classmethod ¶

Tes ¶

uses_tokens property ¶

parse_tes_instances classmethod ¶

check_version_labels ¶

check_audiences ¶

check_removed_keys classmethod ¶

check_urls ¶

ServiceInfo ¶

CustomConfig ¶

check_removed_keys classmethod ¶

split_server_url ¶

misc ¶

has_credentials ¶

strip_auth ¶

utc_now ¶

Controllers¶

server ¶

CancelTask ¶

CreateTask ¶

GetTask ¶

GetTaskProvenance ¶

GetUsage ¶

ListTasks ¶

task_runs ¶

TaskRuns ¶

provenance cached property ¶

create_task ¶

get_task_provenance ¶

list_tasks ¶

get_task ¶

cancel_task ¶

tags_filter ¶

served_size ¶

models ¶

CustomBaseModel ¶

TesCancelTaskResponse ¶

TesCreateTaskResponse ¶

TesExecutor ¶

TesExecutorLog ¶

TesFileType ¶

TesInput ¶

TesOutput ¶

TesOutputFileLog ¶

TesResources ¶

TesState ¶

TesTaskLog ¶

drop_non_string_metadata classmethod ¶

TesTask ¶

TesListTasksResponse ¶

TesEndpoint ¶

DbDocument ¶

states ¶

States ¶

Plugins and TES instances¶

gateway ¶

TASKS_PATH module-attribute ¶

TES_SPEC module-attribute ¶

PLUGINS_PATH module-attribute ¶

RESERVED_HEADERS module-attribute ¶

MAX_PATH_SIZE module-attribute ¶

GATEWAY module-attribute ¶

DEFAULT_API_PATH `module-attribute` ¶

check_api_path `classmethod` ¶

uses_tokens `property` ¶

parse_tes_instances `classmethod` ¶

check_removed_keys `classmethod` ¶

check_removed_keys `classmethod` ¶

provenance `cached` `property` ¶

drop_non_string_metadata `classmethod` ¶

TASKS_PATH `module-attribute` ¶

TES_SPEC `module-attribute` ¶

PLUGINS_PATH `module-attribute` ¶

RESERVED_HEADERS `module-attribute` ¶

MAX_PATH_SIZE `module-attribute` ¶

GATEWAY `module-attribute` ¶

api_path `property` ¶