Pipelines API Reference
Kubeflow Pipelines API
Version: 2.0.5
This file contains REST API specification for Kubeflow Pipelines. The file is autogenerated from the swagger definition.
Summary
Tag: AuthService
| Operation | Description |
|---|---|
| GET /apis/v2beta1/auth |
Tag: ExperimentService
| Operation | Description |
|---|---|
| GET /apis/v2beta1/experiments | Finds all experiments. Supports pagination, and sorting on certain fields. |
| POST /apis/v2beta1/experiments | Creates a new experiment. |
| GET /apis/v2beta1/experiments/{experiment_id} | Finds a specific experiment by ID. |
| DELETE /apis/v2beta1/experiments/{experiment_id} | Deletes an experiment without deleting the experiment's runs and recurring runs. To avoid unexpected behaviors, delete an experiment's runs and recurring runs before deleting the experiment. |
| POST /apis/v2beta1/experiments/{experiment_id}:archive | Archives an experiment and the experiment's runs and recurring runs. |
| POST /apis/v2beta1/experiments/{experiment_id}:unarchive | Restores an archived experiment. The experiment's archived runs and recurring runs will stay archived. |
Tag: HealthzService
| Operation | Description |
|---|---|
| GET /apis/v2beta1/healthz | Get healthz data. |
Tag: PipelineService
| Operation | Description |
|---|---|
| GET /apis/v2beta1/pipelines | Finds all pipelines within a namespace. |
| POST /apis/v2beta1/pipelines | Creates a pipeline. |
| POST /apis/v2beta1/pipelines/create | Creates a new pipeline and a new pipeline version in a single transaction. |
| GET /apis/v2beta1/pipelines/names/{name} | Finds a specific pipeline by name and namespace. |
| GET /apis/v2beta1/pipelines/{pipeline_id} | Finds a specific pipeline by ID. |
| DELETE /apis/v2beta1/pipelines/{pipeline_id} | Deletes an empty pipeline by ID. Returns error if the pipeline has pipeline versions. |
| GET /apis/v2beta1/pipelines/{pipeline_id}/versions | Lists all pipeline versions of a given pipeline ID. |
| POST /apis/v2beta1/pipelines/{pipeline_id}/versions | Adds a pipeline version to the specified pipeline ID. |
| GET /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id} | Gets a pipeline version by pipeline version ID and pipeline ID. |
| DELETE /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id} | Deletes a specific pipeline version by pipeline version ID and pipeline ID. |
Tag: PipelineUploadService
| Operation | Description |
|---|---|
| POST /apis/v2beta1/pipelines/upload | |
| POST /apis/v2beta1/pipelines/upload_version |
Tag: RecurringRunService
| Operation | Description |
|---|---|
| GET /apis/v2beta1/recurringruns | Finds all recurring runs given experiment and namespace. If experiment ID is not specified, find all recurring runs across all experiments. |
| POST /apis/v2beta1/recurringruns | Creates a new recurring run in an experiment, given the experiment ID. |
| GET /apis/v2beta1/recurringruns/{recurring_run_id} | Finds a specific recurring run by ID. |
| DELETE /apis/v2beta1/recurringruns/{recurring_run_id} | Deletes a recurring run. |
| POST /apis/v2beta1/recurringruns/{recurring_run_id}:disable | Stops a recurring run and all its associated runs. The recurring run is not deleted. |
| POST /apis/v2beta1/recurringruns/{recurring_run_id}:enable | Restarts a recurring run that was previously stopped. All runs associated with the recurring run will continue. |
Tag: ReportService
| Operation | Description |
|---|---|
| POST /apis/v2beta1/scheduledworkflows | |
| POST /apis/v2beta1/workflows |
Tag: RunService
| Operation | Description |
|---|---|
| GET /apis/v2beta1/runs | Finds all runs in an experiment given by experiment ID. If experiment id is not specified, finds all runs across all experiments. |
| POST /apis/v2beta1/runs | Creates a new run in an experiment specified by experiment ID. If experiment ID is not specified, the run is created in the default experiment. |
| GET /apis/v2beta1/runs/{run_id} | Finds a specific run by ID. |
| DELETE /apis/v2beta1/runs/{run_id} | Deletes a run in an experiment given by run ID and experiment ID. |
| GET /apis/v2beta1/runs/{run_id}/nodes/{node_id}/artifacts/{artifact_name}:read | Finds artifact data in a run. |
| POST /apis/v2beta1/runs/{run_id}:archive | Archives a run in an experiment given by run ID and experiment ID. |
| POST /apis/v2beta1/runs/{run_id}:retry | Re-initiates a failed or terminated run. |
| POST /apis/v2beta1/runs/{run_id}:terminate | Terminates an active run. |
| POST /apis/v2beta1/runs/{run_id}:unarchive | Restores an archived run in an experiment given by run ID and experiment ID. |
Tag: VisualizationService
| Operation | Description |
|---|---|
| POST /apis/v2beta1/visualizations/{namespace} |
Security
Bearer
Type: apiKey- Name:
authorization
- In:
header
Paths
GET /apis/v2beta1/auth
Tags: AuthService| namespace | query | string | ||
| resources | query | string , x ∈ { UNASSIGNED_RESOURCES (default) , VIEWERS } | ||
| verb | query | string , x ∈ { UNASSIGNED_VERB (default) , CREATE , GET , DELETE } |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
GET /apis/v2beta1/experiments
Tags: ExperimentService| page_token | A page token to request the next page of results. The token is acquried from the nextPageToken field of the response from the previous ListExperiments call or can be omitted when fetching the first page. | query | string | |
| page_size | The number of experiments to be listed per page. If there are more experiments than this number, the response message will contain a nextPageToken field you can use to fetch the next page. | query | integer (int32) | |
| sort_by | Can be format of "field_name", "field_name asc" or "field_name desc" Ascending by default. | query | string | |
| filter | A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto). | query | string | |
| namespace | Which namespace to filter the experiments on. | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
Uses default content-types: application/json
The experiment to be created.
Uses default content-types: application/json
- 200 OK
A successful response.
DELETE /apis/v2beta1/experiments/{experiment_id}
Tags: ExperimentService| experiment_id | The ID of the experiment to be deleted. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
GET /apis/v2beta1/experiments/{experiment_id}
Tags: ExperimentService| experiment_id | The ID of the experiment to be retrieved. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
POST /apis/v2beta1/experiments/{experiment_id}:archive
Tags: ExperimentService| experiment_id | The ID of the experiment to be archived. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
POST /apis/v2beta1/experiments/{experiment_id}:unarchive
Tags: ExperimentService| experiment_id | The ID of the experiment to be restored. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
Uses default content-types: application/json
- 200 OK
A successful response.
- default
| namespace | Optional input. Namespace for the pipelines. | query | string | |
| page_token | A page token to request the results page. | query | string | |
| page_size | The number of pipelines to be listed per page. If there are more pipelines than this number, the response message will contain a valid value in the nextPageToken field. | query | integer (int32) | |
| sort_by | Sorting order in form of "field_name", "field_name asc" or "field_name desc". Ascending by default. | query | string | |
| filter | A url-encoded, JSON-serialized filter protocol buffer (see filter.proto). | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
Uses default content-types: application/json
Required input. Pipeline that needs to be created.
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/pipelines/create
Tags: PipelineServiceUses default content-types: application/json
Uses default content-types: application/json
- 200 OK
A successful response.
- default
GET /apis/v2beta1/pipelines/names/{name}
Tags: PipelineService| name | Required input. Name of the pipeline to be retrieved. | path | string | |
| namespace | Optional input. Namespace of the pipeline. It could be empty if default namespaces needs to be used or if multi-user support is turned off. | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/pipelines/upload
Tags: PipelineUploadServicemultipart/form-data
| uploadfile | The pipeline to upload. Maximum size of 32MB is supported. | formData | file | |
| name | query | string | ||
| description | query | string | ||
| namespace | query | string |
application/json
- 200 OK
- default
POST /apis/v2beta1/pipelines/upload_version
Tags: PipelineUploadServicemultipart/form-data
| uploadfile | The pipeline to upload. Maximum size of 32MB is supported. | formData | file | |
| name | query | string | ||
| pipelineid | query | string | ||
| description | query | string |
application/json
- 200 OK
- default
DELETE /apis/v2beta1/pipelines/{pipeline_id}
Tags: PipelineService| pipeline_id | Required input. ID of the pipeline to be deleted. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
| pipeline_id | Required input. The ID of the pipeline to be retrieved. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
GET /apis/v2beta1/pipelines/{pipeline_id}/versions
Tags: PipelineService| pipeline_id | Required input. ID of the parent pipeline. | path | string | |
| page_token | A page token to request the results page. | query | string | |
| page_size | The number of pipeline versions to be listed per page. If there are more pipeline versions than this number, the response message will contain a valid value in the nextPageToken field. | query | integer (int32) | |
| sort_by | Sorting order in form of "field_name", "field_name asc" or "field_name desc". Ascending by default. | query | string | |
| filter | A url-encoded, JSON-serialized filter protocol buffer (see filter.proto). | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/pipelines/{pipeline_id}/versions
Tags: PipelineServiceUses default content-types: application/json
Required input. Pipeline version ID to be created.
| pipeline_id | Required input. ID of the parent pipeline. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
DELETE /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id}
Tags: PipelineService| pipeline_id | Required input. ID of the parent pipeline. | path | string | |
| pipeline_version_id | Required input. The ID of the pipeline version to be deleted. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
GET /apis/v2beta1/pipelines/{pipeline_id}/versions/{pipeline_version_id}
Tags: PipelineService| pipeline_id | Required input. ID of the parent pipeline. | path | string | |
| pipeline_version_id | Required input. ID of the pipeline version to be retrieved. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
GET /apis/v2beta1/recurringruns
Tags: RecurringRunService| page_token | A page token to request the next page of results. The token is acquired from the nextPageToken field of the response from the previous ListRecurringRuns call or can be omitted when fetching the first page. | query | string | |
| page_size | The number of recurring runs to be listed per page. If there are more recurring runs than this number, the response message will contain a nextPageToken field you can use to fetch the next page. | query | integer (int32) | |
| sort_by | Can be formatted as "field_name", "field_name asc" or "field_name desc". Ascending by default. | query | string | |
| namespace | Optional input. The namespace the recurring runs belong to. | query | string | |
| filter | A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto). | query | string | |
| experiment_id | The ID of the experiment to be retrieved. If empty, list recurring runs across all experiments. | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
POST /apis/v2beta1/recurringruns
Tags: RecurringRunServiceUses default content-types: application/json
The recurring run to be created.
Uses default content-types: application/json
- 200 OK
A successful response.
DELETE /apis/v2beta1/recurringruns/{recurring_run_id}
Tags: RecurringRunService| recurring_run_id | The ID of the recurring run to be deleted. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
GET /apis/v2beta1/recurringruns/{recurring_run_id}
Tags: RecurringRunService| recurring_run_id | The ID of the recurring run to be retrieved. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
POST /apis/v2beta1/recurringruns/{recurring_run_id}:disable
Tags: RecurringRunService| recurring_run_id | The ID of the recurring runs to be disabled. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
POST /apis/v2beta1/recurringruns/{recurring_run_id}:enable
Tags: RecurringRunService| recurring_run_id | The ID of the recurring runs to be enabled. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
GET /apis/v2beta1/runs
Tags: RunService| namespace | Optional input field. Filters based on the namespace. | query | string | |
| experiment_id | The ID of the parent experiment. If empty, response includes runs across all experiments. | query | string | |
| page_token | A page token to request the next page of results. The token is acquired from the nextPageToken field of the response from the previous ListRuns call or can be omitted when fetching the first page. | query | string | |
| page_size | The number of runs to be listed per page. If there are more runs than this number, the response message will contain a nextPageToken field you can use to fetch the next page. | query | integer (int32) | |
| sort_by | Can be format of "field_name", "field_name asc" or "field_name desc" (Example, "name asc" or "id desc"). Ascending by default. | query | string | |
| filter | A url-encoded, JSON-serialized Filter protocol buffer (see filter.proto). | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/runs
Tags: RunServiceUses default content-types: application/json
Run to be created.
Uses default content-types: application/json
- 200 OK
A successful response.
- default
DELETE /apis/v2beta1/runs/{run_id}
Tags: RunService| run_id | The ID of the run to be deleted. | path | string | |
| experiment_id | The ID of the parent experiment. | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
| run_id | The ID of the run to be retrieved. | path | string | |
| experiment_id | The ID of the parent experiment. | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
GET /apis/v2beta1/runs/{run_id}/nodes/{node_id}/artifacts/{artifact_name}:read
Tags: RunService| run_id | ID of the run. | path | string | |
| node_id | ID of the running node. | path | string | |
| artifact_name | Name of the artifact. | path | string | |
| experiment_id | The ID of the parent experiment. | query | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/runs/{run_id}:archive
Tags: RunService| run_id | The ID of the run to be archived. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
| run_id | The ID of the run to be retried. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
| run_id | The ID of the run to be terminated. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/runs/{run_id}:unarchive
Tags: RunService| run_id | The ID of the run to be restored. | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/scheduledworkflows
Tags: ReportServiceUses default content-types: application/json
ScheduledWorkflow a ScheduledWorkflow resource marshalled into a json string.
Uses default content-types: application/json
- 200 OK
A successful response.
POST /apis/v2beta1/visualizations/{namespace}
Tags: VisualizationServiceUses default content-types: application/json
| namespace | path | string |
Uses default content-types: application/json
- 200 OK
A successful response.
- default
POST /apis/v2beta1/workflows
Tags: ReportServiceUses default content-types: application/json
Workflow is a workflow custom resource marshalled into a json string.
Uses default content-types: application/json
- 200 OK
A successful response.
Schema definitions
Type of resources in pipelines system.
Type of verbs that act on the resources.
The Status type defines a logical error model that is suitable for
different programming environments, including REST APIs and RPC APIs. It is
used by gRPC. Each Status message contains
three pieces of data: error code, error message, and error details.
You can find out more about this error model and how to work with it in the API Design Guide.
- code: integer (int32)
The status code, which should be an enum value of [google.rpc.Code][google.rpc.Code].
- message: string
A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the [google.rpc.Status.details][google.rpc.Status.details] field, or localized by the client.
- details: object[]
A list of messages that carry the error details. There is a common set of message types for APIs to use.
A dependent task that requires this one to succeed. Represented by either task_id or pod_name.
- task_id: string
System-generated ID of a task.
- pod_name: string
Name of the corresponding pod assigned by the orchestration engine. Also known as node_id.
Any contains an arbitrary serialized protocol buffer message along with a
URL that describes the type of the serialized message.
Protobuf library provides support to pack/unpack Any values in the form of utility functions or additional generated methods of the Any type.
Example 1: Pack and unpack a message in C++.
Foo foo = ...;
Any any;
any.PackFrom(foo);
...
if (any.UnpackTo(&foo)) {
...
}Example 2: Pack and unpack a message in Java.
Foo foo = ...;
Any any = Any.pack(foo);
...
if (any.is(Foo.class)) {
foo = any.unpack(Foo.class);
}Example 3: Pack and unpack a message in Python.
foo = Foo(...)
any = Any()
any.Pack(foo)
...
if any.Is(Foo.DESCRIPTOR):
any.Unpack(foo)
...Example 4: Pack and unpack a message in Go
foo := &pb.Foo{...}
any, err := anypb.New(foo)
if err != nil {
...
}
...
foo := &pb.Foo{}
if err := any.UnmarshalTo(foo); err != nil {
...
}The pack methods provided by protobuf library will by default use 'type.googleapis.com/full.type.name' as the type URL and the unpack methods only use the fully qualified type name after the last '/' in the type URL, for example "foo.bar.com/x/y.z" will yield type name "y.z".
JSON
The JSON representation of an Any value uses the regular
representation of the deserialized, embedded message, with an
additional field @type which contains the type URL. Example:
package google.profile;
message Person {
string first_name = 1;
string last_name = 2;
}
{
"@type": "type.googleapis.com/google.profile.Person",
"firstName": <string>,
"lastName": <string>
}If the embedded message type is well-known and has a custom JSON
representation, that representation will be embedded adding a field
value which holds the custom JSON in addition to the @type
field. Example (for message [google.protobuf.Duration][]):
{
"@type": "type.googleapis.com/google.protobuf.Duration",
"value": "1.212s"
}- type_url: string
A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. This string must contain at least one "/" character. The last segment of the URL's path must represent the fully qualified name of the type (as in
path/google.protobuf.Duration). The name should be in a canonical form (e.g., leading "." is not accepted).In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme
http,https, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows:- If no scheme is provided,
httpsis assumed. - An HTTP GET on the URL must yield a [google.protobuf.Type][] value in binary format, or produce an error.
- Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.)
Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com.
Schemes other than
http,https(or the empty scheme) might be used with implementation specific semantics.- If no scheme is provided,
- value: string (byte)
Must be a valid serialized protocol buffer of the above specified type.
NullValue is a singleton enumeration to represent the null value for the
Value type union.
The JSON representation for NullValue is JSON null.
- NULL_VALUE: Null value.
Required input. User setting to enable or disable the recurring run. Only used for creation of recurring runs. Later updates use enable/disable API.
- DISABLE: The recurring run won't schedule any run if disabled.
A list of artifact metadata.
- artifact_ids: string[]
A list of artifact metadata ids.
string (int64)
- pipeline: v2beta1Pipeline
Required input. Pipeline (parent) to be created.
- pipeline_version: v2beta1PipelineVersion
Required input. Pipeline version (child) to be created. Pipeline spec will be downloaded from pipeline_version.package_url.
CronSchedule allow scheduling the recurring run with unix-like cron.
- start_time: string (date-time)
The start time of the cron job.
- end_time: string (date-time)
The end time of the cron job.
- cron: string
- experiment_id: string
Output. Unique experiment ID. Generated by API server.
- display_name: string
Required input field. Unique experiment name provided by user.
- description: string
Optional input field. Describes the purpose of the experiment.
- created_at: string (date-time)
Output. The time that the experiment was created.
- namespace: string
Optional input field. Specify the namespace this experiment belongs to.
- storage_state: v2beta1ExperimentStorageState
Output. Specifies whether this experiment is in archived or available state.
v2beta1ExperimentStorageState: string , x ∈ { STORAGE_STATE_UNSPECIFIED (default) , AVAILABLE , ARCHIVED }
Describes whether an entity is available or archived.
- STORAGE_STATE_UNSPECIFIED: Default state. This state in not used
- AVAILABLE: Entity is available.
- ARCHIVED: Entity is archived.
Filter is used to filter resources returned from a ListXXX request.
Example filters:
1) Filter runs with status = 'Running' filter { predicate { key: "status" operation: EQUALS string_value: "Running" } }
2) Filter runs that succeeded since Dec 1, 2018 filter { predicate { key: "status" operation: EQUALS string_value: "Succeeded" } predicate { key: "created_at" operation: GREATER_THAN timestamp_value { seconds: 1543651200 } } }
3) Filter runs with one of labels 'label_1' or 'label_2'
filter { predicate { key: "label" operation: IN string_values { value: 'label_1' value: 'label_2' } } }
- predicates: object[]
All predicates are AND-ed when this filter is applied.
- experiments: object[]
A list of experiments returned.
- total_size: integer (int32)
The number of experiments for the given query.
- next_page_token: string
The token to list the next page of experiments.
- pipelines: object[]
Returned pipelines.
- total_size: integer (int32)
The total number of pipelines for the given query.
- next_page_token: string
The token to list the next page of pipelines. This token can be used on the next ListPipelinesRequest.
- pipeline_versions: object[]
Returned pipeline versions.
- next_page_token: string
The token to list the next page of pipeline versions.
- total_size: integer (int32)
The total number of pipeline versions for the given query.
- recurringRuns: object[]
A list of recurring runs returned.
- total_size: integer (int32)
The total number of recurring runs for the given query.
- next_page_token: string
The token to list the next page of recurring runs.
- runs: object[]
List of retrieved runs.
- total_size: integer (int32)
The total number of runs for the given query.
- next_page_token: string
The token to list the next page of runs.
PeriodicSchedule allow scheduling the recurring run periodically with certain interval.
- start_time: string (date-time)
The start time of the periodic recurring run.
- end_time: string (date-time)
The end time of the periodic recurring run.
- interval_second: string (int64)
The time interval between the starting time of consecutive recurring runs.
- pipeline_id: string
Output. Unique pipeline ID. Generated by API server.
- display_name: string
Required input field. Pipeline name provided by user.
- description: string
Optional input field. A short description of the pipeline.
- created_at: string (date-time)
Output. Creation time of the pipeline.
- namespace: string
Input. A namespace this pipeline belongs to. Causes error if user is not authorized to access the specified namespace. If not specified in CreatePipeline, default namespace is used.
- error: googlerpcStatus
In case any error happens retrieving a pipeline field, only pipeline ID, and the error message is returned. Client has the flexibility of choosing how to handle the error. This is especially useful during listing call.
Runtime information of a task execution.
- run_id: string
ID of the parent run.
- task_id: string
System-generated ID of a task.
- display_name: string
User specified name of a task that is defined in [Pipeline.spec][].
- create_time: string (date-time)
Creation time of a task.
- start_time: string (date-time)
Starting time of a task.
- end_time: string (date-time)
Completion time of a task.
- executor_detail: v2beta1PipelineTaskExecutorDetail
Execution information of a task.
- state: v2beta1RuntimeState
Runtime state of a task.
- execution_id: string (int64)
Execution id of the corresponding entry in ML metadata store.
- error: googlerpcStatus
The error that occurred during task execution. Only populated when the task is in FAILED or CANCELED state.
- inputs: object
Input artifacts of the task.
- outputs: object
Output artifacts of the task.
- parent_task_id: string
ID of the parent task if the task is within a component scope. Empty if the task is at the root level.
- state_history: object[]
A sequence of task statuses. This field keeps a record of state transitions.
- pod_name: string
Name of the corresponding pod assigned by the orchestration engine. Also known as node_id.
- child_tasks: object[]
Sequence of dependen tasks.
Runtime information of a pipeline task executor.
- main_job: string
The name of the job for the main container execution.
- pre_caching_check_job: string
The name of the job for the pre-caching-check container execution. This job will be available if the Run.pipeline_spec specifies the
pre_caching_checkhook in the lifecycle events.- failed_main_jobs: string[]
The names of the previously failed job for the main container executions. The list includes the all attempts in chronological order.
string - failed_pre_caching_check_jobs: string[]
The names of the previously failed job for the pre-caching-check container executions. This job will be available if the Run.pipeline_spec specifies the
pre_caching_checkhook in the lifecycle events. The list includes the all attempts in chronological order.string
- pipeline_id: string
Required input field. Unique ID of the parent pipeline.
- pipeline_version_id: string
Output. Unique pipeline version ID. Generated by API server.
- display_name: string
Required input field. Pipeline version name provided by user.
- description: string
Optional input field. Short description of the pipeline version.
- created_at: string (date-time)
Output. Creation time of the pipeline version.
- package_url: v2beta1Url
Input. Required. The URL to the source of the pipeline version. This is required when creating the pipeine version through CreatePipelineVersion API.
- code_source_url: string
Input. Optional. The URL to the code source of the pipeline version. The code is usually the Python definition of the pipeline and potentially related the component definitions. This allows users to trace back to how the pipeline YAML was created.
- pipeline_spec: object
Output. The pipeline spec for the pipeline version.
- error: googlerpcStatus
In case any error happens retrieving a pipeline version field, only pipeline ID, pipeline version ID, and the error message are returned. Client has the flexibility of choosing how to handle the error. This is especially useful during List() calls.
Reference to an existing pipeline version.
- pipeline_id: string
Input. Required. Unique ID of the parent pipeline.
- pipeline_version_id: string
Input. Required. Unique ID of an existing pipeline version.
Predicate captures individual conditions that must be true for a resource being filtered.
- operation: v2beta1PredicateOperation
- key: string
Key for the operation (first argument).
- int_value: integer (int32)
Integer.
- long_value: string (int64)
Long integer.
- string_value: string
String.
- timestamp_value: string (date-time)
Timestamp values will be converted to Unix time (seconds since the epoch) prior to being used in a filtering operation.
- int_values: PredicateIntValues
Array values below are only meant to be used by the IN operator.
- long_values: PredicateLongValues
List of long integers.
- string_values: PredicateStringValues
List of strings.
v2beta1PredicateOperation: string , x ∈ { OPERATION_UNSPECIFIED (default) , EQUALS , NOT_EQUALS , GREATER_THAN , GREATER_THAN_EQUALS , LESS_THAN , LESS_THAN_EQUALS , IN , IS_SUBSTRING }
Operation is the operation to apply.
- OPERATION_UNSPECIFIED: Default operation. This operation is not used.
- EQUALS: Operation on scalar values. Only applies to one of |int_value|, |long_value|, |string_value| or |timestamp_value|.
- NOT_EQUALS: Negated EQUALS.
- GREATER_THAN: Greater than operation.
- GREATER_THAN_EQUALS: Greater than or equals operation.
- LESS_THAN: Less than operation.
- LESS_THAN_EQUALS: Less than or equals operation
- IN: Checks if the value is a member of a given array, which should be one of |int_values|, |long_values| or |string_values|.
- IS_SUBSTRING: Checks if the value contains |string_value| as a substring match. Only applies to |string_value|.
- recurring_run_id: string
Output. Unique run ID generated by API server.
- display_name: string
Required input field. Recurring run name provided by user. Not unique.
- description: string
Optional input field. Describes the purpose of the recurring run.
- pipeline_version_id: string
This field is Deprecated. The pipeline version id is under pipeline_version_reference for v2.
- pipeline_spec: object
The pipeline spec.
- pipeline_version_reference: v2beta1PipelineVersionReference
Reference to a pipeline version containing pipeline_id and pipeline_version_id.
- runtime_config: v2beta1RuntimeConfig
Runtime config of the pipeline.
- service_account: string
Optional input field. Specifies which Kubernetes service account this recurring run uses.
- max_concurrency: string (int64)
Required input field. Specifies how many runs can be executed concurrently. Range [1-10].
- trigger: v2beta1Trigger
Required input field. Specifies how a run is triggered. Support cron mode or periodic mode.
- mode: RecurringRunMode
- created_at: string (date-time)
Output. The time this recurring run was created.
- updated_at: string (date-time)
Output. The last time this recurring run was updated.
- status: v2beta1RecurringRunStatus
- error: googlerpcStatus
In case any error happens retrieving a recurring run field, only recurring run ID and the error message is returned. Client has the flexibility of choosing how to handle the error. This is especially useful during listing call.
- no_catchup: boolean (boolean)
Optional input field. Whether the recurring run should catch up if behind schedule. If true, the recurring run will only schedule the latest interval if behind schedule. If false, the recurring run will catch up on each past interval.
- namespace: string
TODO (gkclat): consider removing this field if it can be obtained from the parent experiment. Output only. Namespace this recurring run belongs to. Derived from the parent experiment.
- experiment_id: string
ID of the parent experiment this recurring run belongs to.
Output. The status of the recurring run.
- experiment_id: string
Input. ID of the parent experiment. The default experiment ID will be used if this is not specified.
- run_id: string
Output. Unique run ID. Generated by API server.
- display_name: string
Required input. Name provided by user, or auto generated if run is created by a recurring run.
- storage_state: v2beta1RunStorageState
Output. Specifies whether this run is in archived or available mode.
- description: string
Optional input. Short description of the run.
- pipeline_version_id: string
This field is Deprecated. The pipeline version id is under pipeline_version_reference for v2.
- pipeline_spec: object
Pipeline spec.
- pipeline_version_reference: v2beta1PipelineVersionReference
Reference to a pipeline version containing pipeline_id and pipeline_version_id.
- runtime_config: v2beta1RuntimeConfig
Required input. Runtime config of the run.
- service_account: string
Optional input. Specifies which kubernetes service account is used.
- created_at: string (date-time)
Output. Creation time of the run.
- scheduled_at: string (date-time)
Output. When this run is scheduled to start. This could be different from created_at. For example, if a run is from a backfilling job that was supposed to run 2 month ago, the created_at will be 2 month behind scheduled_at.
- finished_at: string (date-time)
Output. Completion of the run.
- state: v2beta1RuntimeState
Output. Runtime state of a run.
- error: googlerpcStatus
In case any error happens retrieving a run field, only run ID and the error message is returned. Client has the flexibility of choosing how to handle the error. This is especially useful during listing call.
- run_details: v2beta1RunDetails
Output. Runtime details of a run.
- recurring_run_id: string
ID of the recurring run that triggered this run.
- state_history: object[]
Output. A sequence of run statuses. This field keeps a record of state transitions.
Runtime details of a run.
- pipeline_context_id: string (int64)
Pipeline context ID of a run.
- pipeline_run_context_id: string (int64)
Pipeline run context ID of a run.
- task_details: object[]
Runtime details of the tasks that belong to the run.
Describes whether an entity is available or archived.
- STORAGE_STATE_UNSPECIFIED: Default state. This state in not used
- AVAILABLE: Entity is available.
- ARCHIVED: Entity is archived.
The runtime config.
- parameters: object
The runtime parameters of the Pipeline. The parameters will be used to replace the placeholders at runtime.
object - pipeline_root: string
v2beta1RuntimeState: string , x ∈ { RUNTIME_STATE_UNSPECIFIED (default) , PENDING , RUNNING , SUCCEEDED , SKIPPED , FAILED , CANCELING , CANCELED , PAUSED }
Describes the runtime state of an entity.
- RUNTIME_STATE_UNSPECIFIED: Default value. This value is not used.
- PENDING: Service is preparing to execute an entity.
- RUNNING: Entity execution is in progress.
- SUCCEEDED: Entity completed successfully.
- SKIPPED: Entity has been skipped. For example, due to caching.
- FAILED: Entity execution has failed.
- CANCELING: Entity is being canceled. From this state, an entity may only change its state to SUCCEEDED, FAILED or CANCELED.
- CANCELED: Entity has been canceled.
- PAUSED: Entity has been paused. It can be resumed.
Timestamped representation of a runtime state with an optional error.
- update_time: string (date-time)
Update time of this state.
- state: v2beta1RuntimeState
The state of a runtime instance.
- error: googlerpcStatus
The error that occurred during the state. May be set when the state is any of the non-final states (PENDING/RUNNING/CANCELING) or FAILED state. If the state is FAILED, the error here is final and not going to be retried. If the state is a non-final state, the error indicates that a system-error being retried.
Trigger defines what starts a pipeline run.
- cron_schedule: v2beta1CronSchedule
- periodic_schedule: v2beta1PeriodicSchedule
- type: v2beta1VisualizationType
- source: string
Path pattern of input data to be used during generation of visualizations. This is required when creating the pipeline through CreateVisualization API.
- arguments: string
Variables to be used during generation of a visualization. This should be provided as a JSON string. This is required when creating the pipeline through CreateVisualization API.
- html: string
Output. Generated visualization html.
- error: string
In case any error happens when generating visualizations, only visualization ID and the error message are returned. Client has the flexibility of choosing how to handle the error.
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.