Run a Pipeline

Execute a pipeline on the KFP backend

The KFP offers three ways to run a pipeline.

1. Run from the KFP Dashboard

The first and easiest way to run a pipeline is by submitting it via the KFP dashboard.

To submit a pipeline to the KFP Dashboard:

  1. Compile the pipeline to IR YAML.

  2. From the Dashboard, select “+ Upload pipeline”.

Upload pipeline button

  1. Upload the pipeline IR YAML to “Upload a file”, populate the upload pipeline form, and click “Create”.

Upload pipeline screen

  1. From the Runs tab, select “+ Create run”:

Create run button

  1. Choose the pipeline you uploaded, provide a name, any run parameters, and click “Start”. Start a run screen

2. Run from the KFP SDK client

You may also programatically submit pipeline runs from the KFP SDK client. The client supports two ways of submitting runs: from IR YAML or from a Python pipeline function. For either approach, start by instantiating a Client using the host URL of your KFP instance:

from kfp.client import Client
client = Client(host='<YOUR_HOST_URL>')

To submit IR YAML for execution use the .create_run_from_pipeline_package method:

client.create_run_from_pipeline_package('pipeline.yaml', arguments={'param': 'a', 'other_param': 2})

To submit a Python pipeline function for execution use the .create_run_from_pipeline_func convenience method, which wraps compilation and run submission into one method:

client.create_run_from_pipeline_func(pipeline_func, arguments={'param': 'a', 'other_param': 2})

See the KFP SDK Client reference documentation for a detailed description of the Client constructor and method parameters.

3. Run from the KFP SDK CLI

The kfp run create command allows you to submit a pipeline from the command line. kfp run create --help shows that this command takes the form:

kfp run create [OPTIONS] [ARGS]...

For example, the following command submits the path/to/pipeline.yaml IR YAML to the KFP backend:

kfp run create --experiment-name my-experiment --package-file path/to/pipeline.yaml

For more information about the kfp run create command, see Command Line Interface in the KFP SDK reference documentation. For a summary of the available commands in the KFP CLI, see Command-line Interface.

Feedback

Was this page helpful?


Last modified May 31, 2024: Regrouped user guides (350dde8a)