Install the Kubeflow Pipelines SDK

This guide tells you how to install the Kubeflow Pipelines SDK which you can use to build machine learning pipelines. You can use the SDK to execute your pipeline, or alternatively you can upload the pipeline to the Kubeflow Pipelines UI for execution.

All of the SDK’s classes and methods are described in the auto-generated SDK reference docs.

Note: If you are running Kubeflow Pipelines with Tekton, instead of the default Kubeflow Pipelines with Argo, you should use the Kubeflow Pipelines SDK for Tekton.

Set up Python

You need Python 3.5 or later to use the Kubeflow Pipelines SDK. This guide uses Python 3.7.

If you haven’t yet set up a Python 3 environment, do so now. This guide recommends Miniconda, but you can use a virtual environment manager of your choice, such as virtualenv.

Follow the steps below to set up Python using Miniconda:

1. Choose one of the following methods to install Miniconda, depending on your environment:

   • Debian/Ubuntu/Cloud Shell:

     apt-get update; apt-get install -y wget bzip2
     wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
     bash Miniconda3-latest-Linux-x86_64.sh
     

   • Windows: Download the installer and make sure you select the option to Add Miniconda to my PATH environment variable during the installation.

   • MacOS: Download the installer and run the following command:

     bash Miniconda3-latest-MacOSX-x86_64.sh
     

2. Check that the conda command is available:

   which conda
   

   If the conda command is not found, add Miniconda to your path:

   export PATH=<YOUR_MINICONDA_PATH>/bin:$PATH
   

3. Create a clean Python 3 environment with a name of your choosing. This example uses Python 3.7 and an environment name of mlpipeline.:

   conda create --name mlpipeline python=3.7
   conda activate mlpipeline
   

Install the Kubeflow Pipelines SDK

Run the following command to install the Kubeflow Pipelines SDK:

pip install kfp==1.8

Note: If you are not using a virtual environment, such as conda, when installing the Kubeflow Pipelines SDK, you may receive the following error:

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python3.5/dist-packages/kfp-<version>.dist-info'
Consider using the `--user` option or check the permissions.

If you get this error, install kfp with the --user option:

pip install kfp==1.8

This command installs the dsl-compile and kfp binaries under ~/.local/bin, which is not part of the PATH in some Linux distributions, such as Ubuntu. You can add ~/.local/bin to your PATH by appending the following to a new line at the end of your .bashrc file:

export PATH=$PATH:~/.local/bin

After successful installation, the command dsl-compile should be available. You can use this command to verify it:

which dsl-compile

The response should be something like this:

/<PATH_TO_YOUR_USER_BIN>/miniconda3/envs/mlpipeline/bin/dsl-compile

Next steps

• See how to use the SDK.
• Build a component and a pipeline.
• Get started with the UI.
• Understand pipeline concepts.