How to Create a New Environment

An environment is automatically installed in each generated notebook server. But in case you want to initialize a new custom Python environment, follow the indications in this page.

Why create a new environment

AI Platform offers IDEs with pre-configured environments that you can choose from when running computations in a notebook. The pre-configured environments are built on Conda and include a wide range of commonly used Python packages to support typical data science and machine learning workflows.

If you need to use packages that are not available in the pre-configured environments, you can install them on-demand from within a notebook cell or in IDE's terminal. But this has to be done each time a compute is restarted or the notebook is culled.

In this tutorial, we'll go through how to create a a new virtual environment on notebooks and make it available as kernel even after notebook restart.

View packages installed in an environment

To view the packages installed in a virtual environment, do the following:

1. From the web-based IDE you generated, open a terminal window:

   1. Click the hamburger icon con the top-left.

   2. Select Terminal > New Terminal.

      Open terminal in IDE

2. From the terminal, run the following command:

   pip freeze

3. Check the output for a complete list of installed packages.

Set up environment

1. From the terminal, run the following command to see the list of existing Conda environments.

   conda env list

2. Create a new conda environment:

   conda create --prefix <SUBDIRECTORY> python=<PYTHON_VERSION>

   • SUBDIRECTORY = directory that will be created in /home/jovyan
   • PYTHON_VERSION = specifies the interpreter version that will be used to run your code
   
   
   Further Resources

   We use --prefix <SUBDIRECTORY> instead of conda create -n <NAME> python=<PYTHON_VERSION> to ensure the environment is created in a persistent location. This is necessary because notebook culling can delete environments stored in the default directory.

   For details, go to:

   • Specifying a location for an environment | Conda

3. Activate the environment:

   conda activate /home/jovyan/<SUBDIRECTORY>

4. You can now install additional packages using conda, pip, a requirements.txt file, or an environment.yml file.

   For example:

   pip install pmdarima

Set kernel

1. To make the environment available as a kernel, you need to install ipykernel:

   pip install ipykernel

2. Next, run ipykernel:

Shell

From the terminal:

python -m ipykernel install --user --name <NAME> --display-name "<DISPLAY_NAME>"

Python

Within the notebook:

!python3 -m ipykernel install --user --name <NAME> --display-name "<DISPLAY_NAME>"

• NAME = sets the internal identifier for the kernel (used in Jupyter config)
• DISPLAY_NAME = sets the human-friendly name you see in JupyterLab/VS Code when selecting kernels; use within quotation marks

This will install the kernelspec in /home/jovyan/.local/share/jupyter/kernels/<NAME>.

Further Resources

For details, go to:

• Kernels for different environments | IPython Documentation

1. Close and reconnect your notebook. Your environment should now appear as an available kernel.

   1. Click Select Kernel or currently selected kernel.
   2. Click Select Another Kernel if the recently created kernel is not visible.

   Select kernel

   3. Click Jupyter Kernel.
   4. Select the kernal.

Kernel is now available