Automating Conda Environment Creation in Azure ML Compute Instances

Imagine that you always use the same conda packages or pip packages, imagine that you always connect to a corporate git repo either Azure Devops, or Github enterprise.

Probably your essential packages are always the same, and you need to manually connect to git from the Azure Terminal.

There is way to automate this in Azure ML Studio which I will explain in the following steps.

1. Create a Folder in Azure Notebooks and add 2 files: environment.yml and SetupCompute.sh

2. The environment.yml file contains the environment name, the conda dependencies and also the pip dependencies.

name: p310v2
dependencies:
- python=3.10
- numpy
- matplotlib
- pandas
- scikit-learn
- pip

3. SetupCompute.sh is the bash script that will be executed when your compute instance is provisioned, here you can add any Linux Commands and its how we will automate everything.

In the code below you can see with commends that we are creating a conda environment, installing ipykernel and also installin all requirements from the yml file. This script is installing azure-ai-ml wich is Azure Python ML SDK V2 which is compatible with Python 3.10

# Examples taken from here: https://github.com/Azure/azureml-examples/tree/main/setup/setup-ci
# ------------------------- Setup Conda Environment ----------------
#You can create env manually or with an env.yml file
#conda env create --Name py310v2 python=3.10
conda env create -f environment.yml
echo "Activating new conda environment"
conda activate p310v2
conda install -y ipykernel
echo "Installing kernel"
sudo -u azureuser -i <<'EOF'
conda activate p310v2
python -m ipykernel install --user --name p310v2 --display-name "p310v2"
echo "Conda environment setup successfully."
pip install azure-ai-ml # Azure ML SDK V2 https://pypi.org/project/azure-ai-ml/  Requires: Python <4.0, >=3.7