4 - Virtualenv

Python Virtual Environments

In Part 2 I used --break-system-packages every time I installed something with pip. That flag exists because Ubuntu 22.04 protects its system Python from being polluted by random packages. The right solution isn’t to force past that protection, it’s to stop installing things into the system Python altogether. Virtual environments are how I do that.

When I install a Python package with pip3 install ansible, it goes into the system-wide Python installation at /usr/lib/python3/. Every Python script on the entire Ubuntu VM now shares that installation.

Here’s the problem: different projects need different versions of the same package.

Project A (ansible-network):   needs ansible==9.x, netmiko==4.x
Project B (legacy-scripts):    needs ansible==2.9, netmiko==3.x
System tools (apt, Ubuntu):    need specific Python packages to function

If I install everything into the system Python, these projects will constantly conflict. Upgrading Ansible for Project A breaks Project B. A bad pip install can break Ubuntu’s own system tools that depend on Python.

A virtual environment solves this by creating a completely isolated Python installation for each project:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

/home/ansible/
├── venvs/
│   ├── ansible-network/ 
│   │   ├── bin/python3  
│   │   ├── bin/pip  
│   │   ├── bin/ansible 
│   │   └── lib/  
│   └── legacy-scripts/  
│       ├── bin/python3
│       └── lib/
└── projects/
    └── ansible-network/

When I activate a virtual environment, my shell uses that environment’s Python and pip instead of the system ones. When I deactivate it, the system Python comes back.

venv vs virtualenv

Both tools create virtual environments, but they come from different places and have slightly different capabilities.

venv is included with Python 3.3+ and ships with Ubuntu 22.04. No installation required. It’s the official, standard way to create virtual environments and covers everything I need.

How to check if venv is available:

python3 -m venv --help

Limitations of venv:

• Cannot create environments for Python versions other than the one it’s called from
• Slightly fewer configuration options than virtualenv

virtualenv is a more mature, feature-rich third-party package that predates Python’s built-in venv. It’s what most older documentation and enterprise tooling references when they say “virtual environment.”

How to install and check version:

pip3 install virtualenv --break-system-packages
virtualenv --version

Additional capabilities of virtualenv:

• Can create environments for different Python versions (e.g., virtualenv -p python3.9 myenv)
• Faster environment creation
• More configuration options
• Required by some tools like tox (a testing framework)

Installing pip and virtualenv

Before creating any environments, I make sure the necessary tools are in place.

Check Python version:

python3 --version

Check Pip version:

pip3 --version

Check if venv module is available:

python3 -m venv --help 2>&1 | head -5

On a fresh Ubuntu 22.04 install, I run:

sudo apt update
sudo apt install -y python3-pip python3-venv

pip has its own updates. Before creating any environments, I upgrade pip:

pip3 install --upgrade pip --break-system-packages

Even though I’ll primarily use venv for this project, I install virtualenv to have both available:

pip3 install virtualenv --break-system-packages
virtualenv --version

Creating a Virtual Environment

Now I create the virtual environment that will house Ansible and all related tools for this lab. I keep all my virtual environments in ~/venvs/ and project code in ~/projects/.

First, I create the venvs directory:

mkdir -p ~/venvs

Then I create the virtual environment:

python3 -m venv ~/venvs/ansible-network

tree ~/venvs/ansible-network/ -L 3

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

/home/ansible/venvs/ansible-network/
├── bin/
│   ├── activate 
│   ├── activate.fish
│   ├── pip
│   ├── pip3
│   ├── python
│   └── python3
├── include/
├── lib/
│   └── python3.10/
│       └── site-packages/
└── pyvenv.cfg

pyvenv.cfg is worth looking at:

cat ~/venvs/ansible-network/pyvenv.cfg

home = /usr/bin
include-system-site-packages = false
version = 3.10.12

The include-system-site-packages = false line means this environment is completely isolated from the system Python packages. Nothing installed system-wide bleeds in.

Use virtualenv instead of venv:

virtualenv ~/venvs/ansible-network-v2

Target a sepcific Python version with virtualenv:

virtualenv -p python3.10 ~/venvs/ansible-network-v3

-p python3.10 - tells virtualenv which Python binary to use. This is the main advantage over venv since I can target any installed Python version.

Naming Conventions

Project-Specific vs Generic naming conventions.

python3 -m venv ~/venvs/ansible-network
python3 -m venv ~/venvs/netbox-scripts
python3 -m venv ~/venvs/legacy-automation

When to use this: When I have multiple projects with different dependency requirements. Each project gets its own environment. The name makes it immediately obvious which project the environment belongs to.

Pros:

• Clear association between environment and project
• Easy to manage when working on multiple projects
• Aligns with the “one virtualenv per project” best practice

Cons:

• If I rename the project, the environment name becomes confusing
• Managing many environments requires discipline

python3 -m venv ~/venvs/ansible-env
python3 -m venv ~/venvs/python-env

When to use this: When I have one general-purpose environment for all Ansible work, or on a dedicated Ansible control node that only ever runs one type of workload.

Pros:

• Simple, one environment to rule them all
• Common in CI/CD pipelines where the environment name doesn’t matter

Cons:

• Over time, the environment accumulates packages from multiple projects
• Hard to tell which packages are actually needed vs. installed by accident
• requirements.txt becomes bloated

For this project I use the project-specific naming approach:

python3 -m venv ~/venvs/ansible-network

This environment is dedicated to this project. When I start a new, separate project later, it gets its own environment.

Activating and Deactivating

Creating the environment doesn’t automatically use it. I need to activate it.

source ~/venvs/ansible-network/bin/activate

• source - runs the activation script in the current shell session (not a subprocess). The environment variables set by activate wouldn’t affect my current shell.
• After activation, my shell prompt changes to show the environment name:

# Before activation
ansible@ubuntu:~$

# After activation
(ansible-network) ansible@ubuntu:~$

That (ansible-network) prefix is my visual confirmation that I’m inside the right environment.

Check where Python is coming from:

which python3

# /home/ansible/venvs/ansible-network/bin/python3

Check where pip coming from:

which pip

# /home/ansible/venvs/ansible-network/bin/pip

Check whhat packages are currently installed:

pip list

# Package    Version
# ---------- -------
# pip        24.x.x
# setuptools 68.x.x

deactivate

My prompt returns to normal and Python/pip point back to the system installation.

which python3

# /usr/bin/python3

I can add the activation command to my ~/.bashrc so it activates automatically on every new shell:

echo "source ~/venvs/ansible-network/bin/activate" >> ~/.bashrc
source ~/.bashrc

Installing the Full Ansible Networking Stack

With the environment activated, I now install everything needed for this project. I no longer need --break-system-packages since I’m inside the virtualenv and pip only touches this isolated environment.

I always upgrade pip inside a fresh virtualenv before installing anything.

pip install --upgrade pip

I’ll install everything in a logical order, explaining what each package is and why I need it.

Core Ansible:

pip install ansible

Ansible linting and validation tools.

pip install ansible-lint
pip install yamllint

SSH connectivity libraries:

pip install paramiko
pip install netmiko

Vendor-neutral network abstraction:

pip install napalm

HTTP library for REST APIs (Netbox, AWX, PAN-OS):

pip install requests

YAML parser (used in helper scripts):

pip install pyyaml

Netbox Python client (for Netbox integration):

pip install pynetbox

Ansible Navigator (modern way to run and inspect playbooks):

pip install ansible-navigator

Or I can install everything in one command:

pip install ansible ansible-lint yamllint paramiko netmiko napalm requests pyyaml pynetbox ansible-navigator

I can verify Ansible is installed and check the version with:

ansible --version

ansible [core 2.17.x]
  config file = None
  configured module search path = ['/home/ansible/.ansible/plugins/modules', ...]
  ansible python module location = /home/ansible/venvs/ansible-network/lib/python3.10/site-packages/ansible
  ansible collection location = /home/ansible/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/ansible/venvs/ansible-network/bin/ansible
  python version = 3.10.12 (main, ...) [GCC 11.4.0]
  jinja version = 3.1.x
  libyaml = True

The key line to confirm: executable location should point into my virtualenv (/home/ansible/venvs/ansible-network/bin/ansible), not /usr/bin/ansible.

Verify the other tools:

ansible-lint --version
yamllint --version
python3 -c "import netmiko; print(netmiko.__version__)"
python3 -c "import napalm; print(napalm.__version__)"
python3 -c "import paramiko; print(paramiko.__version__)"

Collections are Ansible’s plugin/module packages. They’re installed separately from pip packages using ansible-galaxy:

Install network vendor collections:

ansible-galaxy collection install cisco.ios
ansible-galaxy collection install cisco.nxos
ansible-galaxy collection install junipernetworks.junos
ansible-galaxy collection install paloaltonetworks.panos

Install utility collections:

ansible-galaxy collection install ansible.netcommon
ansible-galaxy collection install ansible.utils
ansible-galaxy collection install netbox.netbox

Verify installed collections

ansible-galaxy collection list

Freezing Dependencies with requirements.txt

Right now my virtual environment has a specific set of packages at specific versions. If I delete the environment, move to a new VM, or hand this project to a colleague, I need a way to recreate it exactly. That’s what requirements.txt is for.

First, I make sure the virtualenv is activated, and then freeze all installed packages and their versions.

pip freeze > requirements.txt

View the results:

ansible==9.8.0
ansible-core==2.17.8
ansible-lint==24.9.2
ansible-navigator==24.8.0
cffi==1.17.1
cryptography==43.0.3
Jinja2==3.1.4
MarkupSafe==2.1.5
napalm==5.0.0
netmiko==4.4.0
nornir==3.4.0
packaging==24.1
paramiko==3.5.0
pynetbox==7.4.0
PyNaCl==1.5.0
pyyaml==6.0.2
requests==2.32.3
resolvelib==1.0.1
setuptools==68.2.0
urllib3==2.2.3
yamllint==1.35.1
...

pip freeze captures every single installed package including dependencies of my dependencies. This guarantees a perfectly reproducible environment.

There are two common approaches, and both have their place:

Style 1: Pinned versions for Maximum reproducibility

ansible==9.8.0
ansible-lint==24.9.2
netmiko==4.4.0
paramiko==3.5.0

Use this for production environments where I need guaranteed stability. Every engineer gets the exact same versions.

Style 2: Minimum versions for Maximum flexibility

ansible>=9.0.0
ansible-lint>=24.0.0
netmiko>=4.0.0
paramiko>=3.0.0

Use this for open-source or shared projects where I want to allow newer compatible versions. Less strict, but more likely to get the latest bug fixes.

For this project I use pinned versions from pip freeze for the actual project environment, and keep a separate loose requirements-dev.txt for quick new environment setups where exact versions matter less.

This is the payoff for maintaining requirements.txt. Whether I’m setting up a new VM, onboarding a colleague, or recovering from a broken environment, I can recreate everything in minutes.

Step 1: Clone the project repository:

git clone [email protected]:ernestodiaztech/ansible-network.git
cd ansible-network

Step 2: Create a fresh virtual environment:

python3 -m venv ~/venvs/ansible-network

Step 3: Activate it:

source ~/venvs/ansible-network/bin/activate

Step 4: Upgrade pip:

pip install --upgrade pip

Step 5: Install all packages from requirements.txt:

pip install -r requirements.txt

Step 6: Reinstall Ansible collections:

ansible-galaxy collection install -r collections/requirements.yml

Step 7: Verify

ansible --version

The -r requirements.txt flag tells pip to read the file and install everything listed in it. The entire environment is rebuilt in one command.

Just as requirements.txt tracks Python packages, collections/requirements.yml tracks Ansible collections:

mkdir -p ~/projects/ansible-network/collections

Create ~/projects/ansible-network/collections/requirements.yml:

---
collections:
  - name: cisco.ios
    version: ">=8.0.0"
  - name: cisco.nxos
    version: ">=5.0.0"
  - name: junipernetworks.junos
    version: ">=8.0.0"
  - name: paloaltonetworks.panos
    version: ">=2.0.0"
  - name: ansible.netcommon
    version: ">=6.0.0"
  - name: ansible.utils
    version: ">=4.0.0"
  - name: netbox.netbox
    version: ">=3.0.0"

Install from this file:

ansible-galaxy collection install -r collections/requirements.yml

Setting Up VS Code

VS Code needs to know which Python interpreter to use, specifically the one inside my virtual environment. Without this, VS Code’s IntelliSense, linting, and the Ansible extension won’t see the packages I’ve installed.

With VS Code connected to my Ubuntu VM via Remote-SSH:

1. Press Ctrl+Shift+P to open the Command Palette
2. Type Python: Select Interpreter and press Enter
3. VS Code scans for available Python environments and shows a list
4. I select the one pointing to my virtualenv:

Python 3.10.12 ('ansible-network': venv)
   ~/venvs/ansible-network/bin/python3

If VS Code doesn’t detect it automatically, I click Enter interpreter path… and type the full path:

/home/ansible/venvs/ansible-network/bin/python3

I add a .vscode/settings.json file to my project directory so the interpreter setting is saved with the project and applies automatically when anyone opens it:

mkdir -p ~/projects/ansible-network/.vscode

Create ~/projects/ansible-network/.vscode/settings.json:

{
    "python.defaultInterpreterPath": "/home/ansible/venvs/ansible-network/bin/python3",
    "ansible.python.interpreterPath": "/home/ansible/venvs/ansible-network/bin/python3",
    "ansible.validation.enabled": true,
    "ansible.validation.lint.enabled": true,
    "yaml.schemas": {
        "https://raw.githubusercontent.com/ansible/ansible-lint/main/src/ansiblelint/schemas/ansible.json#/$defs/playbook": "playbooks/*.yml",
        "https://raw.githubusercontent.com/ansible/ansible-lint/main/src/ansiblelint/schemas/ansible.json#/$defs/tasks": "roles/*/tasks/*.yml"
    },
    "editor.tabSize": 2,
    "editor.insertSpaces": true,
    "[yaml]": {
        "editor.tabSize": 2,
        "editor.insertSpaces": true,
        "editor.formatOnSave": true
    }
}

After selecting the interpreter, I open a new VS Code terminal (Ctrl+`` ). The virtual environment should activate automatically. I verify:

which ansible
which python3

# /home/ansible/venvs/ansible-network/bin/ansible
# /home/ansible/venvs/ansible-network/bin/python3

The .vscode/settings.json file should be committed to Git so the entire team gets the same VS Code configuration automatically. However, if some team members have the virtualenv at a different path (e.g., on macOS it would be different), the hardcoded path becomes a problem. A cleaner approach for cross-platform teams is to use a relative path or the ${workspaceFolder} VS Code variable and keep the virtualenv inside the project directory itself. For this guide’s single-user setup, the absolute path is fine.

Best Practices

Correct - separate environments for separate projects

~/venvs/ansible-network/       ← This guide's project
~/venvs/legacy-nornir-scripts/ ← Older project using different library versions
~/venvs/netbox-api-tools/      ← Standalone Netbox scripts

Wrong - one environment for everything

~/venvs/python-env/   ← All projects crammed in here, dependency chaos

The virtual environment directory is large (often 50–200MB), contains compiled binaries specific to my OS, and is completely regeneratable from requirements.txt. It must never go into Git.

I add it to .gitignore:

# .gitignore
venv/
venvs/
.venv/
env/

After setting up the environment and confirming everything works:

pip freeze > requirements.txt
git add requirements.txt
git commit -m "Pin Python dependencies for ansible-network environment"

Every project should have a README.md that tells a new engineer exactly how to recreate the environment:

## Setup

1. Clone the repository:
   git clone git@github.com:ernestodiaztech/ansible-network.git

2. Create and activate a virtual environment:
   python3 -m venv ~/venvs/ansible-network
   source ~/venvs/ansible-network/bin/activate

3. Install Python dependencies:
   pip install --upgrade pip
   pip install -r requirements.txt

4. Install Ansible collections:
   ansible-galaxy collection install -r collections/requirements.yml

5. Verify:
   ansible --version

Managing the Environment Over Time

Activate the environment:

source ~/venvs/ansible-network/bin/activate

Install the new package:

pip install some-new-library

Update requirements.txt immediately:

pip freeze > requirements.txt

Commit the change

git add requirements.txt
git commit -m "Add some-new-library for feature X"

Upgrade a specific package:

pip install --upgrade ansible

Check what changed:

pip freeze > requirements.txt
git diff requirements.txt

Commit if everything still works:

git add requirements.txt
git commit -m "Upgrade ansible from 9.7.0 to 9.8.0"

List outdated packages:

pip list --outdated

Upgrade all packages (use with caution):

pip install --upgrade $(pip freeze | cut -d= -f1)

Test everything still works before freezing:

ansible --version
ansible-lint --version

Then freeze and commit:

pip freeze > requirements.txt

If the virtualenv gets corrupted or I want a clean slate:

Deactivate first if active:

deactivate

Delete the entire environment directory:

rm -rf ~/venvs/ansible-network

Recreate from scratch:

python3 -m venv ~/venvs/ansible-network
source ~/venvs/ansible-network/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
ansible-galaxy collection install -r collections/requirements.yml

Verify:

ansible --version

This takes about 2-3 minutes and produces a perfectly clean environment. The virtualenv is truly disposable because everything needed to recreate it is in requirements.txt.

The environment is now fully set up and reproducible. Everything runs inside an isolated, documented, version-controlled Python environment.