Python pip — Container ImportError from Global Packages

Every Python project you build beyond a single script will eventually depend on code someone else wrote — a web framework, a data parser, a testing library. The moment you install that code, you're participating in Python's packaging ecosystem, whether you realize it or not. Get it wrong and you end up with the classic 'it works on my machine' problem, broken deployments, and version conflicts that take hours to debug and are genuinely difficult to explain to a product manager.

The good news is that Python's packaging ecosystem has matured significantly. The chaos of setup.py, setup.cfg, and MANIFEST.in is giving way to pyproject.toml. pip has better dependency resolution than it did even a few years ago. Virtual environments are a two-command operation. The tools have caught up to the problems they were designed to solve.

This article covers the four things you actually need to know: how virtual environments work and why they're non-negotiable, how to manage dependencies reproducibly with requirements.txt and pip-compile, how to structure a project with pyproject.toml, and how to publish a package to PyPI without making an irreversible mistake. Every section comes from real production experience — I'll tell you what trips people up and why, not just what the commands are.

Why Python pip Can Leak Global Packages Into Your Container

Python pip is the default package installer for Python, pulling libraries from the Python Package Index (PyPI) and placing them into site-packages directories. Its core mechanic is dependency resolution: it reads a requirements file or CLI arguments, resolves version constraints, downloads wheels or source distributions, and installs them into a target environment. The critical detail is that pip installs into whatever Python interpreter it's bound to — system Python, a virtual environment, or a container image layer — and it does not isolate packages from the global site-packages unless explicitly told to.

In practice, pip's behavior depends on the environment's Python path. When you run pip install inside a Docker container without first creating a virtual environment, packages land in /usr/local/lib/python3.x/site-packages. If your container image inherits a base image that already has packages there (e.g., from apt-installed python3-requests), pip's installation can silently shadow or conflict with those global packages. The result: an ImportError that only appears in production because the container's global site-packages differ from your local virtualenv. Pip has no built-in guard against this — it's your responsibility to control the target.

Use pip with virtual environments (or pip install --target) whenever you need reproducible, isolated dependencies. In containerized deployments, always create a fresh virtual environment inside the image and activate it before installing. This prevents the 'works on my machine' class of bugs where a global package version mismatch causes runtime failures. Pip is a tool, not a policy — the isolation is up to you.

Why Virtual Environments Are Non-Negotiable

When you install a package globally with pip, it lands in your system Python installation. That sounds fine until Project A needs requests==2.28 and Project B needs requests==2.31. Python can only hold one version of a package at a time in a given Python installation, so one of your projects will always be broken. Virtual environments solve this by creating a completely isolated Python interpreter and its own package directory for each project. Every project gets its own private spice rack, and they never interfere with each other.

Creating one is a single command, but understanding what it actually does changes how you work. A virtual environment is just a folder containing a Python binary, a pip binary, and a site-packages directory. When you activate it, your shell's PATH is temporarily updated so that typing 'python' points to that isolated binary instead of your system one. When you deactivate, the PATH change is reversed. Nothing magical — just smart path management. The packages you installed in the venv are still there in the folder; they're just not on your active PATH anymore.

This also means you can have as many virtual environments as you want without them interfering with each other or with your system Python. Some developers create one per project (the most common approach), some create per-branch environments for complex multi-branch work. The cost is two commands and a few hundred megabytes of disk. The cost of not using them is dependency hell, broken deployments, and the specific kind of frustration that comes from code that works on your machine and nowhere else.

In 2026, you also have the option of uv — a Rust-based package manager and virtual environment tool from Astral — which creates environments and installs packages significantly faster than the traditional pip + venv combination. Whether you use python -m venv or uv venv, the conceptual model is identical.

# ── CREATING A VIRTUAL ENVIRONMENT ─────────────────────────────────────────
# The 'venv' module ships with Python 3.3+ — no separate install needed
# Always create inside your project folder so it's self-contained
python3 -m venv venv

# ── ACTIVATION ──────────────────────────────────────────────────────────────
# Mac/Linux: updates PATH to point to venv's Python and pip
source venv/bin/activate

# Windows Command Prompt:
# venv\Scripts\activate

# Windows PowerShell:
# venv\Scripts\Activate.ps1

# Your terminal prompt changes to show (venv) — visual confirmation you're isolated

# ── VERIFY YOU'RE IN THE RIGHT ENVIRONMENT ──────────────────────────────────
# Both should point INSIDE your project folder, not /usr/bin or /usr/local
which python
# Good: /home/user/myproject/venv/bin/python
# Bad:  /usr/bin/python3  (you're still global)

which pip
# Good: /home/user/myproject/venv/bin/pip
# Bad:  /usr/local/bin/pip  (wrong environment)

# ── INSTALL PACKAGES ────────────────────────────────────────────────────────
# Only lands inside this venv — completely invisible to other projects
pip install requests flask pytest

# Verify what's installed and where
pip list
# Shows ONLY packages in this venv, not global packages

# ── DEACTIVATION ────────────────────────────────────────────────────────────
deactivate
# PATH reverts. 'which python' now points to the system Python again.
# The packages you installed are still in venv/lib/ — they're just not active

# ── USING uv (faster alternative, Rust-based) ───────────────────────────────
# pip install uv  (or: curl -LsSf https://astral.sh/uv/install.sh | sh)
uv venv .venv          # creates a venv at .venv/ — significantly faster than python -m venv
source .venv/bin/activate
uv pip install requests  # 5-10x faster than pip for most packages

Pinning Dependencies with requirements.txt — The Right Way

A requirements.txt file is a plain-text manifest of every package your project needs to run. pip reads it line by line and installs everything specified. This is what makes your project reproducible across different machines, CI pipelines, teammates' laptops, and production servers — anyone who runs pip install -r requirements.txt gets an environment that behaves identically to yours.

There is a critical distinction that trips up most developers early on: direct dependencies versus transitive dependencies. Direct dependencies are packages you explicitly import in your code — requests, flask, pytest. Transitive dependencies are packages that your dependencies depend on — certifi, urllib3, click. When you run pip freeze, you get every single installed package at exact versions, including all the transitive ones. This is complete and deterministic, which is exactly what you want for application deployments. But it's too restrictive for libraries you publish for others to use, because pinning transitive dependencies to exact versions will almost certainly conflict with your users' other packages.

The professional workflow that solves both problems: create a requirements.in with loose version constraints for your direct dependencies only, then use pip-compile from the pip-tools package to resolve the full dependency graph and generate a deterministic requirements.txt with every transitive dependency pinned and annotated with which direct dependency caused it. This gives you a human-editable source of intent (requirements.in) and a machine-generated lockfile (requirements.txt). Commit both. CI installs from the pinned file. When you need to update dependencies, you edit requirements.in and re-run pip-compile — the diff shows you exactly what changed and why.

# ── APPROACH 1: pip freeze (simple, good for apps) ─────────────────────────
# Captures everything currently installed at exact versions
# Best for: application deployments where you need byte-for-byte reproducibility

pip freeze > requirements.txt
# Output includes direct AND transitive deps, all pinned:
# certifi==2024.2.2
# charset-normalizer==3.3.2
# idna==3.6
# requests==2.31.0

# ── APPROACH 2: Hand-written with ranges (good for libraries) ──────────────
# Only list your direct dependencies with reasonable version constraints
# Best for: libraries others will install alongside their own packages

cat > requirements.txt << 'EOF'
requests>=2.28,<3
flask>=3.0
pytest>=7.0
EOF

# ── APPROACH 3: pip-tools (best of both worlds, production standard) ────────
pip install pip-tools

# Create requirements.in — your direct dependencies with loose constraints
# This is the human-editable file you commit and maintain
cat > requirements.in << 'EOF'
requests>=2.28
flask>=3.0
pytest>=7.0
EOF

# pip-compile resolves the full dependency graph and generates requirements.txt
# --generate-hashes adds SHA256 hashes for supply chain security (optional but recommended)
pip-compile requirements.in
# or with hashes:
pip-compile --generate-hashes requirements.in

# pip-sync installs EXACTLY what's in requirements.txt AND removes anything not listed
# This guarantees your environment matches the lockfile exactly
pip-sync requirements.txt

# ── UPDATING DEPENDENCIES ───────────────────────────────────────────────────
# To update a specific package to a newer version:
pip-compile --upgrade-package requests requirements.in
# Updates requests and adjusts any transitive deps that change as a result

# To update everything to the latest compatible versions:
pip-compile --upgrade requirements.in

# ── CHECKING FOR SECURITY VULNERABILITIES ───────────────────────────────────
# pip-audit checks all installed packages against the Python Advisory Database
pip install pip-audit
pip-audit
# Reports any known CVEs in your installed dependencies

pyproject.toml — The Modern Way to Define a Python Package

For years, Python packaging was a mess of setup.py, setup.cfg, and MANIFEST.in files with inconsistent behaviour and no clear standard. PEP 517 and PEP 518 introduced pyproject.toml as the single unified configuration file for Python projects. Every modern project — whether you're publishing to PyPI or just organising an internal monorepo — should use it. If you're still using setup.py for a new project in 2026, you're taking on unnecessary legacy complexity.

The key insight that makes pyproject.toml important is what it changes architecturally: it separates 'build system requirements' (the tools needed to build your package) from 'project metadata' (what your package actually is). This lets pip know how to build your package before it has even installed your build tool. Previously, setup.py required setuptools to already be installed — a genuine chicken-and-egg problem. With pyproject.toml, pip reads the [build-system] table, creates an isolated build environment, installs the build tools declared there, and then builds. No pre-installed tools required.

The [project] table is where you declare your package name, version, dependencies, and entry points. Entry points are particularly useful in practice — they let you define command-line scripts that get installed alongside your package, so users can run your-tool directly in their terminal rather than typing python -m your_module.cli every time. If you've ever installed a package and gained a new CLI command, that's an entry point.

For build backends, the main options in 2026 are hatchling (part of Hatch, modern and fast), setuptools (the legacy standard, still widely used), flit (minimal, good for pure Python packages), and maturin (for Rust extensions). The choice of backend doesn't affect your package's users — they never see it. It only affects how your package gets built during python -m build.

# pyproject.toml — the single source of truth for your Python project
# Replaces setup.py, setup.cfg, and MANIFEST.in entirely

[build-system]
# Which tools are needed to BUILD this package
# pip downloads and uses these in an isolated temporary environment
# before building — no pre-installed tools required
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "weather-fetcher"              # The name users will 'pip install'
version = "1.0.0"                     # Follow semantic versioning: MAJOR.MINOR.PATCH
description = "Fetch weather data from public APIs with a clean interface"
readme = "README.md"                  # Displayed on PyPI project page
license = {file = "LICENSE"}
requires-python = ">=3.9"             # Minimum Python version your code supports
authors = [
    {name = "Your Name", email = "you@example.com"}
]
keywords = ["weather", "api", "cli"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "License :: OSI Approved :: MIT License",
    "Operating System :: OS Independent",
]

# Direct dependencies — use ranges, never exact pins
# These are installed automatically when someone 'pip install weather-fetcher'
dependencies = [
    "requests>=2.28,<3",
    "python-dotenv>=1.0",
]

[project.urls]
Homepage = "https://github.com/yourname/weather-fetcher"
Documentation = "https://weather-fetcher.readthedocs.io"
Changelog = "https://github.com/yourname/weather-fetcher/blob/main/CHANGELOG.md"
Issues = "https://github.com/yourname/weather-fetcher/issues"

# Optional dependency groups — users install with: pip install weather-fetcher[dev]
# Keep dev tools out of the main dependency list
[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "pytest-cov>=4.0",
    "black>=24.0",
    "mypy>=1.0",
    "ruff>=0.3",
]
docs = [
    "sphinx>=7.0",
    "sphinx-rtd-theme>=2.0",
]

# Entry points: registers a 'weather' command in the user's terminal after install
# Python maps 'weather' to: call the main() function in weather_fetcher/cli.py
[project.scripts]
weather = "weather_fetcher.cli:main"
# After 'pip install .': user can type 'weather London' in any terminal

# ── Project layout that matches this pyproject.toml ─────────────────────────
# weather-fetcher/
# ├── pyproject.toml
# ├── README.md
# ├── LICENSE
# ├── CHANGELOG.md
# └── src/
#     └── weather_fetcher/
#         ├── __init__.py
#         └── cli.py

# The src/ layout is recommended — it prevents accidental imports from the
# project root during development, ensuring you're always testing the installed package

# ── Local development install ────────────────────────────────────────────────
# pip install -e .        (install core only)
# pip install -e .[dev]   (install core + dev tools)
# The -e flag means 'editable' — edit source files and changes are immediately
# reflected without reinstalling the package

Publishing Your Package to PyPI — From Local to the World

Publishing to PyPI (Python Package Index) feels intimidating until you understand it's just two commands after a one-time setup. PyPI is the repository pip queries by default whenever you run pip install. Publishing there means anyone in the world can install your package with pip install your-package-name — no other configuration needed.

The build process produces two distribution formats: a wheel (.whl) — a pre-built binary that installs in milliseconds — and a source distribution (.tar.gz) that gets built on the user's machine. The wheel is the fast path that handles the vast majority of installations. The source distribution is the fallback for unusual platforms or architectures where no pre-built wheel is available. Always publish both.

Before publishing to the real PyPI, test on TestPyPI first. It lives at test.pypi.org, behaves identically to the real PyPI, and is completely separate — packages published there don't appear on the real PyPI and vice versa. You can upload, install, and verify the complete experience without any risk. This step is not optional: you cannot overwrite or delete a published version on the real PyPI. If you publish 1.0.0 with a broken README or a missing file, that version is permanently broken and you'll need to publish 1.0.1 to fix it.

For production-quality publishing workflows, use a GitHub Actions pipeline that triggers on version tag pushes. Store your PyPI API token as a GitHub Secret, use Trusted Publishers (PyPI's OIDC integration) to avoid token management entirely, and build in a clean environment every time. This removes the 'I published from my half-configured dev machine' class of problems entirely.

# ── PREREQUISITES ────────────────────────────────────────────────────────────
# Create accounts at:
#   https://test.pypi.org   (for testing)
#   https://pypi.org        (for production)
# Enable 2FA on both accounts — PyPI requires it for new publishers

pip install build twine

# ── STEP 1: BUILD ────────────────────────────────────────────────────────────
# Reads pyproject.toml and builds both wheel and source dist
# Creates files in the dist/ folder
python -m build
# Output:
#   dist/weather_fetcher-1.0.0-py3-none-any.whl   (wheel — fast install)
#   dist/weather_fetcher-1.0.0.tar.gz              (sdist — fallback)

# ── STEP 2: VERIFY before uploading ─────────────────────────────────────────
# Catches malformed README, missing metadata, and packaging mistakes
# before they become permanent PyPI entries
twine check dist/*
# Expected output: PASSED for both files

# ── STEP 3: TEST on TestPyPI — always do this first ─────────────────────────
twine upload --repository testpypi dist/*
# You'll be prompted for TestPyPI credentials or API token

# Verify the TestPyPI install works end-to-end:
pip install \
    --index-url https://test.pypi.org/simple/ \
    --extra-index-url https://pypi.org/simple/ \
    weather-fetcher
# --extra-index-url allows pip to find dependencies on real PyPI
# since test.pypi.org doesn't have all packages

# Test the entry point works:
weather London

# ── STEP 4: PUBLISH to real PyPI when satisfied ──────────────────────────────
twine upload dist/*
# After this, the package is live at https://pypi.org/project/weather-fetcher/
# Anyone can now: pip install weather-fetcher

# ── API TOKENS (strongly recommended over passwords) ─────────────────────────
# Go to pypi.org > Account Settings > API Tokens
# Create a token scoped to this specific project (not account-wide)
# Then pass it to twine:
export TWINE_USERNAME=__token__
export TWINE_PASSWORD=pypi-AgEIcHlwaS5vcmcA...  # your actual token
twine upload dist/*

# Or store in ~/.pypirc to avoid env vars:
# [pypi]
# username = __token__
# password = pypi-AgEIcHlwaS5vcmcA...

# ── AUTOMATING WITH GITHUB ACTIONS ──────────────────────────────────────────
# .github/workflows/publish.yml
# Triggers on: push of tags matching v*.*.* (e.g., v1.0.0)
# Steps: checkout → build → test on TestPyPI → publish to PyPI
# Use PyPI's Trusted Publishers (OIDC) to skip token management entirely
# See: https://docs.pypi.org/trusted-publishers/

pip vs pip3 vs pip2 — Why Your Container Just Broke at 3 AM

You type pip install requests and it works on your laptop. Your CI pipeline uses pip3 and also works. Then your colleague runs the same command on a legacy image and gets a wall of red. Welcome to Python's packaging identity crisis.

Python 2 is dead. But its ghosts live in enterprise Docker images pinned to Ubuntu 16.04. The binary pip might point to Python 2.7 or Python 3 depending on how Python was installed. pip3 always targets Python 3. pip2 targets Python 2 — and shouldn't exist in any image you deploy today.

The trap: inside a virtual environment, pip is symlinked to the Python version that created the venv. Outside a venv, you're at the mercy of your distro's symlink game. Alpine images ship pip for Python 3. Debian Buster defaults pip to Python 2 if both are installed.

Never rely on pip alone. Always verify with pip --version. Or better: standardise on python -m pip for every invocation. That guarantees you're using the pip associated with the Python interpreter you think you're using. No symlink roulette.

// io.thecodeforge — python tutorial

// Don't let this be your Friday night
$ which pip
/usr/bin/pip
$ pip --version
pip 20.0.2 from /usr/lib/python2.7/dist-packages/pip (python 2.7)

// What you wanted:
$ python3 -m pip --version
pip 21.3.1 from /usr/lib/python3.8/dist-packages/pip (python 3.8)

// Safe pattern for scripts and CI:
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Installing Packages from GitHub — The Dependency Hell You Code Yourself Into

Your startup's internal library isn't on PyPI. It lives on a private GitHub repo. Someone says 'just pip install from git'. Two months later, you can't reproduce the build and your CTO is asking why staging is broken.

pip can install directly from git URLs. That's not the problem. The problem is that pip install git+https://... pulls the latest commit from the default branch unless you pin a refspec. No tag? No commit hash? You just introduced a floating dependency into your production pipeline. Every deployment becomes a Schrödinger's build.

The fix is brutal but simple: always pin to a commit hash or a tag. If the repo doesn't tag releases, you need to have a hard conversation with the maintainers. Also: never put credentials in the URL. Use SSH keys or your CI's deploy token.

Another footgun: pip caches git clones in ~/.cache/pip. If your CI runner shares a home directory across builds (don't), you'll get stale clones. Always do pip install --no-cache-dir git+... in CI to force a fresh clone.

// io.thecodeforge — python tutorial

// DON'T — floating dependency
pip install git+https://github.com/yourorg/internal-auth.git

// DO — pin to commit hash
pip install git+https://github.com/yourorg/internal-auth.git@a1b2c3d4e5f6

// DO — pin to tag
pip install git+https://github.com/yourorg/internal-auth.git@v1.2.3

// requirements.txt format:
git+https://github.com/yourorg/internal-auth.git@a1b2c3d4e5f6#egg=internal-auth

// CI-safe install:
python -m pip install --no-cache-dir -r requirements.txt

UV vs. Poetry — Why Your Next Project Shouldn't Start with Poetry

Poetry was the savior from pip’s dependency hell. Now it’s the bottleneck. UV is a Rust-based drop-in replacement that installs packages 10-100x faster and resolves dependencies without melting your laptop. Poetry still works. UV makes you forget what pip install ever felt like.

The core difference: Poetry treats your project as a package by default. UV treats speed and correctness as the default. UV supports PEP 517/518 builds natively, has zero config for monorepos, and doesn’t force a pyproject.toml lock-in dance. Poetry’s lock file is fine. UV’s resolver is nuclear. If you’re starting a new project today and you aren’t running on a Raspberry Pi from 2015, pick UV.

Production reality: UV integrates with existing requirements.txt and pyproject.toml files. Poetry doesn’t play well with others. UV is what Poetry would be if it had been rewritten by people who debug production outages.

// io.thecodeforge — python tutorial

import subprocess
import time

# Simulate installing 'requests' + 'flask' in a fresh venv
# UV install
start = time.time()
subprocess.run(["uv", "pip", "install", "requests", "flask", "-q"], capture_output=True)
uv_time = time.time() - start

# Poetry install (requires poetry.lock generation)
subprocess.run(["poetry", "init", "-n", "--no-interaction"], capture_output=True)
start = time.time()
subprocess.run(["poetry", "add", "requests", "flask", "-q"], capture_output=True)
poetry_time = time.time() - start

print(f"UV: {uv_time:.2f}s")
print(f"Poetry: {poetry_time:.2f}s")

Can I Use UV With Existing pip requirements.txt Files? — Yes, And It's Stupid Easy

Short answer: yes. Long answer: you should have switched yesterday. UV reads requirements.txt as a native format. No conversion, no uv init ceremony. If your CI/CD pipeline is still crying over pip install -r requirements.txt taking 90 seconds, drop in UV and watch it finish in 4.

Here’s how it works: replace pip install -r requirements.txt with uv pip install -r requirements.txt. That’s it. UV respects the same --no-cache, --target, and --constraint flags. Your Dockerfile? Change one line. Your Makefile? One grep. Your soul? Restored.

Why this matters in production: UV’s resolver is parallelized and stateless. It doesn’t re-download wheels it already verified. If your requirements.txt has torch, tensorflow, or any binary-heavy package, UV will save you 3-5 minutes per build. That’s not nice. That’s do-or-die for deploy latency.

// io.thecodeforge — python tutorial

# Before (slow pip hell)
# FROM python:3.11-slim
# COPY requirements.txt .
# RUN pip install --no-cache-dir -r requirements.txt

# After (UV speed)
FROM python:3.11-slim
COPY requirements.txt .
RUN pip install uv && \
    uv pip install --system --no-cache -r requirements.txt

# Verify no pip leakage
RUN python -c "import requests; print('Deploy ready')"

UV vs. Conda — Why Your Python Environment Manager Might Be the Bottleneck

Conda is a full ecosystem manager — it handles Python, R, C libraries, and binary packages. That breadth comes with a cost: slow dependency resolution and bloated environments. UV, built in Rust, focuses exclusively on Python packages. It resolves dependencies 10–100x faster than Conda. Conda excels when you need non-Python binaries like CUDA drivers or OpenCV with system-level dependencies. UV dominates when you control your Python stack — CI pipelines, Docker images, or development environments. The tradeoff is clear: Conda for cross-language chaos, UV for Python purity. If your project doesn't need Conda's compilers or channels, UV removes the slowness without sacrificing lockfile guarantees. Switching from Conda to UV cuts environment setup from minutes to seconds. Your container builds will thank you.

// io.thecodeforge — python tutorial

# Conda: full ecosystem, but slow
# conda create -n myenv python=3.11 numpy pandas
# conda activate myenv

# UV: Python-only, lightning fast
# uv venv myenv --python 3.11
# uv pip install numpy pandas

# UV resolves in seconds, Conda in minutes
# Use Conda only for non-Python deps

What UV Gives You That Poetry and Conda Don't — Raw Speed and Zero Magic

Poetry adds lockfile management and dependency resolution with a promise of determinism, but hides too much behind magic — you fight it when you need pip-compatibility or custom index URLs. Conda solves cross-language dependencies but is painfully slow and forces you into its channel ecosystem. UV breaks both bottlenecks. It's pip-compatible out of the box — your existing requirements.txt and pyproject.toml work without changes. Resolution happens in milliseconds, not minutes. UV also avoids Poetry's lockfile formatting battles and Conda's environment sprawl. The killer advantage: UV is a single binary with no Python runtime dependency to install itself. Download it, run it. No more 'pip install poetry' bootstrapping or Conda init scripts. For teams shipping containers or CI workflows, UV removes the friction Poetry and Conda introduce.

// io.thecodeforge — python tutorial

# Poetry: Python-dependent, slow, lockfile inflexible
# pip install poetry && poetry install  # 10s+ bootstrapping

# Conda: cross-platform, but 2-5x slower resolution
# conda env create -f environment.yml

# UV: zero bootstrap, pip-compatible, instant
# curl -LsSf https://astral.sh/uv/install.sh | sh
# uv sync  # uses existing pyproject.toml