Using virtual environments in Python

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Concise guides to Python features and patterns most useful for data engineering — from built-ins to standard library modules.
Here are the basics you need to know about installing Python packages. Apart from the built-in modules that come by default with the Python installation (the Standard Library), you need to install third-packages packages to be able to use them in you...
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

title: "Using virtual environments in Python" seoTitle: "Python Virtual Environments: A Guide" seoDescription: "Learn how to use Python virtual environments to manage project dependencies effectively and avoid conflicts with this comprehensive guide" datePublished: Sun Jul 28 2024 20:14:25 GMT+0000 (Coordinated Universal Time) cuid: clz600vjp000909l8dbk8e9i2 slug: using-virtual-environments-in-python cover: https://cdn.hashnode.com/res/hashnode/image/stock/unsplash/9_eMsFKGJiM/upload/e2c91d8eba015021565e0e3bae4bc359.jpeg tags: programming-blogs, python, virtual-environment
In my yesterday's post, we looked at the basics of installing packages in Python using pip.
The next immediate step is to learn about virtual environments. When you install packages with pip, they are added to your 'global' library.
However, if you have multiple Python projects on your machine, each with different package requirements and dependencies, managing these packages globally can become a complex task.
A solution to this problem is to use virtual environments. A virtual environment is a lightweight, isolated Python installation specific to each project (read: folder), with its own list of installed packages.
This way, Project A can use version 1.0 of CoolPythonModule while Project B uses version 2.0, without conflicts.
There are multiple tools for managing environments: conda or poetry also do that, but one of the simplest options is venv.
To create a virtual environment, use the following command:
python3 -m venv .venv
# This creates a virtual environment in the '.venv' folder inside the current working directory
You can now activate the virtual environment:
source .venv/bin/activate
The name of the virtual env is displayed next to your user name. You can now install and do everything you need in terms of packages - they will after only this virtual env.
If you'd like to deactivate it, you do:
deactivate
Don't know which Python interpreter you're using?
which python
This will show you the path of the active interpreter.
P.S. Activate/deactivate scripts can do interesting things like assigning an environment variable upon activation.
Found it useful? Subscribe to my Analytics newsletter at notjustsql.com.
Enjoyed this? Here are some related articles you might find useful: