Using SETS in Python

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Concise guides to Python features and patterns most useful for data engineering — from built-ins to standard library modules.
One of the most versatile aspects of Python dictionaries is the ability to unpack them. Ever wondered how to use dictionary values as function arguments? Simply employ the ** operator**: def greet(name, age): return f"Hello {name}, you're {age} y...
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

Using the right tool for the job is central to programming. This holds when considering what data structure to use for a particular problem.
Python offers a variety of built-in data structures, and one of the most versatile among them is the set. Let's explore its strengths, weaknesses, and ideal use cases.
✅ Strengths:
- Uniqueness: Sets automatically remove duplicates, ensuring each element is unique. You can convert a list into a set and back to a list to obtain a list of unique values in the original list.
- Fast Membership Testing: Checking if an item is in a set is much faster, typically constant time O(1), than in a list O(n).
- Set Operations: Supports union, intersection, difference, and more, making it powerful for mathematical operations.
❌ Weaknesses:
- Unordered: Sets do not maintain the order of elements.
- Immutable Elements: Only hashable (immutable) elements can be added to a set. So, lists can't be set elements, but tuples can.
- No Indexing: You can't access or modify an element of a set using an index or key.
🤔 When to Use:
- When you need to ensure elements are unique.
- When you want to perform set operations like union, intersection, etc.
- When you require fast membership testing.
# Creating a set
vegetables = {"eggplant", "cucumber", "squash"}
# Adding an element
vegetables.add("potato")
# Test membership
print("potato" in vegetables) #True
# Removing duplicates from a list
unique_nums = set([1, 2, 2, 3, 4, 4, 5])
print(unique_nums) # {1, 2, 3, 4, 5}
# Set operations
a = {1, 2, 3}
b = {3, 4, 5}
print(a.union(b)) # {1, 2, 3, 4, 5}
In conclusion, the set container is a powerful tool in Python's arsenal. While it has its limitations, understanding when and how to use it can greatly enhance your coding efficiency!
👉 Follow me for more insights on Python, SQL and analytics!
Found it useful? Subscribe to my Analytics newsletter at notjustsql.com.