Python for Data Engineers
Concise guides to Python features and patterns most useful for data engineering — from built-ins to standard library modules.
- Using tempfile module in Python
Use Python's tempfile module for creating and managing temporary files and directories with context managers for automatic cleanup
- A quick look at the json module in Python
Learn how to handle JSON in Python using the json module for encoding, decoding, and pretty-formatting
- Retrying in Python using tenacity
Learn how to simplify retry logic in Python using the tenacity package for handling exceptions and implementing back-off strategies
- sorted(List) vs List.sorted() in Python
Learn the key differences between sorted() and List.sort() in Python, crucial for effective data handling and management
- Using virtual environments in Python
Learn how to use Python virtual environments to manage project dependencies effectively and avoid conflicts with this comprehensive guide
- Installing Python packages with pip
Learn how to install Python packages with pip, use requirements files, install from GitHub, and upgrade or uninstall packages.
- Using zip in Python
Python's zip() lets you iterate over multiple iterables simultaneously, pairing elements into tuples. It stops at the shortest iterable, making it a...
- Decorators in Python
Learn how Python decorators work using the @ pie notation to modify function behavior without changing function structure.
- Partial functions in Python
Learn how to use functools.partial in Python to create specialized versions of existing functions by pre-filling one or more arguments.
- Comprehensions in Python
A concise guide to list, dictionary, set, and generator comprehensions in Python with practical examples. Learn when comprehensions improve code...
- Dunder / magic functions in Python
Explains Python dunder (magic) methods like __init__, __repr__, __eq__, and __lt__ with a hands-on class example.
- Context Managers in Python
Learn what Python context managers do, why data engineers need them for safe resource handling, and how to implement __enter__ and __exit__ in a custom...
- Using Enums in Python
Learn how Python Enum classes improve code readability, type safety, and maintainability by replacing scattered magic values with centrally defined named...
- Python Showdown: Namedtuple vs SimpleNamespace vs DataClass
Compare Python's three lightweight data containers — namedtuple, SimpleNamespace, and dataclass — on mutability, memory, and verbosity to pick the right...
- Understanding Generators in Python 🐍
Understand how Python generators work using yield, why they are memory-efficient for large datasets, and when to use them over lists in data engineering...
- Structural Pattern Matching in Python
Explore the Python 3.10 match statement for structural pattern matching, how it compares to if-elif-else chains, and where it handles complex conditions...
- Using TUPLES in Python
Compare Python tuples and lists on mutability, memory use, and valid use cases. Covers when to choose tuples as dictionary keys and for protecting data...
- Using SETS in Python
A concise guide to Python sets covering uniqueness, O(1) membership testing, and set operations. Learn when sets outperform lists and where their...
- Dictionary Unpacking in Python
Learn how to use Python's ** operator to unpack dictionaries as function keyword arguments, with a practical API call templating example that keeps config...
- A few thoughts about recent Python integration into Excel
A practical take on Microsoft's Python integration in Excel, covering the analytical possibilities it unlocks and the governance risks teams should...
- 6 scenarios I’ve encountered when using SQLAlchemy in my Flask project
Covers six practical SQLAlchemy patterns in a Flask app: many-to-many relationships, slug generation, object comparison, event-based seeding, and...
- Analyzing Twitter data using Python
Learn to explore Twitter Data Using Python tools including nltk, matplotlib and pandas
- An Intro to NLP and Sentiment Analysis using Python
Follow along this tutorial to find out how to Transform Twitter data using Python and Pandas
- Extracting Twitter data using Python
Pull tweets with Python: set up Twitter developer credentials, search tweets with the TwitterSearch package and flatten the JSON into a pandas DataFrame.