My Data Journey
Personal reflections, lessons learned, and project deep-dives from a data engineer's career — tools, decisions, and the occasional hard truth.
- For quality work, understand your data grain first
Understand data grain to ensure accurate data analysis and avoid incorrect queries, joins, and duplicates in data processing
- Why how the data is collected matters
Explore the importance of data collection methods and their impact on analysis in retail, featuring loyalty programs and consumer patterns
- My day as a Data Engineer 8 years ago
A candid look at what data engineering looked like in 2016 — manual pipelines, Hadoop clusters, and the contrast with today's cloud-native, serverless...
- Breaking Free from Tutorial Hell: Tips for Aspiring Data Professionals
Endlessly consuming courses without building anything is a common trap for aspiring data professionals. Learn how to break the cycle with hands-on...
- Lessons from Chernobyl and how it relates to software engineering
The Chernobyl disaster resulted from misaligned incentives, silenced concerns, and flawed processes — patterns that also appear in software teams.
- Why does your Data Warehouse need to look more like a pharmacy than a retail store?
Uncontrolled self-service data access leads to conflicting metrics and siloed reporting. Data engineers should act as stewards, guiding consumers to...
- A portable Data Analytics stack using Docker, Mage, dbt-core, DuckDB and Superset
A containerized end-to-end data analytics stack using Mage, dbt-core, DuckDB, and Superset, all wired together with Docker Compose.
- Hands-on with DuckDB
A practical introduction to DuckDB as a lightweight OLAP database for local analytics, prototyping, and data warehouse proofs of concept — including...
- A portable data stack with Dagster, Docker, DuckDB, dbt and Superset
Build a fully containerized local data stack with Dagster, DuckDB, dbt Core, and Apache Superset using Docker Compose — a realistic, runnable data...
- 📢 Why Having a Pet Project Can Supercharge Your Programming Learning Journey! 🚀🖥️
Pet projects teach git, Docker, testing, and frameworks faster than tutorials alone. Learn how building a real side project can bridge the gap between...
- Preparing for the AWS Certified Data Analytics - Specialty (DAS-C01) Certification
Preparing for the AWS Certified Data Analytics - Specialty (DAS-C01) Certification
- Analyzing Reddit data using Scala, Spark and Spark-SQL
Learn how to read and query Reddit JSONL data using Scala, Spark, and Spark SQL with typed DataFrames. Run SQL joins and aggregations against millions of...
- Building a Business Intelligence suite: revisiting an old idea after six years in the industry
A walkthrough of building a containerized Business Intelligence suite using dbt, Airflow, PostgreSQL, and Superset.
- Preparing for the AWS Certified Developer-Associate exam
Reviews the best resources for passing the AWS Certified Developer Associate exam, from the official study guide to A Cloud Guru and Tutorials Dojo.
- 9 issues I’ve encountered when setting up a Hadoop/Spark cluster for the first time
A hands-on guide to 9 common Hadoop and Spark cluster setup errors, from JAVA_HOME not set to spark.yarn.jars. Each issue includes the exact fix that...
- How I set up my first Hadoop / Spark cluster: Preparation
A practical guide to building a Hadoop/Spark cluster on Raspberry Pi hardware, covering OS setup, SSH config, and cluster networking.