Using GAPS_FILL in BigQuery

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Short, practical posts on SQL and BigQuery — from core language features to advanced query patterns. A reference for data practitioners at every level.
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

Another new time series function in BigQuery in addition to the ones previously presented is the GAPS_FILL table-valued function.
It allows us to fill in a time series (DATE, DATETIME, TIMESTAMP) with missing rows to a desired time grain.
Previously, one could have solved this by joining with a date dimension or by using GENERATE_DATE_ARRAY for example.
It's simpler now, you just need to provide:
- the table you'd like to fill in
- the column you'd like to fill in (for example a DATE column)
- the interval you'd like the filling in to happen (time grain of the table)
In the example below, it allows us to fill in the time series with 2 missing days.
Since it's a table-valued function, it acts like a table so you select FROM it.
Obligatory remark that this is in 'Preview' for now.

Found it useful? Subscribe to my Analytics newsletter atnotjustsql.com.