Extracting keys from JSON in BigQuery

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Working with JSON in BigQuery — the native JSON datatype, extracting and transforming JSON data, LAX conversion functions, and handling semi-structured payloads.
The JSON datatype in BigQuery. This topic has been sitting in my Notion list of post ideas for some time. So in our beloved BQ it is a native, standalone data type, not just another STRING 😁, although strings can of course hold json-like strings. As...
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

A couple of months ago, I've posted about dynamically extracting key-value pairs from JSON in BigQuery SQL which leveraged regex (check comments).
Shortly after that post, we've gotten a new built-in function to dynamically extract the keys occurring in a JSON. It allows us to retrieve all the keys occurring in a JSON value, with a few controls on how this is done.
The function is JSON_KEYS. Apart from the json input, we can tweak:
- max_depth: for many levels of nesting we should go through to extract keys
- mode: strict/lax/lax recursive - controls if we extract keys from arrays.
The usual note - still in preview.
Found it useful? Subscribe to my Analytics newsletter at https://www.notjustsql.com.
Enjoyed this? Here are some related articles you might find useful: