The JSON datatype in BigQuery

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Working with JSON in BigQuery — the native JSON datatype, extracting and transforming JSON data, LAX conversion functions, and handling semi-structured payloads.
In one my previous posts, we've briefly introduced the JSON datatype in BigQuery. But did anyone notice how most of the JSON functions have signatures for both a JSON-type input and a json-formatted string input? What is the difference between the tw...
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

The JSON datatype in BigQuery. This topic has been sitting in my Notion list of post ideas for some time.
So in our beloved BQ it is a native, standalone data type, not just another STRING 😁, although strings can of course hold json-like strings.
As data engineers we typically consume them in our pipelines, but let's first understand how to create them.
There are a couple of ways to express a JSON value:
- using the JSON literal
- by parsing a json-like STRING with JSON_PARSE()
- from SQL objects (including an entire row) with TO_JSON()
- creating a json_object from key-value pairs with JSON_OBJECT()
- creating a JSON_ARRAY() from BQ ARRAY
Note that, for some of the above options, since JSON also have quotes, use multi-line strings """ """ or escape quotes with \.
Defining our JSON objects as such will allow us to use JSON functions with them and, of course, store heterogeneous data in the same column.
Stay tuned for the next posts on this topic.
Found it useful? Subscribe to my Analytics newsletter at notjustsql.com.
Enjoyed this? Here are some related articles you might find useful: