Using the bq CLI utility with BigQuery

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Short, practical posts on SQL and BigQuery — from core language features to advanced query patterns. A reference for data practitioners at every level.
If you filter the data in SQL with WHERE using multiple logical conditions tied with AND & OR, PLEASE use the parentheses them accordingly. Because if you don't, your fellow team members are going to have a harder time understanding your intent. You ...
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

Remember that in additional to the Cloud Console GUI or client libraries, you can also perform BigQuery tasks using the bq command-line interface.
It's included with the gcloud SDK and once you have logged in, you can perform a number of operations such as:
- run queries against BQ
- create, copy and delete tables
- load data into tables
- view IAM policies for a BQ asset and much more
How I've used it until now:
- impersonate service accounts
- perform operations that are/were not supported in the SQL interface, such as copying partitions between tables
Any other interesting ways you're using the bq cli tool?
Found it useful? Subscribe to my Analytics newsletter at notjustsql.com.
Enjoyed this? Here are some related articles you might find useful: