A quick walkthrough BigQuery Remote Functions

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
Search for a command to run...

Senior Data Engineer • Contractor / Freelancer • GCP & AWS Certified
No comments yet. Be the first to comment.
Hands-on guides to data tooling, infrastructure, and operations — scheduling, pipelines, access control, and the engineering behind data workflows.
Google Cloud Workflows is a serverless orchestration platform that allows us to combine services into repeatable and observable sets of actions, connecting typically other GCP services. These are called, you guessed it, workflows. While working as a ...
Here's a useful Dataform concept: pre_operations and post_operations. As the name implies, these represent a set of actions that run before and after the main operation (table, view, or SQL operations

BigQuery has always been a SQL engine for tabular data. Object tables add an interesting twist to that. Instead of rows containing values, an object table gives you one row per file — pointing at da

Query your data lake with warehouse-grade security and performance — without moving a single file.

Ever run a heavy BigQuery SQL query, processed gigabytes of data — and then accidentally closed the tab or forgot to save the results? 😬 Don't re-run it. Your results are still there. BigQuery automa

You can use query parameters in BigQuery hashtag#SQL (now in the console as well!) — but how are they different from variables, and when should you use each? Both parameters and variables act as place

In a previous post, I mentioned Remote Functions—a powerful way to send data from BigQuery to an external service for processing, including a Cloud Run function.
This is especially useful when SQL lacks built-in support for your specific needs, and writing a UDF isn’t an option (for example, if you need a highly specialized Python function).
Before building your own, check out bigfunctions—many common use cases have already been solved by others!
These are a special type of function that delegates processing of input to an external resource, allowing us to:
send data from BigQuery to Google Cloud Functions or other external services
process it using a programming language
return results to our query
A Google Remote function can encapsulate any kind of logic in major programming languages.
This opens the door to vast a ecosystem of libraries such at the Python packages.
Let’s look at a step-by-step example of creating a Remote Function.






Found it useful? Subscribe to my Analytics newsletter at notjustsql.com.