Skip to content
Free since 2015. Built for the engineers who keep things running.

cat /articles/data-flow-tutorial-dealing-with-bigquery-schema-changes.md

Cloud

Data Flow Tutorial: Dealing With BigQuery Schema Changes

Imagine you have a pipeline that writes data from a PubSub/Kafka instance, or any other source, really, to BigQuery, and everything streams well.  Then, suddenly, you learn that your source JSON event had changed, and a new field should be added.

dzone.com13 Mar 2019 · summary

article --summary

The summary

We walk through a tutorial on how to use customize your BigQuery data schema in order to deal with changes in your data flow and streaming requirements.

Editorial note: This is a summary of an article first published by dzone.com. The full piece is on their site.

articles --latest

Latest articles

New from the blogs we follow and the community.

All articles

newsletter --weekly

DevOps Weekly, minus the noise.

The best reads, new jobs and upcoming events, in one email a week.