Schema Change Detection

In the Bigeye catalog, you can see when your sources, schemas, and tables have last been modified. This helps you trace and track schema changes that may have impacted downstream tables or BI reports.

Schema change detection is available for all supported data sources including Snowflake, BigQuery and Redshift data warehouse sources.

The All sources Catalog page lists the last schema change for each source. Each level contained within has a last schema change time as well as a Schema Change tab which lists all the schema change events in the elements it contains.

At the source level, the schema change tab lists adds and removes of schemas, tables, views, columns, as well as column type changes. At the schema level, the schema change tab lists adds and removes of tables, views, columns, as well as column type changes. At the table level, the schema change tab lists adds and removes of columns as well as column type changes. At the column level, the schema change tab lists when it was added as well as any column type changes.

Each level also lists the initial schema change time, last detected schema change time and last schema scan. If the last schema change is after the latest schema scan, you can trigger a rescan at that level to get any more recent schema updates.

Any column adds, removes, or type changes will have the new and previous types listed.

When a column delete is detected, any associated metrics and their histories are dropped.

When a column type change is detected, existing metrics and their histories remain. The metrics that are no longer applicable will fail on their next run attempts.

📘

The only type changes captured are fundamental type changes such as boolean to string or string to numeric. Bigeye does not capture differences in precision. For example, a varchar(20) altered to varchar(50) will not be presented as a schema change in Bigeye.