Row Creation Time

Bigeye metrics are calculated by running a query on the specified column. By default, these queries include all rows in the table each metric run. You can optimize these queries by setting a Row Creation Time for tables monitored by Bigeye.

Setting a Row Creation Time enables Bigeye metric queries to filter to newly added or updated rows only. This improves query performance, and enhances Bigeye’s anomaly detection by windowing the data. Further, setting a Row Creation Time allows you to backfill a metric’s history which enables Autothresholds immediately without a training period.

When to use Row Creation Time

Row Creation Time is recommended for incremental tables, or data tables that append new rows each update. Using Row Creation Time and a Lookback Window filters the rows queried each metric run to the most recent data. This optimizes the query performance and improves anomaly detection in new data.

Row Creation Time is not recommended for full refresh tables, or data tables that truncate and reload entirely each update.

How to set Row Creation Time

Choose a Row Creation Time column

Row Creation Time is a metric-level setting. You can set Row Creation Time while deploying new metrics or from configuration page in the individual metric; all valid timestamp columns are available as options. You should select the column that represents the time the row was added or updated to the table.

At the table level, you can now set the "default" for that table. The default will not override existing metrics in that table, but will be the default Row Creation Time for all new metrics created within that table.

Alternatively, when enabling Autometrics, the Row Creation Time will default to what you've set for that table. If you haven't set a row creation time, it will default to none. You should select the column that represents the time the row was added or updated to the table.

Once a Row Creation Time is set for a metric, that metric will be windowed by the selected column.

Configure the Lookback Window (Optional)
The Lookback Window, which determines which rows are included in each metric query, is a metric-level setting that can be adjusted in the metric edit page. The default is 2 days. We recommend this default lookback for most cases to catch any late arriving data and data updates.

Advanced Options (Optional)

You can further configure metric window definitions by choosing one of the following window types in the metric edit page: data time, data time window, or clock time. See here for more information on window types.

Use Custom Timestamp Format to convert string columns into timestamps and run metrics as if they were actual timestamps. When the user provides a custom timestamp format in the advanced settings, the application identifies the text column as a timestamp column, runs all the timestamp column metrics on it, and sets it as the row creation time column for the table. Custom timestamp format setting is available only for Snowflake instances.