Bigeye metrics are calculated by running a query on the specified column. By default, these queries include all rows in the table each metric run. You can optimize these queries by setting a Row Creation Time for tables monitored by Bigeye.
Setting a Row Creation Time enables Bigeye metric queries to filter to newly added or updated rows only. This improves query performance, and enhances Bigeye’s anomaly detection by windowing the data. Further, setting a Row Creation Time allows you to backfill a metric’s history which enables Autothresholds immediately without a training period.
Row Creation Time is recommended for incremental tables, or data tables that append new rows each update. Using Row Creation Time and a Lookback Window filters the rows queried each metric run to the most recent data. This optimizes the query performance and improves anomaly detection in new data.
Row Creation Time is not recommended for full refresh tables, or data tables that truncate and reload entirely each update.
- Choose a Row Creation Time column
Row Creation Time is a table-level setting. You can view and set the Row Creation Time from the table page in the catalog, all valid timestamp columns are available as options. You should select the column that represents the time the row was added or updated to the table.
Alternatively, when enabling Autometrics, if you have not yet set Row Creation Time for selected data tables, you will be prompted to choose a valid timestamp column. You should select the column that represents the time the row was added or updated to the table. Note that if your table is a full refresh table, we recommend that you select “none” here.
Once a Row Creation Time is set for a table, all metrics created on the table will be windowed by the selected column. Existing metrics on the table will also have the window applied.
- Configure the Lookback Window (Optional)
The Lookback Window, which determines which rows are included in each metric query, is a metric-level setting that can be adjusted in the metric edit page. The default is 2 days. We recommend this default lookback for most cases to catch any late arriving data and data updates.
- Advanced Options (Optional)
You can further configure metric window definitions by choosing one of the following window types in the metric edit page: data time, data time window, or clock time. See here for more information on window types.
Updated 4 months ago