It’s easy to get started with Bigeye. Take control of your data pipelines today.

Overview

1. Connect to your source(s) quickly and securely
2. Deploy Autometrics for instant coverage of your most important data assets
3. Create SLAs to organize and optimize your alerts
4. Triage Issues to resolve data problems before they impact your business
5. Explore advanced features to ensure even your business-specific data requirements are monitored

1. Connect your source(s)

You can find our supported sources and instructions to connect each type under Data Sources. In general, we recommend creating a new service account and provisioning it only with the required permissions for the data you wish to monitor with Bigeye. To add a new source, go to Catalog > Add source. Note only admin users will be able to add new sources.

For some sources (Snowflake, BigQuery, Redshift), you will have the option to deploy Metadata Metrics right away. Metadata Metrics ensure every table is monitored for “time since last load” and “number of rows inserted”. This ensures you have broad coverage monitoring for the most common data quality problems.

Once your source is connected, Bigeye will automatically profile your data in order to recommend Autometrics, or suggested metrics to ensure your data is consistently accurate and reliable. To learn more about the heuristics used to recommend Autometrics, see Available Metrics.

2. Deploy Autometrics

Autometrics are suggested metrics that monitor anomalies in column-level data. Autometrics are automatically generated for all new datasets added to Bigeye. You can find Autometrics in the Catalog when viewing a source, schema, table or column page—simply navigate to the Autometrics tab. You can search, or use filters, to find metrics relevant to your monitoring goals. Select the relevant Autometrics, and click “enable”. By default, Bigeye metrics use Autothresholds to detect anomalies in your data and send alerts, so there’s no need to manually define thresholds. You have the option to configure metrics by setting a row creation time, will backfill metric history and enable Autothresholds without a training period.

3. Create SLAs & configure notifications

SLAs group relevant metrics together to consolidate statuses, notifications, and issues for data owners and stakeholders. For example, you may wish to group all metrics on ingest tables together in an ingest SLA, all metrics for a specific pipeline in a pipeline SLA, or all metrics on a dashboard reporting table in a dashboard SLA.

To add metrics to an SLA, select “metrics” in the catalog and click “action” > “Add to SLA”. Here you can create a new SLA or select an existing SLA.

On the SLA page you can view the status of all metrics in the SLA and triage related issues. We recommend setting notifications, via either Slack or email, on an SLA.

4. Triage issues

When Bigeye metrics detect values outside of the expected threshold, an Issue is created. Issues help your team triage and resolve data quality problems.

If you have notifications configured on a metric, or any SLA that metric is a part of, you will receive a notification when an Issue is created with a link. Alternatively, you can triage all issues from an SLA page, any catalog page, or the Issues tab in the Bigeye app, depending on what data is relevant to you.

You can acknowledge, mute, or close issues to track progress on resolution. Choosing to acknowledge an issue will automatically mute it for 24 hours.

5. Advanced features

Once you’ve setup Bigeye metrics and SLAs on your priority datasets, you can explore Bigeye’s advanced features to ensure even business-specific data requirements are monitored.

Take advantage of virtual tables and template metrics to monitor business rules. Setup deltas to monitor data migrations between sources or changes between dataset versions.