Monitor Data Replication Accuracy with Bigeye Deltas

Let's say that you're performing a data replication, creating a copy of Table A in Table B. But how can you continually assure that the data replication is performed correctly and the two copies actually match?

With Bigeye you can use the Deltas feature to compare the contents of two tables and determine if any differences exist.

1. Login to your Bigeye account and on the top menu click the Deltas tab. Then click New Delta.

2. Create your Delta

a. Select your sources and tables.

Let us say we are replicating the demo prod customer db into a standard demo customer db table. To create a Delta over these tables, select the sources and the names for both these tables from the dropdowns.

b. Edit column mappings, add filters, add Group By. (optional)

Since we are replicating the tables, Deltas will map the columns between the tables automatically based on the column names. You can make adjustments if required.

You can also add filters to exclude certain rows from the Delta comparisons and add Group Bys to understand column differences at a finer granularity.

c. Name your Delta

You can give the Delta a representative name like Table A Replication Check.

d. Add a schedule to run your Delta (optional)

By default, the Delta will run once and give you the comparison results. If your replication is a long process and you want to run the Delta checks periodically, you can also set a schedule to run it.

You can find more details about all of these steps in creating a Delta in Bigeye here.

After filling in all the details, click Create Delta.

3. Monitor the accuracy of the data replication using the created Delta.

Find the name of the Delta you just created in the list on the Deltas page. If you click on the name of a Delta you will see the details of the most recent run.

The header will give you a summary of details like the % match of row count between the tables you are trying to replicate and the number of column pairs where the metrics do not match.

The list shows each column pair, the metric, the metric value from each column, and the difference. If there is a difference, you will receive an alert. If the metric failed to run, it will show as Failed.

You can find more details about viewing Delta results here.