Connect Snowflake

Connect your Snowflake source to Bigeye.

1. Allow Bigeye's IP Address

Bigeye will make calls to your warehouse from one of the following static IPs:

35.163.65.120

By default, Snowflake instances are open to every IP address, so no action is required. However, if you've set up network policies to restrict the IP addresses communicating with your Snowflake instance, modify these policies to allow the Bigeye IP address.

There are two types of network policies you might have set up on your Snowflake instance: account level and user level. Account-level policies apply to all users unless a user-level policy overrides the policy. User-level policies only apply to the users to whom the policy is assigned, and user-level policies override account-level policies.

If you’ve configured account-level network policies, you must edit your network policy to allow incoming connections from Bigeye's IP address. Refer to the Snowflake documentation for more information on modifying account-level network policies.

If your organization uses user-level network policies, create a new policy to apply to the read-only user you create, allowing incoming connections from Bigeye's IP address. Refer to the Snowflake documentation for more information on modifying user-level network policies.

2. Connect to Snowflake

To follow these commands in Snowflake, open your Snowflake Worksheets page to run the statements below in the console. Only a user with the required permissions can run these commands, for example, ACCOUNTADMIN or SYSADMIN.

2.1 Define variables

-- set variables for use during creation - note that these must be ALL CAPS except password
set role_name = '<BIGEYE_ROLE>';
set user_name = '<BIGEYE_USER>';
set user_password = '<BIGEYE_USER_PASSWORD>';
set warehouse_name = '<BIGEYE_WAREHOUSE_NAME>';

2.2 Create a warehouse

Create a dedicated warehouse in Snowflake for Bigeye with the following objectProperties:

CREATE WAREHOUSE identifier($warehouse_name) WITH WAREHOUSE_SIZE='X-SMALL' AUTO_SUSPEND=60;

To learn more about creating a warehouse, see Create Warehouse in Snowflake documentation.

2.3 Create a Role for Bigeye

Create a dedicated, read-only role for Bigeye and grant permissions on the warehouse and tables you want Bigeye to monitor.

-- create role for Bigeye
create role if not exists identifier($role_name);
grant imported privileges on database "SNOWFLAKE" to role identifier($role_name);

-- grant Bigeye role access to warehouse
grant USAGE on warehouse identifier($warehouse_name) to role identifier($role_name);

-- TODO - repeat for every database, schema, table you want 
-- grant Bigeye access to data that you want to monitor
grant USAGE on database <database> to role identifier($role_name);
grant USAGE on schema <database>.<schema> to role identifier($role_name);
grant select on future tables in schema <database>.<schema> to role identifier($role_name);
grant select on all tables in schema <database>.<schema> to role identifier($role_name);
grant select on future views in schema <database>.<schema> to role identifier($role_name);
grant select on all views in schema <database>.<schema> to role identifier($role_name);

2.3.1 Grant Bigeye role permissions to SYSADMIN - Optional but recommended

Snowflake recommends creating a role hierarchy where lower roles have their permissions granted to higher roles. We recommend granting the Bigeye role to SYSADMIN so that SYSADMIN always has permission to do anything the Bigeye role does. Note that this does not grant SYSADMIN to the Bigeye role.

grant role identifier($role_name) to role SYSADMIN; 

2.4 Create a User for Bigeye

create user if not exists $user_name
password = $user_password
default_role = $role_name
default_warehouse = $warehouse_name
type = 'LEGACY_SERVICE';

grant role identifier($role_name) to user identifier($user_name);

-- flush everything we just did
commit;

2.5 All of the commands from above in a single copy/paste block

-- set variables for use during creation - note that these must be ALL CAPS except password
set role_name = '<BIGEYE_ROLE>';
set user_name = '<BIGEYE_USER>';
set user_password = '<BIGEYE_USER_PASSWORD>';
set warehouse_name = '<BIGEYE_WAREHOUSE_NAME>';

-- create warehouse
CREATE WAREHOUSE identifier($warehouse_name) WITH WAREHOUSE_SIZE='X-SMALL' AUTO_SUSPEND=60;

-- create role for Bigeye
create role if not exists identifier($role_name);
grant imported privileges on database "SNOWFLAKE" to role identifier($role_name);

-- grant Bigeye role access to warehouse
grant USAGE on warehouse identifier($warehouse_name) to role identifier($role_name);

-- TODO - repeat for every database, schema, table you want 
-- grant Bigeye access to data that you want to monitor
grant USAGE on database <database> to role identifier($role_name);
grant USAGE on schema <database>.<schema> to role identifier($role_name);
grant select on future tables in schema <database>.<schema> to role identifier($role_name);
grant select on all tables in schema <database>.<schema> to role identifier($role_name);
grant select on future views in schema <database>.<schema> to role identifier($role_name);
grant select on all views in schema <database>.<schema> to role identifier($role_name);

-- grant the Bigeye role to SYSADMIN (optional but recommended)
grant role identifier($role_name) to role SYSADMIN; 

-- create the user
create user if not exists identifier($user_name)
password = $user_password
default_role = $role_name
default_warehouse = $warehouse_name;

grant role identifier($role_name) to user identifier($user_name);

-- flush everything we just did
commit;

3. Gather connection details from Snowflake

Gather the following connection details from Snowflake

4. Add Snowflake as a data source in Bigeye

From the Catalog page, click Add source. On the modal that appears, select Snowflake.

On the Configure source modal that opens, enter the following details:

📘

Check Roles

The username will use its default role in Snowflake. Please make sure that role has the permissions it needs to see and query your tables.

5. Test connection

The system queries your warehouse to ensure the user you provided has the required permissions to read data and run all Bigeye metric types. If the test connection returns an error, return to Step 2: Create a user for Bigeye and verify that the user has all the required permissions in Snowflake.

6. Add metrics

If the test connection is successful, you will have the option to deploy Metadata Metrics on your schemas. We recommend broad coverage of these metrics across your warehouse to detect the most common data quality issues.

📘

Metadata-Based Freshness and Volume

Metadata-based Freshness and Volume are only available if you are using the Enterprise Edition of Snowflake and not on Snowflake Standard Edition. If you are on Snowflake standard, you must use column-based Row Count and Hours Since Latest Value for volume and freshness.

After you've configured the source, Bigeye loads and profiles your tables. It can take up to 24 hours for the profiling to complete and your autometrics and autothresholds to populate. See how to deploy autometrics in the Getting Started guide.