Alation
Bigeye integrates with Alation via the Bigeye Data Health Agent.
Version requirement
Required Alation API Version: greater than 2022.3
How it works
The Health tab will contain a list of Bigeye Metrics (rules) each with an associated column name (object name), status, the current observed value for the metric, description of the metric, and last updated timestamp. Users can click on the Rule link to access the metric in Bigeye, where they can get additional detail about an alert and review issues that have been raised for an alert. They can also click on the object name to access the Alation Column page and see the status for each.
Source Configuration
In order to identify the appropriate tables in Alation, users will need to add the data source ID from Alation to the corresponding source in Bigeye. To do so, a user with the Admin or Owner role must select Enable Alation Integration under Settings > Advanced Settings > Integrations in Bigeye.
Customer Credential
Each customer needs to provide the authentication detail for both Bigeye and their external catalog. Follow the below credential format for Alation.
Multiple workspaces
An additional parameter called
bigeye_workspace_id
needs to be added to the credential below when multiple workspaces are in place. See below on where to find it. In addition, the customer will need a separate credential per workspace. Although not required, if each credential has its own unique Alation refresh token, then the integration can update the token automatically when it is close to expiration.
API Key support
When using an API key for Bigeye authentication, the parameter
bigeye_workspace_id
MUST be configured as part of the credential.
{
"alation_user_id": 1234,
"alation_refresh_token": "1234sometoken1234",
"alation_base_url": "https://myorg.alation.com",
"bigeye_username": "[email protected]",
"bigeye_password": "myserviceaccountpasswordisfake",
"bigeye_base_url": "https://app.bigeye.com",
"bigeye_api_key": "bigeye_apk_somekey",
}
NOTE: When using an API key for Bigeye, you do not need the parameters bigeye_username or bigeye_password.
Sourcing Credential Details
- Basic auth credential for Bigeye.
- Bigeye URL
- https://app.bigeye.com by default OR the customer’s vanity URL.
- Bigeye Service Account User ID
- Bigeye user IDs are email addresses
- Bigeye Service Account Password
- ********For Bigeye SSO:********
- Go to
https://<customer base URL>/request-password-reset?email=<email>
- Set password
- Go to
- ********For Bigeye SSO:********
- Bigeye Workspace ID
- Found in Bigeye URL. e.g. https://app.bigeye.com/w/<bigeye_workspace_ID>
- Bigeye URL
- API credential for external catalog.
- Alation
- Alation Base URL
- https://CUSTOMER.alationcatalog.com for cloud SaaS OR the customer’s vanity URL.
- Alation User ID (Minimum role of Source Admin)
- Find this in the URL for the
User Profile
https://<<customer>>.alationcatalog.com/user/1234/
1234 is the user ID
- Find this in the URL for the
- Alation Refresh Token
- Create a refresh token under the
Account Settings :: Authorization
menu. Ensure you are creating a refresh token and not an API token.
- Create a refresh token under the
Optional Features
Bigeye can surface additional information in Alation such as Virtual Tables, and data quality summaries at the data source and schema levels of the catalog. More info on the features below.
Bigeye Virtual Schema
Virtual tables with monitoring can appear in Alation by utilizing the catalog’s Virtual Data Source and Technical Metadata features. The integration searches for a virtual data source called Bigeye Virtual Schema. If it exists, tables and metrics will appear in the data source on the next run.
Table names in Alation
How tables appear in Alation will depend on if the
bigeye_workspace_id
parameter is set in the credential above.Ex:
When it is not set, tables will follow the naming convention
Bigeye.<bigeye_source_name>.<virtual_table_name>
When it is set, tables follow the form below
<bigeye_workspace_name>.<bigeye_source_name>.<virtual_table_name>
Naming conventions
Virtual table names cannot contain special characters in Alation; e.g.
.
,/
,:
, etc. It causes failures with the data health API. Keep that in mind when deciding on names for workspaces, sources, and virtual tables within Bigeye.
Create virtual data source
To create a virtual data source in Alation choose Sources from the Apps dropdown. From there, click +Add…
and select Virtual Data Source
The title of the data source MUST be Bigeye Virtual Schema
; i.e. Bigeye virtual schema or anything else will not work. On the next page set Database Type as Custom DB
and select the check box for Use Catalog in Table Structure
After the next run of the integration, users should see virtual tables that are monitored in Bigeye in the newly created virtual data source. In addition, the SQL used to create them will also appear in Alation for users to better understand the virtual table.
Bigeye Data Quality Summary
The integration can provide summaries in Alation based on Bigeye metrics. These appear at the data source and schema levels of the catalog. The summaries utilize a rich text, custom field in Alation called Bigeye Data Quality Summary.
The data source summary tracks the number of metrics deployed and scores them along with a color coded status.
The schema summary groups metrics by dimension and scores them along with a color coded status.
Create custom field
To create the custom field in Alation navigate to settings in the top right corner. Under the Catalog Admin
section choose Customize Catalog
. In the Custom Fields
tab, scroll down to find Rich Texts. Click the + sign to add a new field. It MUST be named Bigeye Data Quality Summary
; i.e. Bigeye data quality summary or anything else will not work.
After that has been saved, click on the Custom Templates
tab. Under Data Object Templates
, the custom field needs to be added to the Data Source
and Schema
objects. Choose Insert… below Description for each and select Custom Field
. Filter on Bigeye and find the Bigeye Data Quality Summary field.
Updated 14 days ago