Administration > Data observability > Configuring data observability events
  

Configuring data observability events

You can apply filters to narrow the data set so that you receive anomaly notifications in Data Governance and Catalog only on relevant and applicable data.
    1Open the catalog source for which you want to configure data observability.
    2In the Configuration tab of the wizard, go to the Data Observability tab.
    The option to enable data observability appears. See the Data Observability tab under the Configuration option in a catalog source.
    3Enable data observability.
    The configuration parameters for data observability appear. The Data Observability toggle is enabled and highlighted on a catalog source configuration wizard.
    4Configure the parameters for data observability.
    The following table lists the parameters you can configure:
    Option
    Description
    Minimum Number of Data Points
    Specify the minimum number of profiling runs that are required for data observability to start detecting anomalies. For example, if you enter 4 here, anomalies are detected for 4 and subsequent profiling runs. The default value is 3.
    Enter a number between 3 and 10.
    Maximum Events to Generate
    Specify the maximum number of anomaly events to generate for each catalog source run. The default value is 1000.
    Specify freshness and volume filters
    Add filters to observe freshness and volume metrics on the tables that you specify.
    Specify data profiling filters
    Add filters to objects that you want to profile. You can run data observability on only the profiled objects.
    Metric Filters
    Select an option to indicate whether you want to further filter the profiled data elements.
    You can select one of the following options:
    • - No filters. Do not filter the profiled data elements. Detect anomalies on the data that you have configured for metadata extraction and profiling.
    • - Filter conditions: Select one or more conditions to filter the profiled data elements. Detect anomalies by creating a subset of data after metadata extraction and profiling.
    Inclusion or exclusion criteria
    Select the filter condition to apply on the profiled data.
    You can select one of the following options:
    • - Include Metric. Specify an inclusion criteria. Detect anomalies on the profiled data that meets the filter criteria.
    • - Exclude Metric. Specify an exclusion criteria. Exclude profiled data that meets the filter criteria.
    You can further narrow down the results by clicking Add to add further filter conditions.
    Metrics
    Select the metric for which data observability notifies users of anomalies.
    Sensitivity
    Select the sensitivity of the anomaly.
    You can select any of the following options:
    • - Normal. Notify you of anomalies about normal changes to data.
    • - Sensitive. Notify you of anomalies about sensitive changes to data.
    • - Severe. Notify you of anomalies about severe changes to data.
    Detection rules
    Select one or more rules to apply on the profiled data to detect anomalies.
    You can select any of the following options:
    • - Static Data. Detect the following anomalies:
      • - Percentage variation
      • - Count variation
    • - 100% or 0% Change Detection. Detect the following types of percentage-based anomalies:
      • - Drop from maximum
      • - Surge from minimum
    • - Standard Deviation. Detect the following anomalies:
      • - Drop in transition
      • - Surge in transition
      • - Deviation
    • - Breaking Trends. Detect the following types of count-based anomalies:
      • - Drop
      • - Surge
    Metric Configuration
    Select how you want to measure metadata volume.
    You can select one of the following options:
    • - Statistics. Data observability volume that is extracted from the objects of the source system. Here, the volume measured might be outdated.
    • - Calculated. Data observability volume that is measured when a data observability job is run on the catalog source. Here, the volume measured is more accurate.
To understand the various types of anomalies and their corresponding metrics in data observability, see the How-to Library article Understand Data Observability Anomalies in Data Governance and Catalog.