A rule occurrence is a set of metrics you can create from a rule specification linked to a profile. A rule specification is the building block for a rule occurrence. You can configure multiple rule occurrences for a rule specification associated with a profile. You cannot use other Data Quality assets or mapplets to create rule occurrences. After you create rule occurrences in a profile, you can run the profile and view scorecards.
A scorecard is the graphical representation of valid values for a column in a profile. A scorecard is a collection of rule occurrences and represents data quality scores calculated when you profile a source dataset. You can use scorecards to measure data quality scores and monitor data quality progress. A measure of data quality in the source data is critical information in the management of the data asset in an enterprise. You can drill down on live data in a scorecard.
You can use scorecards to measure data quality progress for existing and new profiles. To view the scorecard, use the scorecard dashboard in Data Governance and Catalog.
Prerequisites to view scorecards
The following prerequisites must be fulfilled to view the scorecard dashboard in Data Governance and Catalog:
•You must have Intelligent Cloud Data Management and Data Governance and Catalog licenses.
•The Intelligent Cloud Data Management user must be assigned the Governance User role.
Important: Scorecard feature depends on the availability of Data Quality and Data Governance and Catalog on the pod where the organization is located.
Prerequisites to create rule occurrences
Before you create a rule occurrence, you must verify the configuration and output of the rule specification that you link to the profile.
Verify the following prerequisites:
•The rule specification that you select must be defined with a dimension.
A dimension is a one-word summary of the data quality issue that a rule specification represents. The dimension reflects the primary purpose of the business rule. Use the built-in dimensions on the rule specification to configure scorecard metrics. You can also use the custom dimensions that you create from the Dimension attribute in the Asset Customization > Attributes tab of the Customize page to configure the metrics. You can define a maximum of 12 values.
Note: The source of truth for dimensions is the Dimension attribute in Metadata Command Center.
•The output of the rule specification must be one of the following values for rows that pass the quality check:
- Valid
- True
- 1
- Yes
- Ok
For rows that do not pass the quality check, the output of the rule specification can be any value.
Creating rule occurrences
Create rule occurrences on the Metrics tab. The tab appears after you select Scorecard Metrics from the Menu option. After you create rule occurrences, you must save and run the profile. You can use only rule specifications to create a rule occurrence and other Data Quality assets or mapplets cannot be used.
You can create rule occurrences on a profile with rules that have single or multiple output fields. To create a rule occurrence with multiple output fields, select an output field that is marked as available. Data Profiling marks the output field name as available to indicate that the scorecard uses it. The output field name is marked available and ranked in precedence in the following order: isValid, isTrue, out_valid, out_status, PrimaryRuleSet.
Note: Data Profiling does not list rules with multiple output fields on the Metrics tab, if the field names are not isValid, isTrue, out_valid, out_status, or PrimaryRuleSet.
If you create rule occurrences on a profile with rules that have multiple input fields, the scorecard dashboard displays the scores corresponding to only one column selected randomly from the input fields when you run the profile.
Perform the following steps to create a rule occurrence:
1On the Metrics tab, click Add ().
2In the Create Rule Occurrence dialog box, choose a rule specification output to measure. You can either select a single output field or multiple output field.
3Click Next.
4In the Create Rule Occurrence dialog box, perform the following steps:
aSpecify a name and description for the rule occurrence in the General Details section.
bDefine valid threshold values for scorecard generation in the Rule Occurrence Thresholds section. You can modify a threshold after you create a rule occurrence without running the profile again.
5Click OK.
The rule appears on the Rule Occurrences tab.
6Continue to add more rule occurrences to the Rule Occurrences tab as necessary.
7Choose one of the following options to save and run the profile:
- Click Save to save the profile.
- Click Run to save and run the profile.
You can view the scorecard results only when you run the profile again after you create or update rule occurrences.
Note: When you delete a rule that is associated with a data profiling task, the corresponding rule occurrences are automatically deleted from the profile.
Viewing scorecards
Use scorecards to measure data quality scores and monitor data quality progress for existing and new profiles.
Click the View Scorecard button to view the scorecard dashboard in Data Governance and Catalog.
The following table lists the widgets that you can view with the scorecard dashboard:
Widget
Description
Average Latest Scores by Dimensions
Donut charts that display the average latest data quality scores based on dimensions. The scores are rounded to two decimal places.
Number of Rule Occurrences by Dimensions
Number of rule occurrences for each dimension based on Good, Acceptable, and Not Acceptable threshold values.
Rule Occurrences
Shows the following details of rule occurrences:
- Latest data quality score
- Dimension of the rule specification
- Date and time of latest profile run
- Total number of rows processed
- Total number of failed rows
- Input column or primary data element
- Preview valid and failed rows. To preview valid or failed rows, hover over the rule occurrence and click the ellipsis button. Then, select Preview of Valid Rows or Preview of Failed Rows options.
Every time you run a data profiling task with rule occurrences, the scores on the scorecard dashboard are updated. If you define a rule occurrence but do not execute the profile, then the rule occurrence appears on the scorecard dashboard without any score.
Example
You are a data analyst. You create and run profiles on a Customer table. You want to check the validity of the data available in the latest profile run.
You perform the following tasks:
1Create a rule specification with the appropriate rule logic in Data Quality and set the dimension to Validity. When you apply a Validity dimension to a rule, the output data conforms to defined business rules and falls within allowable parameters when those rules are applied.
2Create a profile and associate the rule specification.
3Create a rule occurrence on the rule specification with Good, Acceptable, and Not Acceptable threshold values to be considered for scoring.
4Save and run the profile.
5View the metrics in the Data Governance and Catalog scorecard dashboard. You can use the metrics to verify the data quality progress in the Customer table.
View stakeholder information
You can view users that have been designated as stakeholders for the rule occurrences on the Overview and Stakeholder tab in Data Governance and Catalog. A stakeholder is an authorized user who is responsible for the rule occurrences, can approve or reject change requests for the occurrence, provide inputs to the properties of the rule occurrence, and are interested in following the asset to monitor changes.
A user who creates a rule occurrence gets assigned as a stakeholder of that particular rule occurrence provided the user has the necessary privilege assigned to the user role. To assign stakeholders to the rule occurrences, the organization administrator must enable the Data Governance Administrator privilege for the user role.
For more information about the stakeholders, see the Asset Details and Working with Assets guides in the Data Governance and Catalog documentation.
View notifications for status change of scores
When there is a score status change to the rule occurrence, an alert or notification is generated in Data Governance and Catalog. You can view application notifications and receive email notifications for the changes to the rule occurrence status. To configure email notifications, you can click the settings link () on the Notifications page, and then enable the Email Summary and Email Event options for the Data Quality notification type.
You can receive notifications for the following changes to the rule occurrences:
•The status changes from good to not acceptable
•The status changes from acceptable to not acceptable
•The status changes from good to acceptable
The users or user groups who are assigned as stakeholders to the rule occurrence receive the notifications. A user with the Governance Administrator role gets assigned as a stakeholder for the rule occurrences that they create.
To add a user with a custom role as a stakeholder, the user must ensure that the following permissions and privileges are met:
•Enable create, read, update, and delete permissions on the data quality assets for Metadata Command Center service.
•Enable the Stakesholdership feature for the Data Governance and Catalog service.
For more information about the notifications for scores status, see the Working with Assets guide in the Data Governance and Catalog documentation.
Download rows of rule occurrences and metrics
You can download rows in rule occurrences and metrics from the scorecard dashboard in Data Governance and Catalog. You download a maximum of 100 rows to delimited and legend files. To download rows, click the download link from the Preview of Valid Rows window.
).
) on the