Define a data quality rule to be run on a data element that is the part of the metadata extracted by Metadata Command Center. To define the rule, create a data quality rule occurrence, specify the properties of the rule occurrence, and associate the rule occurrence with the data element on which you want to run the rule.
Before you run the data quality rules, enable the Data Quality capability for the catalog source in Metadata Command Center.
The rule you define is created in the data quality application that is integrated with Data Governance and Catalog. The data quality application runs the rule on the data element. After the rule is run, Data Governance and Catalog retrieves and displays the data quality scores of the data element in the rule occurrence.
Note:
The following guidelines apply for associating rule occurrences to data elements:
•The rule occurrences can be manually created, automatically generated, or can be associated with a data quality score card. For automatically generated rule occurrences, you can view the individual data quality scores of the data elements on which the automated data quality rules have been run. You can also manually run a data quality rule to generate manually created rule occurrences.
•When a profile that has a rule associated with it is run in Data Profiling, you can view the data quality score card of the rule occurrence in Data Governance and Catalog. You can't view the total runs of the rule occurrence, and can't preview failed rows immediately after the profile run. The Total Runs data appears within one hour after the profile run. Rule occurrences related to the score card are generated in Data Governance and Catalog. You can associate a data element to a rule occurrence that is generated from a score card.
•You can manually associate rule occurrences with data elements. When you purge a catalog source, the system also deletes the associated rule occurrences.
1Open the data element where you want to run the rule, and click the Action menu in the right header.
2From the Action menu, select Create Data Quality Rule Occurrence.
3Alternatively, click New > Business Rules and select Data Quality Rule Occurrence.
4Enter the following properties of the rule occurrence:
Field
Description
Rule Template
Data quality rule template that defines the parameters of the rule that you want to run on the data element.
Sync with Rule Template
Syncs the rule occurrence with the specified rule template.
When synced, the rule occurrence inherits parameters from the rule template. The rule occurrence score uses the technical rule reference, target, and threshold values from the template. Rule occurrence scores update according to the rule automation schedule in the template.
When not synced, the rule occurrence is an independent asset. Template parameters don't affect it.
Name
Name of the rule occurrence.
Description
Description of the rule occurrence.
Reference ID
Unique identifier for the rule occurrence.
Dimension
Data quality dimension for which the data quality rule is run.
Measuring Method
Method by which the data quality rule for the rule occurrence is evaluated. This field can have one of the following values:
- Business Extract. The rule is measured on data that is exported for a particular business case.
- System Function. The rule is measured by a data quality system.
- Technical Script. The rule is measured by a script that is manually run by an analyst.
- Informatica Cloud Data Quality. The rule is measured by Informatica Cloud Data Quality.
Note:
If you select Informatica Cloud Data Quality as the measuring method, the Technical Rule Reference field is mandatory. This field is not mandatory for other measuring methods.
Primary Data Element
Data element where the data quality rule runs.
This data element is the input port for the rule in the integrated data quality system. The system always generates the data quality score for the primary data element.
To add a primary data element:
aSelect a system asset or catalog source.
bSelect a technical data set or business data set.
cSelect data elements from the data set.
If you do not enable the Data Quality option in Metadata Command Center for a catalog source, the options in this field are disabled.
Editing the primary data element
Whether you can edit the primary data element depends on how you created the rule occurrence:
- Manually created rule occurrences: You can edit the primary data element until you run the rule occurrence for the first time. You can't edit it after the rule runs.
- Rule occurrences from data profiling scorecards: You can add the primary data element when you associate the rule occurrence. You can't edit it after you add it. If you don't add a primary data element, no data quality score appears.
- Auto-generated rule occurrences: You can't edit the primary data element.
You can't edit the primary data element for auto-generated rule occurrences.
Note:
Ensure that the first four characters of the primary data element name don't contain the following special characters: comma (,), hyphen (-), semi-colon (;), backslash (\), single quote ('), double quotes ("), angle brackets (< and >), equal sign (=), parentheses ({ and }), braces (( and )), square brackets ([ and ]), and period (.)
Secondary Data Element
Secondary input port for the data quality rule that is run on the Primary Data Element. You can edit the secondary data element after the rule occurrence is created, but can't edit the element after the rule occurrence is executed.
Technical Rule Reference
Create a reference to a rule specification in the data quality application. You can create a new rule specification, or select from an existing rule specification.
- To create a new rule specification in the data quality application, click Create a rule. Enter a description for the rule using natural language construction, and click View Recommendations. CLAIRE® reads the description that you enter and intelligently recommends a rule that it can create in the data quality application.
If you want to use the Create a rule option, your organization administrator must enable the Enable CLAIRE Generative AI Services setting in Administrator.
- To select from an existing rule specification in the data quality application, click Pick an existing rule.
After you select a rule option, click OK to go back to the rule occurrence creation page.
Note:
If you select Data Quality as the measuring method, the Technical Rule Reference field is mandatory. This field is not mandatory for other measuring methods.
When you manually create a rule occurrence, you can specify input parameters from multiple data elements. This evaluates asset quality based on multiple fields.
For example, to validate a candidate's hire date, you can use two input parameters: hire date and date of birth. The system uses both parameters to calculate data quality scores.
Input parameter mapping is mandatory for a data quality rule occurrence if you are using a multi input parameter rule from the integrated data quality system. Data quality scores are always reported on the primary data element.
Map each rule input parameter to a unique data element. Make sure that at least one rule input parameter is mapped to a primary data element.
Criticality
Criticality of the rule occurrence. The value can be High, Medium, or Low.
Target
Minimum acceptable data quality value for the asset to be considered "Good."
The target value is higher than the threshold value. For example, you can set the threshold value to 50 and the target value to 85.
Threshold
Minimum acceptable data quality value for the asset to be considered "Acceptable."
The target value is higher than the threshold value. For example, you can set the threshold value to 50 and the target value to 85.
Frequency
Frequency of running the data quality rule that is defined by the rule template.
- Select Daily to run the rule once in a day.
- Select Weekly to run the rule once in a week.
- Select Monthly to run the rule once in a month.
The rule is run when you click Create.
The following image shows the dialog box to select the primary data element:
The following example image shows a rule description and its CLAIRE® recommendation:
The following image shows the addition of rule reference for a single input parameter from an existing rule specification:
The following image shows the addition of a rule with multiple input parameters and the Show Preview icon to view the rule:
The following image shows how to map multiple input parameters to data elements:
The following image shows the input parameters mapped to the primary and secondary data elements:
5When all the details of the rule occurrence are ready, click Create.
When you click Create, the data quality application runs the rule on the data element. If you selected a schedule in the Frequency field, the data quality application runs the rule on the data element according to the rule schedule you specified in the Frequency field. When the data quality score is ready, Data Governance and Catalog displays the score in the rule occurrence.
Technical hierarchy of a rule occurrence
For an automatically generated rule occurrence, the catalog source is the highest unit and the data elements are the smallest units in the technical hierarchy.
Note:
If you bulk import the assets, the rule occurrences do not have the mentioned technical hierarchy. As a result, the rule occurrences are not removed when you purge a catalog source.