Metrics
A metric is a column of a data source or output of a rule that is part of a scorecard. When you create a scorecard, you can assign a weight to each metric. Create a metric group to categorize related metrics in a scorecard into a set.
Metric Weights
When you create a scorecard, you can assign a weight to each metric. The default value for a weight is 0.
When you run a scorecard, the Analyst tool calculates the weighted average for each metric group based on the metric score and weight you assign to each metric.
For example, you assign a weight of W1 to metric M1, and you assign a weight of W2 to metric M2. The Analyst tool uses the following formula to calculate the weighted average:
(M1 X W1 + M2 X W2) / (W1 + W2)
Value of Data Quality
A measure of data quality in the source data is critical information in the management of the data assets in the organization. The cost of invalid data in metrics represented in a scorecard helps organizations derive value in monitoring data quality of the source data. As a data analyst, you might want to associate a value, such as a currency unit or any custom unit, to metrics and metric groups. You can then run the scorecard to view the total cost of invalid data in the source data.
You can define the cost unit for a metric based on the business needs. You can also configure a variable or fixed cost for each metric when you create a scorecard or edit it.
Fixed Cost
Fixed cost is a constant value that you can assign to a metric in a scorecard. You can choose a predefined cost unit or create a custom cost unit that meets the business needs.
Variable Cost
Variable cost is a value that you assign to a metric based on the values in a numeric column of a data source. The Data Integration Service calculates the variable cost for the metric based on the column or virtual column that you assign to the cost.
Example
As a mortgage loan officer, you need to provide your customers with payment books so that the customers can submit the mortgage payments. You can use a scorecard to measure the accuracy of your customer addresses to ensure the delivery of the payment books. You might want to set the variable cost to the Monthly Payment Amount column for the Address Accuracy metric. Run the scorecard to compute the total cost that the mortgage organization loses if customers did not pay the monthly amount on time.
Adding Columns to a Scorecard
After you run a profile, you can add columns from a profile to a scorecard, configure the valid values for the columns, and add the cost of invalid data for each metric. If you add a column to a scorecard from a profile with a filter or sampling option other than All Rows, profile results might not reflect the scorecard results.
1. Click a profile to open it.
The profile appears in the Discovery workspace.
2. Click Actions > Run Profile to run the profile.
3. Click Actions > Add to > Scorecard.
The Add to Scorecard wizard appears.
Note: Use the following rules and guidelines before you add columns to a scorecard:
- - You cannot add a column to a scorecard if both the column name and scorecard name match.
- - You cannot add a column twice to a scorecard even if you change the column name.
4. Select Existing Scorecard to add the columns to a predefined scorecard.
The New Scorecard option is selected by default.
5. Click Next.
6. Select the scorecard that you want to add the columns to, and click Next.
7. Select the columns and rules that you want to add to the scorecard as metrics. Optionally, click the check box in the left column header to select all columns. Optionally, select Column Name to sort column names.
8. Click Next.
9. Select each metric in the Metrics pane and configure the valid values from the list of all values in the Score using: Values pane.
You can select multiple values in the Available Values pane and click the right arrow button to move them to the Valid Values pane. The total number of valid values for a metric appears at the top of the Available Values pane.
10. Select each metric and configure the cost of invalid data. To assign a constant value to the cost for the metric, select Fixed Cost.
Optionally, click Change Cost Unit to change the unit of cost.
11. To attach a numeric column as a variable cost to the metric, select Variable Cost and click Select Column to select a numeric column.
12. In the Metrics pane, select each metric.
13. In the Metric Thresholds pane, configure metric thresholds.
You can set thresholds for Good, Acceptable, and Unacceptable scores.
14. Click Next.
15. Set up the metric weight for each metric. You can double-click the default metric weight of 0 to change the value.
16. In the Metric Group Thresholds pane, set up metric group thresholds.
17. Click Save to save the scorecard, or click Save & Run to save and run the scorecard.
Running a Scorecard
Run a scorecard to generate scores for columns.
1. In the Assets panel, choose the scorecard that you want to run.
2. Click the scorecard to open it.
The scorecard appears in the Scorecards workspace.
3. Click Actions > Run Scorecard.
4. Select a score from the Metrics pane and select the columns from the Columns pane to drill down on.
5. In the Drilldown option, choose to drill down on live data or staged data.
For optimal performance, drill down on live data.
6. Click Run.
Viewing a Scorecard
Run a scorecard to see the scores for each metric. A scorecard displays the score as a percentage and bar. View data that is valid or not valid. You can also view scorecard information, such as the metric weight, metric group score, score trend, and name of the data object.
1. Run a scorecard to view the scores.
2. Select a metric that contains the score you want to view.
3. Click Actions > Drilldown to view the rows of valid data or rows of data that is not valid for the column.
The Analyst tool displays the rows of data that is not valid by default in the Drilldown section.
Editing a Scorecard
Edit valid values for metrics in a scorecard. You must run a scorecard before you can edit it.
1. In the Assets panel, choose a scorecard.
2. Click the scorecard to open it.
The scorecard appears in the Scorecards workspace.
3. To change the general properties of the scorecard, click Actions > Edit > General.
The Edit Scorecard dialog box appears.
4. Edit the name and description of the scorecard as required.
5. Click the Metrics tab.
6. Select each score in the Metrics pane and configure the valid values from the list of all values in the Score using: Values pane.
7. In the Metric Thresholds pane, make changes to the score thresholds as required.
8. Review the cost of invalid data for each metric and make changes as required.
9. Click the Metric Groups tab.
10. Create, edit, or remove metric groups.
You can also edit the metric weights and metric thresholds on the Metric Groups tab.
11. Click the Notifications tab.
12. Make changes to the scorecard notification settings as required.
You can set up global and custom settings for metrics and metric groups.
13. Click Save to save changes to the scorecard, or click Save & Run to save the changes and run the scorecard.
Defining Thresholds
You can set thresholds for each score in a scorecard. A threshold specifies the range in percentage of bad data that is acceptable for columns in a record. You can set thresholds for good, acceptable, or unacceptable ranges of data. You can define thresholds for each column when you add columns to a scorecard, or when you edit a scorecard.
Complete one of the following prerequisite tasks before you define thresholds for columns in a scorecard:
- •Open a profile and add columns from the profile to the scorecard in the Add to Scorecard dialog box.
- •Optionally, click a scorecard in the Library workspace and select Actions > Edit to edit the scorecard in the Edit Scorecard dialog box.
1. In the Add to Scorecard dialog box or the Edit Scorecard dialog box, select each metric in the Metrics pane.
2. In the Metric Thresholds pane, enter the thresholds that represent the upper bound of the unacceptable range and the lower bound of the good range.
3. Click Next or Save.