Creating Application Integration processes for data enrichment
Before you configure data enrichment, you need to create a process that connects to an external data provider. The process accepts a set of fields as input from MDM SaaS. The process then sends a request to the data provider. When the data provider returns data for enrichment, the process sends a response back to MDM SaaS. The response includes a set of output fields that contain data from the data provider.
When you create a process, you need to define the input fields that MDM SaaS sends to the process. You also need to define the output fields that the process sends to MDM SaaS.
To define the input and output fields, you need to create process objects that you can use within your process. Process objects group and structure data for use in processes. For example, instead of creating separate items for each type of demographic data, you can have a process object that contains the name, address, and phone number fields.
When you create process objects for input and output fields, ensure that the input and output formats of the process are compatible with the required MDM SaaS request and response formats.
Request format
Currently, you can process one record in each request.
The following table lists the parameters that an Application Integration process requires from an MDM SaaS enrichment request:
Parameter
Description
docId
A system-generated unique identifier of the record for which data enrichment is triggered.
payload
List of fields that you need to include in the request to an external data provider.
payload > sys > groupId
A system-generated unique identifier of a field group entry that's mapped at the root level.
payload > (field group) > sys > groupId
A system-generated unique identifier of a nested field group entry.
To create process objects in Application Integration, you might use the following sample request from MDM SaaS to a process:
To define a set of fields that you want to display as input and output fields in Business 360 Console, create a process and define process objects that contain the fields.
As a best practice, define input and output fields based on the data model of the business entity.
1In Application Integration, create a folder within a project.
2Create a process object named System with the text field named groupId.
The following image shows a sample process object named System with the field named groupId:
3To define the payload for the request that MDM SaaS sends, create a process object named InputPayload.
- Add the data provider fields that you want to display as input fields in Business 360 Console.
- Add a field named sys, and reference the System process object that you created in step 2.
The following image shows a sample process object named InputPayload with two fields named Addresses and sys:
4To define payload for the response to MDM SaaS, create a process object named OutputPayload.
- Add the data provider fields that you want to display as output fields in Business 360 Console.
- To display the data from the data provider as recommendations, add a text field named confidenceScoreInPercent.
The following image shows a sample process object named OutputPayload with two fields named confidenceScoreInPercent and Addresses:
5To define the request structure, create a process object named InputPerRecord.
- Add a field named payload, set its type to ObjectList, and select the InputPayload process object.
- Add a text field named docId.
The following image shows a process object named InputPerRecord with two fields named payload and docId:
6To define the response structure, create a process object named OutputPerRecord.
- Add a text field named docId.
- Add a field named payload, set its type to ObjectList, and select the OutputPayload process object.
The following image shows a process object named OutputPerRecord with two fields named docId and payload:
7Create a process.
8In the Start Properties panel of the process, perform the following steps:
aClick the Start tab, and select Allow anonymous access.
bClick the Input Fields tab.
cSelect Whole Payload, add an input field named request, and set its type to List of InputPerRecord by selecting the InputPerRecord process object.
The following image shows the input field named request and its type as List of InputPerRecord:
dClick the Output Fields tab.
eSelect Whole Payload, add an output field named response, and set its type to List of OutputPerRecord by selecting the OutputPerRecord process object.
The following image shows the output field named response and its type as List of OutputPerRecord:
9Based on your enrichment requirements, add necessary process steps to the process.
- Add a service connection.
- Add steps to receive input fields from MDM SaaS and send output fields to MDM SaaS.
Guidelines for creating Application Integration processes
Consider the following guidelines when you create an Application Integration process:
•When you define input and output fields for a process, model them after the data model of the business entity instead of the data provider. This approach helps minimize issues related to field mappings in Business 360 Console. You can use assignment steps in the process to specify the desired request and response formats of the data provider.
•Ensure that you include the docId parameter in both request from and response of the process. The docId parameter is required to track the record for which enrichment is triggered.
•Ensure that you include the sys > groupId parameter in the input format for a process. This parameter is required to track the field group entries in the MDM SaaS request.
•Include the sys > groupId parameter in the response that you send to MDM SaaS. Based on the groupId value, the field group entries are enriched. If the groupId value isn't available, a new field group entry is created.
•If you want to enrich the existing entries of a field group, ensure that you include the field group as one of the input fields. For example, to enrich the existing Phone field group entries, include the Phone field group as an input field. When you configure a rule association in Business 360 Console, you can map the Phone field group of the business entity to the input field.
•To display the data from the data provider as CLAIRE recommendations in the business applications, ensure that you include confidence scores for each recommendation when you send a response to MDM SaaS.
•Ensure that the process has anonymous or public access.
Creating an Application Integration process scenario
You work for Acme, and your goal is to enrich the Person records with correct addresses. You use AddressMaster as the data provider for this enrichment.
Consider the following sample request that AddressMaster requires you to send:
{ "addressline1":"GP", "postalcode":"", }
Also, consider the following sample response that you receive from AddressMaster:
To create a process for enriching records with data from AddressMaster, perform the following steps:
1Create process objects to define input and output formats.
2Create a process.
Step 1: Create process objects for input and output fields
You want to define input and output fields for a process to enrich records with the data from AddressMaster. Define process objects in Application Integration for this scenario.
1In Application Integration, create a folder within your project.
2Create a process object named System with a text field named groupId.
The following image shows a process object named System with a field named groupId:
3To define a set of fields that you want to accept as input fields from MDM SaaS, create a process object named InputAddress with the following fields:
Field
Label
Type
addressline1
addressline1
Text
postalcode
postalcode
Text
sys
sys
Reference
Note: Reference the System process object that you created in step 2.
Note: Even though the request to AddressMaster requires a flat structure, you need a field group in this scenario because addressline1 and postalcode are child attributes of the Address field group in the Person business entity.
The following image shows a process object named InputAddress with the fields named addressline1, postalcode, and sys:
4To define a set of fields that you want to send back to MDM SaaS, create a process object named OutputAddress with the following fields:
Field
Label
Type
addressline1
addressline1
Text
addressline2
addressline2
Text
city
city
Text
state
state
Text
postalcode
postalcode
Text
country
country
country
sys
sys
Reference
Note: Reference the System process object that you created in step 2. If you include this parameter, the original field group entry is updated. Otherwise, a new field group entry is added to the record when a user accepts a CLAIRE recommendation.
The following image shows a process object named OutputAddress with the fields named addressline1, addressline2, city, state, postalcode, country, and sys:
5To define the payload for the request from MDM SaaS, create a process object named InputPayload with the following fields:
Field
Label
Type
Addresses
Addresses
ObjectList
Note: Select the InputAddress process object that you created in step 3.
sys
sys
Reference
Note: Reference the System process object that you created in step 2.
The following image shows a process object named InputPayload with the fields named Addresses and sys:
6To define payload for the response to MDM SaaS, create a process object named OutputPayload with the following fields:
Field
Label
Type
Addresses
Addresses
ObjectList
Note: Select the OutputAddress process object that you created in step 4.
confidenceScoreInPercent
confidenceScoreInPercent
Text
The following image shows a process object named OutputPayload with the fields named confidenceScoreInPercent and Addresses:
7To define the request structure, create a process object named InputPerRecord with the following fields:
Field
Label
Type
payload
payload
ObjectList
Note: Select the InputPayload process object that you created in step 5.
docId
docId
Text
Note: docId is a required parameter in the response.
The following image shows a process object named InputPerRecord with the fields named payload and docId:
8To define the response structure, create a process object named OutputPerRecord with the following fields:
Field
Label
Type
docId
docId
Text
payload
payload
ObjectList
Note: Select the OutputPayload process object that you created in step 6.
The following image shows a process object named OutputPerRecord with the fields named docId and payload:
Step 2: Create a process
To enrich records with data from the AddressMaster data provider, you need a process that receives a set of input fields from MDM SaaS and sends a response back to MDM SaaS.
1In Application Integration, create a process.
2In the Start Properties panel of the process, perform the following steps:
aClick the Start tab, and select Allow anonymous access.
bClick the Input Fields tab.
cSelect Whole Payload, add an input field named request, and set its type to List of InputPerRecord by selecting the InputPerRecord process object.
The following image shows the input field named request and its type as List of InputPerRecord:
dClick the Output Fields tab.
eSelect Whole Payload, and add an output field named response and set its type to List of OutputPerRecord by selecting the OutputPerRecord process object.
The following image shows the output field named response and its type as List of OutputPerRecord:
3Create a temp field named temp_recordId, and set its type to Text.
4Add an assignment step to the process.
5In the Properties panel of the assignment step, click the Assignment tab, and set the temp_recordId field to the $input.request[1]/docId formula.
Note: Additionally, you can create a process object that represents the request to the data provider. You can then assign the process input fields to the AddressMaster request fields. This sample process doesn't connect to a real data provider.
The following image shows that the $input.request[1]/docId formula is assigned to the temp_recordId field:
6Add an assignment step to the process.
7In the Properties panel of the assignment step, click the Assignment tab and perform the following assignment tasks:
aAdd a field named response.
bIn the Assigned Using field, select Formula.
cAdd the response that you want to send to MDM SaaS, such as the following sample response:
$items represents a sample response from AddressMaster. In the return block, you can loop through the fields in the AddressMaster response and assign the required fields to the fields in the OutputPayload process object.
The following image shows that the response variable is assigned to a formula: