REST V2 Connector > Mappings and mapping tasks with REST V2 Connector > REST source transformation in mappings

REST source transformation in mappings

When you configure a Source transformation, select the REST V2 connection and choose an operation to represent a web service source.

You can select an operation for the source through the connection. Configure the request message using the request message template. You can parameterize the input values in the request XML. Configure the paging attributes.

You can view the response structure in the field mapping. When you map the elements from the response structure to the output fields, the Secure Agent creates the output groups, along with the primary and foreign keys for the field names. When you deploy the mapping in a mapping task and run the task, the Secure Agent reads the data from web service.

Advanced source properties

In a mapping, you can configure a source to represent a web service application source. For the REST source connections used in a mapping, you can configure advanced properties in the Source tab in the Mapping Designer.

The following table describes the advanced properties that you can configure in a source:

Property	Description
Paging Type	Specify one of the following values: Page. Enables paging and considers the values of Page Parameter, Start Page, End Page, and End of Response Expression properties. None. Ignores the values of Page Parameter, Start Page, End Page, and End of Response Expression properties.
Page Parameter	The name of the parameter that you want to use for the paging operation. You can use a query parameter or a path parameter. The parameter must be of the integer type and from the request message.
Start Page	The page number that indicates the first page in the range, on which you want to perform the paging operation.
End Page	The page number that indicates the last page in the range, on which you want to perform the paging operation. The default is 10000. Paging stops when the End Page is reached or the End of Response Expression is met.
Page Increment Factor	An integer to increment the Page Parameter. Page Increment Factor must be same as the number of records being fetched per request. If the Page Increment Parameter option is not same as the number of records being fetched per request, you might have missing or duplicate records between two calls.
End of Response Expression	Specify an expression or a string to control paging. You can observe one of the following behaviors: - If you specify a string or an expression, the paging stops when the value matches with the page response. It does not parse the page that has matching end of response. - If you do not specify a string or an expression, the paging stops on reaching a page that has an empty or a Null response. If an empty or a Null response is never reached, the paging stops at the default end page. - If you specify both, End Page and End of Response Expression, the paging stops on whichever condition is met first. - If you do not specify End Page and End of Response Expression, the paging stops on reaching an empty or a Null response. If an empty or a Null response is never reached, the paging stops at the default end page.
Override URL	Overrides the URL specified in the swagger specification. The override URL cannot have query parameters. When a path parameter is included in the Override URL, enclose the path parameter with curly brackets {}. For example: URL specified in the swagger specification: http://invr28pers102:13080/sample/day/20170505?a:b If you define 20170505 as a path variable with the path variable name as path1, the Override URL will be as follows: http://invr28pers102:13080/sample/day/{path1}
Tracing Level	Amount of detail that appears in the log for the source. Use the following tracing levels: - Terse - Normal - Verbose Initialization - Verbose Default is normal.
Cache Size for Web Service Response (KB)	Memory available for the web service response. If the web service response contains many rows or columns, you might want to increase the cache size. Default is 1024 KB. Maximum is 99999 KB.

Sample End of Response Expression

Use End of Response Expression for paging. The following snippet shows a sample response for a page :

{

"code": "SUCCESS",

"validationResult": [],

"systemErrors": [],

"patientResponseData": [],

"count": 0,

"message": "Unable to retrieve the implant details"

}

For End of Response Expression, you can use the string, Unable to retrieve the implant details. If you want to match multiple conditions in the response page, you can use the following expression:

(.*)"patientResponseData": [](.*)Unable to retrieve the implant details

The above expression ensures that the paging will stop when both "patientResponseData": [] and Unable to retrieve the implant details are matched in the page response.

REST source mapping example

You are a human resources administrator and you want to extract contact information, such as first name, last name, email, and phone number of employees from Apache CouchDB to a flat file.

1Create a REST connection. Verify that you specify the absolute path or the hosted URL of the swagger specification file, and the authentication method in the connection properties.

The following image shows the configured REST connection:

2Create a flat file connection to write data to the flat file.

3Create a REST mapping.

The following image shows the REST mapping:

4Add a Source transformation. Specify a name and description in the general properties.

5On the Source tab, perform the following steps:

aIn the Connection field, select the configured REST connection to connect to the Couch database.
bIn the Operation field, select CouchDB as the operation.

The following image shows the CouchDB operations:

cIn the Request Options section, configure the request message in the following XML format, specify the attributes in the message, and validate the message:

<proc:CouchDB_INPUT xmlns:proc="http://xml.schemas/infa/procedure/">

<CouchDB>

<emp>
$$empID
</emp>
</CouchDB>
</proc:CouchDB_INPUT>

The request message fetches the details of an employee based on the specified employee ID.

dIn the Advanced Properties section, set the tracing level to Normal, and use the default cache size of 1024 KB.

6On the Field Mapping tab, select the elements, such as _id, FirstName, and LastName in the response structure that you want to map to the output fields.

The following image shows the response structure on the left pane in a hierarchical format and the output groups on the right pane in a relational format:

The Secure Agent creates two output groups, workPhoneNumber and couchDB_out, which results in two relational output files. Primary and foreign keys are auto-generated.

7Add three Target transformations and specify the target objects for each of the transformations. Perform the following steps:

aSelect a flat file connection for each of the target transformations to write data to the flat files.
bCreate target objects workPhoneNumber.csv, couchDB_out.csv, and NewTarget.csv files to write data to the target.
cSelect the output group from CouchDB that you want to link to the target objects.

8Add an Expression transformation, and include a SetVariable function and the parameterized value $$empID in the expression so that the task updates the value of empID by one at the end of each run.

The following image shows the configured expression that utilizes the in-out parameters:

9Click New > Tasks > Mapping Task, and select the mapping for the task.

10When you save and run the mapping task, the Secure Agent retrieves employee records from CouchDB and writes the data to the corresponding flat files.