Mappings in advanced mode example

You work for one of the largest community college that maintains millions of records in their ongoing student database. The college has more than 10,000 faculty members teaching at 45 campuses and 700 locations across the globe. The college has a very large IT infrastructure and about 15 TB of information gets downloaded on daily basis from the Internet.

To avoid performance, scalability, and high cost challenges, the college plans to port its entire data from its operational data stores to Microsoft Azure Data Lake Storage Gen2 within a short span of time. Configure a mapping in advanced mode to achieve faster performance when you read data from the operational data stores and write data to the Microsoft Azure Data Lake Storage Gen2 target.

1In Data Integration, click New > Mappings > Mapping.

2In the Mapping Designer, click Switch to Advanced.

The following image shows the Switch to Advanced button in the Mapping Designer:

In the Mapping Designer, the header includes the Switch to Advanced button.

3In the Switch to Advanced dialog box, click Switch to Advanced.

The Mapping Designer updates the mapping canvas to display the transformations and functions that are available in advanced mode.

4 Enter a name, location, and description for the mapping.

5On the Source transformation, specify a name and description in the general properties.

6On the Source tab, perform the following steps to provide the source details to read data from the source:

aIn the Connection field, select the required source connection.
bIn the Source Type field, select the type of the source.
cIn the Object field, select the required object.
dIn the Advanced Properties section, provide the appropriate values.

Note: When you read a flat file, ensure that the file has a header row. Else, the Secure Agent fails to read the first row.

7On the Fields tab, map the source fields to the target fields.

8On the Target transformation, specify a name and description in the general properties.

9On the Target tab, perform the following steps to provide the target details to write data to the Microsoft Azure Data Lake Storage Gen2 target:

aIn the Connection field, select the Microsoft Azure Data Lake Storage Gen2 target connection.
bIn the Target Type field, select the type of the target.
cIn the Object field, select the required object.
dIn the Operation field, select the required operation.
eIn the Advanced Properties section, provide appropriate values for the advanced target properties.

10Map the source and target.

11Click Save > Run to validate the mapping.

In Monitor, you can monitor the status of the logs after you run the task.