When you configure a mapping, you describe the flow of data from the source to the target.
A mapping defines reusable data flow logic that you can use in mapping tasks.
When you create a mapping, you define the Source, Target, and Lookup transformations to represent a Databricks object. Use the Mapping Designer in Data Integration to add the Source, Target, or Lookup transformations in the mapping canvas and configure the Databricks source, target, and lookup properties.
After you create a mapping, you can run the mapping or you can deploy the mapping in a mapping task. The mapping task allows you to process data based on the data flow logic defined in a mapping.
In advanced mode, the Mapping Designer updates the mapping canvas to include transformations and functions that enable advanced functionality.
You can use Monitor to monitor the jobs.
The following table lists the functionalities that are supported by SQL warehouse, all-purpose cluster, and job cluster:
Property
SQL warehouse1
All-purpose cluster2
Job cluster3
Source transformation
Yes
Yes
Yes
Target transformation
Yes
Yes
Yes
Filter transformation
Yes
No
Yes
Lookup transformation
Yes
No
No
Sorter transformation
Yes
No
Yes
SQL transformation
Yes
No
No
Dynamic schema handling
Yes
No
No
Identity columns
Yes
No
No
1The Secure Agent connects to the SQL warehouse at design time and runtime.
2The Secure Agent connects to the all-purpose cluster to import the metadata at design time.
3The Secure Agent connects to the job cluster to run the mappings.