When you configure a mapping, you describe the flow of data from the source to the target.
A mapping defines reusable data flow logic that you can use in mapping tasks.
When you create a mapping, you define the Source, Target, and Lookup transformations to represent a Databricks object. Use the Mapping Designer in Data Integration to add the Source, Target, or Lookup transformations in the mapping canvas and configure the Databricks source, target, and lookup properties.
After you create a mapping, you can run the mapping or you can deploy the mapping in a mapping task. The mapping task allows you to process data based on the data flow logic defined in a mapping.
In advanced mode, the Mapping Designer updates the mapping canvas to include transformations and functions that enable advanced functionality.
You can use Monitor to monitor the jobs.
The following table lists the transformations that are supported by SQL warehouse and Databricks cluster:
Property
SQL warehouse1
Databricks cluster2
Source
Yes
Yes
Target
Yes
Yes
Filter
Yes
Yes
Lookup
Yes
No
Sorter
Yes
Yes
SQL
Yes
No
1The Secure Agent connects to the SQL warehouse at design time and runtime.
2The Secure Agent connects to the all-purpose cluster to import the metadata at design time and to the job cluster to run the mappings.