You can configure key range partitioning when you use a mapping to read data from Snowflake sources. The partition type controls how the agent distributes data among partitions at partition points. Partitioning is not applicable for mappings in advanced mode.
Partitioning optimizes the mapping performance at run time. When you run a mapping configured with key range partitioning, the agent distributes rows of source data based on the field that you define as partition keys. The agent compares the field value to the range values for each partition and sends rows to the appropriate partitions.
Click Add New Key Range to define the number of partitions and the key ranges based on which the agent must partition data.
The following image shows an example of the configured partitioned ranges on the Partitions tab:
Use a blank value for the start range to indicate the minimum value. Use a blank value for the end range to indicate the maximum value.
Use key range partitioning for columns that have an even distribution of data values. Otherwise, the partitions might have unequal size. For example, a column might have 10 rows between key values 1 and 1000 and the column might have 999 rows between key values 1001 and 2000. If the mapping includes multiple sources, use the same number of key ranges for each source.
When you define key range partitioning for a column, the agent reads the rows that are within the specified partition range. For example, if you configure two partitions for a column with the ranges as 10 through 20 and 30 through 40, the agent does not read the rows 20 through 30 because these rows are not within the specified partition range.
You can configure a partition key for fields of the following data types:
•Integer
•String
•Any type of number data type. However, you cannot use decimals in key range values.
•Datetime. Use the following format to specify the date and time: YYYY-MM-DD HH24:MI:SS. For example, 1971-01-01 12:30:3
If you specify the date and time in any other format, the task fails.
When you run a mapping enabled for key range partitioning, you can't filter source data even though you configure a filter in the Source transformation.
