Kafka Connector > Mappings for Kafka > Kafka targets in mappings
  

Kafka targets in mappings

To write data to a Kafka target, configure a Kafka object as the Target transformation in a mapping.
Specify the name and description of Kafka target. Configure the target and advanced properties for the target object in mappings.
The following table describes the target properties that you can configure for a Kafka target:
Property
Description
Connection
Name of the Kafka connection that is associated with a Kafka cluster.
Target Type
Type of the Kafka target objects available.
You can write data to a single Kafka target object or parameterize the object. You cannot write data to multiple objects.
Object
Name of the Kafka target object based on the target type selected.
Parameter
Select a parameter for the target object, or click New Parameter to define a new parameter for the target object.
The Parameter property appears only if you select Parameter as the target type.
Create New at Runtime²
Creates a target.
Enter a name for the target object and path for the target object and select the source fields that you want to use. By default, all source fields are used.
The target name can contain alphanumeric characters. You cannot use special characters in the file name except the underscore character (_).
When you write data to a field of hierarchical data type, you cannot parameterize the target at runtime.
Format
The file format to write data to a Kafka topic.
You can select from the following file format types:
  • - None. Writes data to the Kafka topics in binary format.
  • - Avro. Writes data to the Kafka topics in Avro format that use the binary encoding type.
  • - JSON. Writes data to the Kafka topics in JSON format.
To configure the formatting options for the file, click Formatting Options. For more information about format options, see Formatting options for Kafka topics.
Operation
Select Insert as the target operation.
You cannot perform update, upsert, or delete operations on a Kafka target topic.
Preview Data
Preview the data of the Kafka target object.
When you preview data, Data Integration displays the first 10 rows.
By default, Data Integration displays the fields in native order. To display the fields in alphabetical order, enable the Display source fields in alphabetical order option.
¹ Applies only to mappings.
² Applies only to mappings in advanced mode.
The following table describes the advanced properties that you can configure for a Kafka target:
Property
Description
Metadata Fetch Timeout in milliseconds
The time after which the metadata is not fetched.
Default value is 10000.
Batch Flush Time in milliseconds
The interval after which the data is published to the target.
Default value is 1000.
Batch Flush Size in bytes
The batch size of the events after which the Secure Agent writes data to the target.
Default value is 16384.
Producer Configuration Properties
The configuration properties for the producer. Overrides the Additional Connection Properties or Additional Security Properties specified in the Kafka connection.
For more information about Kafka producer configuration properties, see Kafka documentation.
Forward Rejected Rows
Not applicable for Kafka Connector.