Kafka Connection
The Kafka connection is a Messaging connection. Use the Kafka connection to access an Apache Kafka broker as a source or a target. You can create and manage a Kafka connection in the Developer tool or through infacmd.
The Kafka broker maintains configuration information in Apache ZooKeeper. Apache ZooKeeper is a centralized service that maintains the configuration information of the Apache brokers.
When you configure a Kafka connection, you configure the following properties:
General Properties
The following table describes the general connection properties for the Kafka connection:
Property | Description |
|---|
Name | The name of the connection. The name is not case sensitive and must be unique within the domain. You can change this property after you create the connection. The name cannot exceed 128 characters, contain spaces, or contain the following special characters: ~ ` ! $ % ^ & * ( ) - + = { [ } ] | \ : ; " ' < , > . ? / |
ID | The string that the Data Integration Service uses to identify the connection. The ID is not case sensitive. It must be 255 characters or less and must be unique in the domain. You cannot change this property after you create the connection. Default value is the connection name. |
Description | The description of the connection. Enter a string that you can use to identify the connection. The description cannot exceed 4,000 characters. |
Location | The domain where you want to create the connection. Select the domain name. |
Type | The connection type. Select Messaging/Kafka. |
Kafka Broker Properties
The following table describes the Kafka broker properties for the Kafka connection:
Property | Description |
|---|
Kafka Broker List | The IP address and port combinations of the Kafka messaging system broker list. The IP address and port combination has the following format: <IP Address>:<port> You can enter multiple comma-separated IP address and port combinations. |
ZooKeeper Host Port List | The IP address and port combination of Apache ZooKeeper which maintains the configuration of the Kafka messaging broker. The IP address and port combination has the following format: <IP Address>:<port> You can enter multiple comma-separated IP address and port combinations. |
Retry Timeout | Number of seconds the Integration Service attempts to reconnect to the Kafka broker to read data. If the source or target is not available for the time you specify, the mapping execution stops to avoid any data loss. |
Kafka Broker Version | The version of the Kafka messaging broker. You can select one of the following values: - - 0.8.x.x-0.10.0.x
- - 0.10.1.x-1.0.0
|
Note: When you click Test Connection to verify that you entered the connection properties correctly, the Data Integration Service verifies the connection to ZooKeeper and does not verify the connection to the Kafka broker.
Creating a Kafka Connection using Informatica Command Line Program
You can use the infacmd command line program to create a Kafka connection. Access the command from the <Informatica installation directory>/clients/DeveloperClient/infacmd directory.
On UNIX, run the following command:
sh infacmd.sh createConnection
On Windows, run the following command:
infacmd.bat createConnection
Enter connection options in the following format:
... -o option_name='value' option_name='value' ...
For example, run the following command to create a Kafka connection on UNIX:
sh infacmd.sh createConnection -dn <domain name> -un <domain user> -pd <domain password> -cn <connection name> -cid <connection id> -ct Kafka -o zkHostPortList=<host1:port1>,<host2:port2>,<host3:port3> kfkBrkList=<host1:port1>,<host2:port2>,<host3:port3>
For more information about the CreateConnection command, see the Informatica Command Reference Guide.