User Guide > Connections > AmazonKinesis Connection
  

AmazonKinesis Connection

The AmazonKinesis connection is a Messaging connection. Use the AmazonKinesis connection to access Amazon Kinesis Data Streams as source or Amazon Kinesis Data Firehose as target. You can create and manage an AmazonKinesis connection in the Developer tool or through infacmd.
When you configure an AmazonKinesis connection, you configure the following properties:

Prerequisites

Before you create an AmazonKinesis connection, perform the following tasks:
  1. 1. Generate an Access Key ID and Secret Access Key for the user in AWS. You can provide these key values when you create an AmazonKinesis connection.
    2. If you do not provide these key values, the values are picked up from the AWS configuration file location on the Developer client machine, during metadata import. Similarly, at runtime, the values are picked up from the default AWS configuration file location on the Hadoop cluster.
  2. 2. If you use Amazon Kinesis Streams as a source, perform the following task:
    1. a. Verify that the credentials that belong IAM user that you specify in the Access Key ID has the consumer permissions that are part of the IAM policy.
      2. For the list of permissions, see the AWS documentation at https://docs.aws.amazon.com/streams/latest/dev/learning-kinesis-module-one-iam.html
    2. b. Verify that you have the following permissions to fetch metadata: kinesis:DescribeStream, kinesis:GetShardIterator, and kinesis:GetRecords
  3. 3. If you use the Amazon Kinesis Firehose service as a target, perform the following tasks:
    1. a. Verify that you have an AWS account with the required IAM permissions to use the AWS services Kinesis Firehose, S3, Redshift, and Elastic Search.
    2. b. If you are writing to Amazon Redshift, verify that you have the Redshift INSERT privilege to copy data from the Amazon S3 bucket to the Redshift cluster.
    3. c. Define a Firehose Delivery Stream with either S3 or Redshift or Elasticsearch as its destination. Configure source as Direct PUT or other sources.
    4. d. Verify that the IAM user credentials have required permissions, based on the target you are writing to.
      5. For a list of permissions, see the AWS documentation at https://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html#access-to-firehose

General Properties

The following table describes the general connection properties for the AmazonKinesis connection:
Property
Description
Name
The name of the connection. The name is not case sensitive and must be unique within the domain. You can change this property after you create the connection. The name cannot exceed 128 characters, contain spaces, or contain the following special characters:
~ ` ! $ % ^ & * ( ) - + = { [ } ] | \ : ; " ' < , > . ? /
ID
The string that the Data Integration Service uses to identify the connection. The ID is not case sensitive. It must be 255 characters or less and must be unique in the domain. You cannot change this property after you create the connection. Default value is the connection name.
Description
The description of the connection. Enter a string that you can use to identify the connection. The description cannot exceed 4,000 characters.
Location
The domain where you want to create the connection. Select the domain name.
Type
The connection type. Select Messaging/AmazonKinesis.

Connection Properties

The following table describes the connection properties for the AmazonKinesis connection:
Property
Description
Service
The type of Kinesis Service that the connection is associated with.
Select one of the following service types:
  • - Kinesis Firehose. Select this service to write to Kinesis Firehose Delivery Stream.
  • - Kinesis Streams. Select this service to read from Kinesis Streams
AWS Access Key ID
The access key ID of the Amazon AWS user account.
AWS Secret Access Key
The secret access key for your Amazon AWS user account.
Region
Region where the endpoint for your service is available. You can select one of the following values:
  • - us-east-2. Indicates the US East (Ohio) region.
  • - us-east-1. Indicates the US East (N. Virginia) region.
  • - us-west-1. Indicates the US West (N. California) region.
  • - us-west-2. Indicates the US West (Oregon) region.
  • - ap-northeast-1. Indicates the Asia Pacific (Tokyo) region.
  • - ap-northeast-2. Indicates the Asia Pacific (Seoul) region.
  • - ap-northeast-3. Indicates the Asia Pacific (Osaka-Local) region.
  • - ap-south-1. Indicates the Asia Pacific (Mumbai) region.
  • - ap-southeast-1. Indicates the Asia Pacific (Singapore) region.
  • - ap-southeast-2. Indicates the Asia Pacific (Sydney) region.
  • - ca-central-1. Indicates the Canada (Central) region.
  • - cn-north-1. Indicates the China (Beijing) region.
  • - cn-northwest-1. Indicates the China (Ningxia) region.
  • - eu-central-1. Indicates the EU (Frankfurt) region.
  • - eu-west-1. Indicates the EU (Ireland) region.
  • - eu-west-2. Indicates the EU (London) region.
  • - eu-west-3. Indicates the EU (Paris) region.
  • - sa-east-1. Indicates the South America (São Paulo) region.
Connection Timeout (in msec)
Number of milliseconds that the Integration service waits to establish a connection to the Kinesis Stream or Kinesis Firehose after which it times out.

Creating an AmazonKinesis Connection using Informatica Command Line Program

You can use the infacmd command line program to create an AmazonKinesis connection. Access the command from the <Informatica installation directory>/clients/DeveloperClient/infacmd directory.
On UNIX, run the following command:
sh infacmd.sh createConnection
On Windows, run the following command:
infacmd.bat createConnection
Enter connection options in the following format:
... -o option_name=value option_name='value' ...
For example, to create an AmazonKinesis connection to Kinesis Streams on UNIX, run the following command:
infacmd createConnection -dn <domain name> -un <domain user> -pd <domain password> -cn <connection name> -cid <connection id> -ct AMAZONKINESIS -o AWS_ACCESS_KEY_ID=<access key id> AWS_SECRET_ACCESS_KEY=<secret access key> ConnectionTimeOut=10000 Region=<RegionName> ServiceType='Kinesis Streams'
To create an AmazonKinesis connection to Kinesis Firehose on UNIX, run the following command:
infacmd createConnection -dn <domain name> -un <domain user> -pd <domain password> -cn <connection name> -cid <connection id> -ct AMAZONKINESIS -o AWS_ACCESS_KEY_ID=<access key> AWS_SECRET_ACCESS_KEY=<secret access key> ConnectionTimeOut=10000 Region=<RegionName> ServiceType='Kinesis Firehose'
For more information about the CreateConnection command, see the Informatica Command Reference Guide.