Amazon S3 Connector sources and targets represent delimited file data objects that are read from or written to Amazon S3 buckets as delimited files.
Use Amazon S3 Connector to read delimited files from Amazon S3 and to insert data to delimited files in Amazon S3 buckets.
Amazon S3 Object Format
Amazon S3 objects are delimited files. All fields in a delimited file are of string data type with a data format that you cannot change and with a defined precision of 256. Data in Amazon S3 delimited files is written in String 256 format.
An Amazon S3 delimited file uses the following data format by default:
•The delimiter is a comma.
•The qualifier is a double-quote.
•The escape character is a backslash.
Use Formatting Options to override the default data format values.
When you write data to an Amazon S3 file, the application might display an exception when you select incorrect Formatting Options. You must select valid Formatting Options and proceed with the task.
Ensure that the Amazon S3 file that you read is in a valid format. If the format is not valid, a null pointer exception occurs and an appropriate error message does not appear.
Backslash is the default escape character in the formatting options. Specify a different escape character when you read data from an Amazon file and escape is a part of data.
Note: The Formatting Options dialog box does not display backslash as the default escape character. However, backslash is the default escape character. Specify a different escape character when you read from an Amazon S3 file and escape is a part of data.
When you write data to an Amazon S3 file, if there is a single or double quote in the source data, an extra quote is added to the target.
cannot specify space, semi colon, and comma as delimiters in the Other option under Formatting Options.
Data Encryption in Amazon Targets
To protect data, you can enable server-side encryption or client-side encryption to encrypt data inserted in Amazon S3 buckets. You can encrypt data by using the master symmetric key.
Note: If you use AWS KMS to encrypt data, you require the tag "s3:x-amz-server-side-encryption":"aws:kms" in the Amazon S3 bucket policy irrespective of the allow or deny permission configured for the Amazon S3 bucket policy.
Master symmetric key is a 256-bit AES encryption key in the Base64 format that is used to enable client-side encryption. You can generate master symmetric key by using a third-party tool.
Server-side Encryption
Enable server-side encryption if you want to use Amazon S3-managed encryption key to encrypt the data while uploading the delimited files to the buckets. To enable server-side encryption, select Server Side Encryption as the encryption type in the advanced target properties.
Client-side Encryption
Enable client-side encryption if you want the Secure Agent to encrypt the data while uploading the delimited files to the buckets. To enable client-side encryption, you must provide a master symmetric key when you create an Amazon connection. Then, select Client Side Encryption as the encryption type in the advanced target properties.