Amazon S3 Objects
Amazon S3 Connector sources and targets represent delimited file data objects that are read from or written to Amazon S3 buckets as delimited files.
Use Amazon S3 Connector to read delimited files from Amazon S3 and to insert data to delimited files in Amazon S3 buckets.
Amazon S3 Object Format
Amazon S3 objects are delimited files. All fields in a delimited file are of string data type with a data format that you cannot change and with a defined precision of 256. Data in Amazon S3 delimited files is written in String 256 format.
Amazon S3 Connector accepts target data with a precision greater than 256. You do not need to change the precision in the Target transformation.
To read source data with a precision greater than 256, increase the precision in the Source transformation to view the complete data.
To write Amazon S3 source data to any relational target data source, you can specify field expressions in the Field Mapping page. The Secure Agent converts the Amazon S3 string data to the target data format.
An Amazon S3 delimited file uses the following data format by default:
- •The delimiter is a comma.
- •The qualifier is a double-quote.
- •The escape character is a backslash.
Use Formatting Options to override the default data format values.
When you read data from or write data to an Amazon S3 file, the application might display an exception when you select incorrect Formatting Options. You must select valid Formatting Options and proceed with the task.
Backslash is the default escape character in the formatting options. Specify a different escape character when you read data from an Amazon S3 file and escape is a part of data.
When you write data to an Amazon S3 file, if there is a single or double quote in the source data, an extra quote is added to the target.
For Amazon S3, you cannot specify space, semi colon, and comma as delimiters in the Other option under Formatting Options.
Data Encryption in Amazon S3 Targets
To protect data, you can enable server-side encryption or client-side encryption to encrypt data inserted in Amazon S3 buckets. You can encrypt data by using the master symmetric key or customer master key. Do not use the master symmetric key and customer master key together.
Customer master key is a user managed key generated by AWS Key Management Service (AWS KMS) to encrypt data.
Master symmetric key is a 256-bit AES encryption key in the Base64 format that is used to enable client-side encryption. You can generate master symmetric key by using a third-party tool.
Server-side Encryption
Enable server-side encryption if you want to use Amazon S3-managed encryption key or AWS KMS-managed customer master key to encrypt the data while uploading the delimited files to the buckets. To enable server-side encryption, select Server Side Encryption as the encryption type in the advanced target properties on the Schedule page.
Client-side Encryption
Enable client-side encryption if you want the Secure Agent to encrypt the data while uploading the delimited files to the buckets. To enable client-side encryption, perform the following tasks:
- 1. Provide a master symmetric key or customer master key ID when you create an Amazon S3 connection.
Note: The administrator user of the account can use the default customer master key ID to enable the client-side encryption.
- 2. Select Client Side Encryption as the encryption type in the advanced target properties on the Schedule page.
- 3. Ensure that an organization administrator updates the security policy .jar files on each Secure Agent machine in the runtime environment.