Amazon S3 Connector Guide > Amazon S3 Event Sources > Amazon S3 Event Source Properties
  

Amazon S3 Event Source Properties

After you define an Amazon S3 connection and deploy it in a process, you can monitor a bucket for new objects. When the S3 Connector detects a new object, it reads the S3 object and sends it to the process that subscribed to the Event Source, which acts as a start event.
For each connection, you can add multiple event sources.
A connection begins processing objects in the bucket immediately after it is published, even if you have not yet defined any processes to consume or subscribe to the events. You can use the initial delay setting to postpone processing the first bucket until you have published a process to handle events from the Event Source.
The following table describes the file location settings applicable to all S3 event source types:
Property
Description
S3 Bucket Name
Required. Amazon S3 bucket name that contains the objects to be processed by the event source.
The question mark (?) character is prohibited in this field.
S3 Prefix
The S3 prefix that determines which objects are processed. Event source can use this attribute to consume only subset of S3 objects that share common prefix. If no prefix is specified, the event source reads all objects in the bucket.
The following table describes the Object Operations Settings applicable to all S3 event source types:
Property
Description
Delete S3 Objects
Required. Use this property to set whether you want to delete none, some, or all processed S3 objects from the source bucket.
Choose one of the following options:
  • - No: Do not delete any processed S3 object.
  • - Consumed: Delete all successfully processed S3 objects.
  • - All: Delete all, faulted or successfully processed, S3 objects.
Note: If you select No or Consumed, the connector uses the move settings to prevent an object from being consumed twice. If you have not set a Move to Bucket name, the connector uses the default bucket.
For example, if you select No, the connector uses the Move To Bucket and Move Prefix properties to move processed S3 objects.
Default: No.
Move To Bucket
Optional. Specify the bucket to which you want to move successfully processed S3 objects.
If you do not specify a Move Prefix, it is a good practice to specify a move to bucket different from the source bucket.
Default: Source bucket.
Move Prefix
Optional. Specify the prefix you want to add to the keys of successfully processed S3 objects.
If you want to move processed objects to the source bucket, it is a good practice to specify a move prefix.
Default: .ae-done/
Move Failed to Bucket
Optional. Specify the bucket to which you want to move failed S3 objects.
If you do not specify a Move Failed Prefix, it is a good practice to specify a move failed to bucket that is different from the source bucket and the move to bucket.
Default: Source bucket.
Move Failed Prefix
Optional. Specify the prefix you want to add to the keys of failed S3 objects.
If you want to move failed objects to the source bucket, it is a good practice to specify a move prefix.
Default: .ae-error/
Backup In Folder
Optional. Specify a local directory in which you want to store a copy of successfully processed S3 objects.
Backup Failed In Folder
Optional. Specify a local directory in which you want to store a copy of failed S3 objects.
The following table describes the Failure Handling Settings applicable to all S3 event source types:
Property
Description
Max Retry Interval
In case of a fault, specify the maximum time (in seconds) that you want an event source to wait before making a recovery attempt.
The delay interval between retries increases exponentially until the connector reaches the maximum retry interval.
Default maximum: 3600 seconds or 1 hour.
Minimum allowed: 1 second
Maximum allowed: 86400 seconds or 24 hours.
The following table describes the polling and other settings applicable to all S3 event source types:
Property
Description
Initial Delay
Seconds to wait before polling the S3 bucket.
Default: 1 second.
Set the initial delay that you need, for example, to publish the related process or take other required steps before polling can begin.
Delay
Seconds to wait before the next poll to check whether new files have arrived.
Default is 1 second.
Set a longer delay between polls to reduce the number of S3 API calls.
Max Messages Per Poll
The maximum number of objects to retrieve in each poll.
Default: 10.
Set a value that enables you to segment the load and manage performance if you anticipate that the monitor will find a large number of objects to process during each poll.