Data Synchronization Task Example
You are a data administrator in a product organization. You want to collate legacy sales data from multiple sources and archive it on Amazon S3.
You can read data from multiple sources and use Amazon S3 Connector to upload data to Amazon S3. Configure a Data Synchronization task to consolidate sales data based on product ID and upload data to Amazon S3.
You perform the following Data Synchronization tasks:
- Define the Data Synchronization task.
- Configure a Data Synchronization task to use the insert operation.
- Create the MySQL source objects.
- The source for the mapping is a MySQL connection that connects to the sales data. The MySQL object contains multiple source objects in the Data Synchronization task.
- The sales_record MySQL object includes the Row_id, Order_id, Order_Quantity, Order_Date, Unit_Price, Region, and Product_Category source fields.
- The dim_product MySQL object includes the Product_ID, Product_Name, and Load_Date source fields.
- Define a relationship the multiple source objects.
- Add a join condition between the source field in the sales_record object and the dim_product object to define the following relationship: sales_record.Product_Category=dim_product.Product_ID
- Create an Amazon S3 target object.
- The target for the mapping is an Amazon S3 bucket. Specify the target connection as Amazon S3, the target object as the name of the delimited file into which you want to insert the data. Select the target operation as the insert operation.
- Configure a field mapping.
- Map the source fields to the target fields.
- When you run the task, the Data Synchronization application writes the collated source data to the target delimited file. If you specify the name of an existing file, the Secure Agent replaces all data in the file.
The following image shows a mapping of the MySQL source objects and the Amazon S3 target file: