MongoDB Data Synchronization Example
A large online music store, Moose, uses MongoDB as the operational data store for the business inventory details.
The business analysts at Moose use a business intelligence tool that does not support reading data from MongoDB. The tool requires the input data to be in a relational database or a flat file. The data warehouse includes a collection called Music_Contents and Movie_Contents. The collection Music_Contents contains a catalog of all of the songs in the store. The collection Movie_Contents contains the details of the movie DVDs. You need to move the data in both the collections to a flat file to use the data for business analysis.
Create a mapping with the MongoDB data objects as the read transformation to read the records from the collection. Include a flat file data object as the target in the mapping so that the business intelligence tool can consume the data.
You perform the following Data Synchronization task:
- Define the Data Synchronization task.
- Configure a Data Synchronization task to use the insert operation.
- Create a MongoDB source.
- The source objects for a Data Synchronization task is a MongoDB collection called Music_Contents and Movie_Contents. Use a ODBC connection to connect to the data warehouse and read the data. Use multiple sources to include the MongoDB source objects in the Data Synchronization task. Use a user defined join on the name of the music collection and the name of the movie content.
- The following sample document shows the contents of the Music_Contents source object:
{
"Name" : "City Lights",
"Artist" : ["Patty Hill", "Mildred J. Hill", "Derek Underhill"],
"Units" : 1000,
"Price" : {
"Cost_Price" : 1,
"Sale_Price" : 3
}
}
- The following sample document shows the contents of the Movie_Contents source object:
{
"Name" : "City Lights",
"Type" : "Blu-ray",
"Director" : "Charlie Chaplin",
"Artist1" : "Charle Chaplin",
"Artist2" : ["Mildred J. Hill", "Derek Underhill"],
"Units" : 1000,
"Price" : {
"Cost_Price" : 10,
"Sale_Price" : 15
}
}
- Create a flat file target.
- The target in a Data Synchronization task is a flat file. Use the flat file connection to connect to the flat file. Use the FF_MDB_Inventory as the target object in the Data Synchronization task.
Include the Name, Type, Director, Artist1, Artist2, Units, and Price target fields from the source object that you want to insert into the target object.
- Configure a field mapping.
- Map the fields of the MongoDB source object to the flat file target object.
- Save and run the task. The Secure Agent retrieves the data from the MongoDB collections and writes to the flat file based on the fields you mapped in the Data Synchronization task.
The following sample shows the data that the Secure Agent writes to the flat file:
{
"Name" : "City Lights",
"Type" : "Blu-ray",
"Director" : "Charlie Chaplin",
"Artist1" : "Charle Chaplin",
"Artist2" : ["Mildred J. Hill", "Derek Underhill"],
"Units" : 1000,
"Price" : {
"Cost_Price" : 10,
"Sale_Price" : 15
},
"AudioCDArtist" : ["Patty Hill", "Mildred J. Hill", "Derek Underhill"],
"AudioCDUnits" : 1000,
"AudioCDPrice" : {
"AudioCDCost_Price" : 1,
"AudioCDSale_Price" : 3
}
}
Business analysts can use the information to analyze the data.