Introduction to IDMC System IPU Metering Log Capture recipe

The recipe enhances the IDMC system visibility to enable effective monitoring, troubleshooting, and overall management of users' IDMC environments. It facilitates custom analytics and charge back requirements by enabling processes to systematically store internal IDMC log data to a user's cloud storage.

The IDMC System IPU Metering Log Capture recipe includes a taskflow designed to periodically extract system IPU metering information from an IDMC organization and write it to a cloud storage location. The taskflow extracts SUMMARY, ASSET, and PROJECT_FOLDER IPU metering information.

The taskflow is designed to run periodically, with the default schedule set run once a day and extract IPU information from the previous day. For example, if the taskflow runs at 1 AM on April 11, it will extract log data from midnight April 9 to midnight April 10. With proper configuration, it can retrieve data from any IDMC organization or sub-organization, not necessarily limited to the organization where the recipe is installed.

The target location and type, such as AWS, Azure, Google, or Oracle, are determined by the connection object assigned to the taskflow’s mapping task, along with the target load path provided in the in_tgtPath input parameter.

This recipe is designed to run on Linux because it employs shell scripts to manage staging files. It utilizes both Data Integration and Application Integration, and it is not compatible with a serverless Application Integration server.