Deploy and Run the Workflow
After you complete the cluster workflow, deploy and run the workflow.
You can monitor cluster provisioning tasks on the AWS or Azure web console. If you configured a log location, view the logs at the location that you configured in the Create Cluster task properties.
You can also monitor Data Integration Service jobs in the Administrator tool.
Note: After the workflow begins executing tasks, the task to provision the cluster may take several minutes.
Monitoring Azure HDInsight Cluster Workflow Jobs
You can access mapping log URLs through the Monitoring tab in the Administrator tool to monitor workflow jobs that run on an Azure HDInsight cluster. The log location depends on the run-time engine that each mapping uses.
Blaze and Spark engines
To access the monitoring URL for mappings that run on Blaze or Spark, expand the workflow and the mapping in the Monitoring tab. Select the Grid Task and view the value for the Monitoring URL property in the lower pane. Use this path to find the log.
Hive engine
To access the monitoring URL for mappings that run on Hive, expand the workflow and the mapping in the Monitoring tab. Select a Hive Query job, and then expand the MR Job Details node in the lower pane. The Job ID is hyperlinked, but clicking on the link does not lead to the log. To find the job monitoring log, copy the URL path and read it to find the log. Repeat the steps for each Hive Query job.