Use a Command Task step to run shell scripts or batch commands from multiple script files on the Secure Agent machine.
For example, you can use a command task to move a file, copy a file, zip or unzip a file, or run clean scripts or SQL scripts as part of a taskflow. You can use the Command Task outputs to orchestrate subsequent tasks in the taskflow.
When you add a Command Task step to a taskflow, you configure the following properties:
General Properties
In the general properties, you can specify a descriptive name for the Command Task step.
The name can contain only alphanumeric characters, underscores (_), spaces, and Unicode characters. The name can't contain curly brackets {}, full width lowline (＿), and hyphen (-).
Input Fields
You can use the following system variables in the input fields to define the script file name, input argument, and work directory:
•$PMRootDir
•$PMLookupFileDir
•$PMSessionLogDir
•$PMBadFileDir
•$PMCacheDir
•$PMStorageDir
•$PMTargetFileDir
•$PMSourceFileDir
•$PMExtProcDir
•$PMTempDir
•$PMWorkflowLogDir
These variables are pre-defined for the Data Integration Server service in Administrator. To use the system variables in the Command Task step, the Common Integration Components and Data Integration Server services must be enabled and up and running on the Secure Agent.
You can use environment variables in the input fields to define the script file name, input arguments, and work directory. To use environment variables, the Secure Agent must have the Common Integration Components service version 14 or later and the command executor package version 140 or later.
Configure the following input fields for a Command Task step:
Fail task if any script fails
When you use multiple script files, you can configure a Command Task step to fail if any script fails.
When you configure one or more scripts in a Command Task step and select this option, if any script fails, the status of the Command Task step is set to failed and the taskflow does not run further scripts or the subsequent steps.
When you configure one or more scripts in a Command Task step and disable this option, the taskflow runs all the scripts. If any script fails, the status of the failed script is set to failed and the status of the Command Task step is set to warning on the All Jobs, Running Jobs, and My Jobs pages. However, Data Integration treats the Command Task step execution as a success.
This option is disabled by default.
Runtime Environment
Required. The runtime environment that runs the command.
Use one of the following options to specify the value for this field:
- Content. Select a Secure Agent group.
- Field. Select the field that the Taskflow Designer uses to write the runtime environment into this field when this step executes. You can select a runtime environment that was added as an input field or temporary field in any other step in the taskflow. The value must be a valid secure agent group name.
- Formula. Create an expression for the runtime environment with a valid secure agent group name.
Note: When you import and open an attached Taskflow, a task command may appear stating that the runtime doesn't exist, and the selected agent may not be visible in the Agent Group dropdown list. To view the updated list of agent groups, clear your browser cache and reload the page.
Max Wait (Seconds)
In the Max Wait (Seconds) field, you can configure the maximum length of time in seconds that the Command Task step waits for the task to complete. Specify a value between 1 and 86400 seconds. Default is 86400 seconds. If the task is not completed within the maximum time specified in the field, the task stops running and the subsequent task in the taskflow starts running.
Note: If the specified value is lesser than 1 or greater than 86400, the maximum wait time is automatically set to 86400 seconds.
Scripts
In a Command Task step, you can add multiple scripts. You can configure a script by specifying the script file name, input arguments, and work directory.
To add more scripts, click the Add icon from the Scripts panel. If you have multiple scripts in the command task, you can drag and drop the scripts to reorder the scripts.
To run scripts from different Secure Agent groups, you must add separate Command Task steps for different runtime environments.
You can view all the scripts in a taskflow instance as subtasks of the command task from the All Jobs, Running Jobs, or My Jobs page. If a script fails, you can also download the log file to understand the reason for the script failure.
Configure the following input fields for a script:
Script File Name
Required. The path and name of the script file that you want to run.
Use one of the following options to specify the value for this field:
▪ Content. Enter the path to the script that you want to run.
▪ Field. Select the field that the Taskflow Designer uses to write the script file name into this field when this step executes. You can select a script file name that was added as an input field, temporary field, or output field in any other step in the taskflow.
▪ Formula. Create an expression for the script file.
You can use system variables to define the script file that you want to run.
For example, to run the script.bat file located in the root directory, you can enter the value as $PMRootDir/script.bat.
You can use an environment variable to define the script file path based on the operating system.
For example, you have a java_script.bat file located in the following directory:
C:\Scripts
You have created an environment variable named ScriptHome for the directory. To run the java_script.bat file, you can enter the value as %ScriptHome%\java_script.bat for Windows and $ScriptHome\java_script.bat for Linux.
You can also create an environment variable for the full path including the script file name.
You can also provide the EFS or NFS mounted directories including the script file name to run the scripts.
For example, you have an aws_script1.sh file located in the following EFS or NFS directory:
mountDir/scripts
To run the aws_script1.sh file, you can enter the value as /mountDir/scripts/aws_script1.sh.
For more information about EFS and NFS, see Runtime Environments.
Input Arguments
Optional. The arguments that you want to pass to the script when it is executed.
Use one of the following options to specify the value for this field:
▪ Content. Enter one or more arguments that you want to pass to the script. Enclose each argument and values in double quotes (") and separate arguments with a comma (,).
For example, if you want to execute the following command:
▪ Field. Select the field that the Taskflow Designer uses to write the input arguments into this field when this step executes. You can select an input argument that was added as an input field, temporary field, or output field in any other step in the taskflow.
▪ Formula. Create an expression for the input arguments. Enclose each argument in double quotes (") and separate arguments with a comma (,).
You can define input arguments in the following ways:
Using system variables
You can use system variables to define the input arguments that you want to pass to the script when it is executed. For example, to pass the arguments from the root directory or the temp directory, enter the following value:
"$PMRootDir","$PMTempDir"
Using environment variables
You can use an environment variable to define the input arguments based on the operating system. For example, you have created an environment variable named ScriptHome for the following directory:
C:\Scripts
To pass the arguments from this directory, enter the value as "%ScriptHome%" for Windows and "$ScriptHome" for Linux.
Work Directory
Optional. The path to the working directory where the output of the script is saved. By default, the output is saved in the path where the script is saved.
Use one of the following options to specify the value for this field:
▪ Content. Enter the path to the working directory where you want to save the output of the script.
▪ Field. Select the field that the Taskflow Designer uses to write the working directory into this field when this step executes. You can select a work directory that was added as an input field, temporary field, or output field in any other step in the taskflow.
▪ Formula. Create an expression for the work directory.
You can use system variables to define the working directory where you want the output of the script to be saved.
For example, if you want to use the source file directory as the working directory, you can enter the value as $PMSourceFileDir.
You can use an environment variable to define the working directory based on the operating system.
For example, you created an environment variable named ScriptHome for the following directory:
C:\Scripts
To use this directory as the working directory, you can enter the value as %ScriptHome% for Windows and $ScriptHome for Linux.
Note: When you use curly brackets {} in the script or in the input fields, you must add an additional set of curly brackets {{}}. Otherwise, an error occurs. This is because curly brackets are considered as special characters in XQuery.
Output Fields
When you run a taskflow, the following output fields are generated for the Command Task step:
Output Field
Type
Description
Run Id
Text
The run ID of the command task.
Start Time
Date Time
The start time of the command task.
End Time
Date Time
The end time of the command task.
Exit Code
Integer
The exit code returned after command task execution. The exit code can have one of the following values:
- 0. Indicates that the command task was executed successfully.
- 1. Indicates that the command task failed.
Execution Status
Text
Displays the status of the command task as successful.
Std Error
Text
Displays the error message.
To view the values of each output field, run the taskflow and go to the Taskflow Instance Detail page in Monitor. For more information about the Taskflow Instance Detail page, see Monitor.
Events
Configure the Events properties to add timers to a command task.
Use a Timer event to perform an action based on a schedule. The action could be either at a specific time or after an interval.
When you add a timer to a Command Task step, a new branch appears. Add an event to this branch and specify whether you want the event to run At a specific time or After an interval.
In the following image, the event on the timer branch is a Data Decision step (Decision 1) that occurs five minutes after the main command task (Command Task 1) starts:
1Main branch
2Timer branch
In the example, the timer branch runs five minutes after Command Task 1 starts. If Command Task 1 finishes before Decision 1, the timer branch is not executed.
You can select the Interrupting option if you want the timer to interrupt the main command task.
The following image shows an interrupting timer set to occur five minutes after the main command task starts:
1Main branch
2Timer branch
In the example, Command Task 2 executes after five minutes and interrupts Command Task 1. The taskflow executes only Command Task 2 and then ends. Command Task 1 has no output in this taskflow instance. You see no output fields for Command Task 1 in the job details for the taskflow.
If Command Task 1 completes before the timer, the taskflow executes only Command Task 1 and ends.
If you delete the End step on the timer branch of an interrupting timer, the timer branch rejoins the main branch.
The following image shows an interrupting timer branch with the End step deleted:
1Main branch
2Timer branch
In the example, Command Task 2 executes after five minutes and interrupts Command Task 1. The timer branch rejoins the main branch. The taskflow executes Command Task 2, a Decision step, a Parallel Paths step, and then ends.
If Command Task 1 completes before the timer, the taskflow executes Command Task 1, a Decision step, a Parallel Paths step, and then ends.
Error Handling
Use the Error Handling tab to indicate how you want the taskflow to behave when a Command Task step encounters an error. You can also configure the taskflow behavior when the Command Task step fails or does not run.
After you select a task, configure the following error handling properties:
On Error
The path that a taskflow takes when it encounters an error in a Command Task step.
An error occurs when a Command Task step fails. You can choose from the following options:
- Select Ignore to ignore the error and move to the next step.
Note: If you select Ignore for a Command Task step with a subsequent Notification Task step and the command task fails, the email notification that you receive does not contain the fault details. To get the fault details in the email, select Custom error handling.
- Select Suspend Taskflow to move the taskflow to the suspended state when it encounters an error. You can resume the taskflow instance from the All Jobs, Running Jobs, or My Jobs page.
The taskflow resumes from the step at which it was suspended. If you know the reason for the error, correct the issue and then resume the taskflow.
- Select Custom error handling to handle the error in a manner you choose. If you select Custom error handling, two branches appear. The first branch is the path the taskflow follows if no error occurs. The second branch is the custom path the taskflow follows if an error occurs.
Default is Suspend Taskflow.
Fail taskflow on completion
The taskflow behavior when the Command Task step fails or does not run.
You can configure a taskflow to fail on its completion if the Command Task step fails or does not run. If the step fails or does not run, the taskflow continues running the subsequent steps. However, after the taskflow completes, the taskflow status is set to failed.
If you configure both the Suspend on Fault taskflow advanced property and the Fail taskflow on completion property, the Suspend on Fault property takes precedence. In this case, if the Command Task step fails or does not run, the taskflow is suspended. The taskflow does not run the subsequent steps after the Command Task step.
The following image shows a Custom error handling path with an Assignment step and another Command Task step:
Fault
The fault fields are displayed only if the command task fails due to a script failure. The details help you analyze the reason for the fault. You can then take an appropriate action on the faulted command task and proceed with the execution of the taskflow.
You can add the details in fault fields as parameters to the subsequent steps of the taskflow as shown in the following image:
If the command task has faulted, you see the following fault details:
Property
Type
Description
Run Id
Text
The run ID of the command task.
Start Time
Date Time
The start time of the command task.
End Time
Date Time
The end time of the command task.
Exit Code
Integer
The exit code returned after the command task execution. The exit code can have values between 0 and 255. The value 0 indicates that the command task ran successfully.
A failed command returns a non-zero value. The value can be based on the error type or pre-configured code in the shell script.
Execution Status
Text
Displays the status of the command task as failed.
