Maximum Number of Concurrent Jobs on a Grid
You can increase the maximum number of concurrent jobs that a Data Integration Service grid can run.
The Maximum Execution Pool Size property on the Data Integration Service determines the maximum number of jobs that each Data Integration Service process can run concurrently. Jobs include data previews, mappings, profiling jobs, SQL queries, and web service requests. The default value is 10.
When the Data Integration Service runs on a grid, the maximum number of jobs that can run concurrently across the grid is calculated as follows:
Maximum Execution Pool Size * Number of running service processes
For example, a Data Integration Service grid includes three running service processes. If you set the value to 10, each Data Integration Service process can run up to 10 jobs concurrently. A total of 30 jobs can run concurrently on the grid.
When you increase the pool size value, the Data Integration Service uses more hardware resources such as CPU, memory, and system I/O. Set this value based on the resources available on the nodes in the grid. For example, consider the number of CPUs on the machines where Data Integration Service processes run and the amount of memory that is available to the Data Integration Service.
Note: If the Data Integration Service grid runs jobs in separate remote processes, additional concurrent jobs might not run on compute nodes after you increase the value of this property. You might need to override compute node attributes to increase the number of concurrent jobs on each compute node. For more information, see
Override Compute Node Attributes to Increase Concurrent Jobs.