Data flow run limits
A large number of data flow runs that are active at the same time can deplete your system resources. To ensure the efficient processing of data flows, you can configure dynamic system settings to limit the number of concurrent active data flow runs for a node type.
If the number of active runs reaches the limit, the system adds subsequent runs to a queue. When the Data Flow service has enough capacity, which means that there are fewer active runs for the node type than the configured limit, the oldest queued runs start automatically.
You can limit the number of active data flow runs by using the following dynamic system settings:
- dataflow/max_active_runs
- This setting defines the maximum number of active runs for node types that do not have their own limits configured.
- dataflow/<node_type>/max_active_runs
- A dynamic system setting in this format defines the maximum number of active runs for a particular node type. The <node_type> value is case-sensitive, for example, BackgroundProcessing.
The following table shows the max_active_runs dynamic system settings that are included with the system, with their default values. You can modify these settings or create additional settings for other node types. When the value is set to a number greater than zero, the specified limit is used. When the value is set to –1, an unlimited number of active runs is allowed.
| Dynamic system setting | Default limit |
|---|---|
| dataflow/max_active_runs | 10 |
| dataflow/Batch/max_active_runs | 10 |
| dataflow/BackgroundProcessing/max_active_runs | –1 |
| dataflow/RealTime/max_active_runs | –1 |
| dataflow/Search/max_active_runs | –1 |