Changes to the architecture of the Data Flow service
Valid from Pega Version 8.4
In Pega Platform™ 8.4, the architecture of batch and real-time data flows uses improved node handling to increase the stability of data flow runs. As a result, there are fewer interactions with the database and between the nodes, resulting in increased resilience of the Data Flow service.
If you upgrade from a previous version of Pega Plaftorm, see the following list for an overview of the changes in the behavior of the Data Flow service compared to previous versions:
- Updates to lifecycle actions
- Starting a run
- Triggering pre- and post-activities
- Selecting a node fail policy
- No service nodes and active runs
Nodes no longer communicate and trigger each other, but run periodic tasks instead. As such, triggering a new run does not cause the service nodes to immediately start the run. Instead, the run starts a few seconds later. The same applies to user actions such as stopping, starting, and updating the run. The system also processes topology changes as periodic tasks, so it might take a few minutes for new nodes to join runs, or for partitions to redistribute when a node leaves a run.
To make lifecycle actions more intuitive, the Stop action consolidates both the Stop and Pause actions. The Start action consolidates both the Resume and Start actions.
You can resume or restart stopped and failed runs with the Start and Restart actions. The Start action is only available for resumable runs and continues the run from where it stopped. The Restart action causes the run to process from the beginning. Completed runs can only be restarted. If a run completes with failures, you can restart it from the beginning, or process only the errors by using the Reprocess failures action.
New data flow runs have the Initializing status, and start automatically. You no longer need to manually start a new run, so the New status is now removed.
If there are no nodes available to process a run, the run gets the Queued status and waits for an available node.
The system now triggers pre-activities on a random service node, rather than on the node that triggered the run.
The system triggers post-activities only for runs that complete, fail, or complete with failures. If you manually stop a run with the Stop action, the post-activity does not trigger. However, restarting the run with the Restart action triggers first the post-activity, and then the pre-activity.
You can no longer choose to run pre- and post-activities on all nodes.
For resumable runs, you can no longer select a node fail policy. If a node fails, the partitions assigned to that node automatically continue the run on different nodes.
For non-resumable runs, you can choose to restart the partitions assigned to the failed node on different nodes, or to fail the partitions assigned to the failed node.
If the last data flow node for an in-progress run fails, the run remains in the In Progress state, even if no processing takes place. This behavior results from the fact that data flow architecture now prevents unrelated nodes from affecting runs.
Discontinued support for list view and summary view rules
Valid from Pega Version 7.2
The list view rule and the summary view rule are deprecated, and support for these rules has been discontinued. Existing list view and summary view reports will continue to work on supported browsers. However, the last version of Internet Explorer that list view and summary view rules support is Internet Explorer 11.
You cannot create new list view or summary view reports. Instead, create list reports and summarized reports by using the report definition rule. To prevent display issues, re-create the custom list view and summary view reports that you need as report definition reports.