To monitor all the models that are part of an adaptive model, use the Monitor tab of an adaptive model in Prediction Studio. The predictive performance and success rate of individual models provide information that can help business users and strategy designers refine decision strategies and adaptive models.
In the bubble chart that is displayed on the Monitoring tab, each bubble represents a model for a specific proposition. The size of a bubble represents the number of responses (positive and negative). When you hover the cursor over a bubble, you can view the number of responses, the performance, and the success rate.
The Performance axis indicates the accuracy of the outcome prediction. The model performance is expressed in Area Under the Curve (AUC) unit of measurement, which has a range between 50 and 100. The higher the AUC, the better a model is at predicting the outcome.
The Success rate axis indicates the success rate expressed in percentages. The system calculates this rate by dividing the number of positive responses by the total number of responses.
Adaptive models table
The data that is used to visualize the models in the bubble chart is displayed below the
chart. Models are identified by their model context (business issue, group, proposition
name, and so on). For each model response count, success rate and performance is shown.
- Viewing summarized adaptive model reports
You can create customized reports that pertain to all adaptive models that you created in Prediction Studio. You can download those reports as PDF or CSV files to view them outside of your application.
- Generating and downloading a model report
Download a report that contains a summary on all predictive, adaptive, and text analytics models in your application. Reports contain data that can help you evaluate the model predictive performance (for example, area under the curve). You might need to store model reports for auditing purposes, or share them with other people in your organization who do not have access to Pega Platform. In the reports, you can check the status and predictive performance of the models and identify who made the last change to the model and when.
- Viewing a model report
To analyze an adaptive model, you can view a detailed model report that lists active predictors, inactive predictors, the score distribution, and a trend graph of model performance. You can also zoom into a predictor distribution.
- Viewing the predictors overview
In the predictors overview you can see how often a predictor is actively used in a model. This overview can help you identify predictors that are not used in any of the models.