Edit online
A distribution test simulation provides valuable insights into the performance of a Decision Strategy by analyzing how Actions are distributed and prioritized. You can use this method to refine Engagement Policies, optimize decision-making, and ensure that actions align with business goals. By using built-in reports, you can make data-driven decisions to enhance customer interactions and improve strategic outcomes.

Distribution test overview

Distribution testing helps validate changes in Engagement Policies or Decision Strategies by running a simulation on a defined customer population. The results provide insights into Action distribution across various channels, average Action Propensity, and the ranking of Actions. Understanding these metrics is crucial when implementing or modifying Decision Strategies, as it helps predict the potential impact of changes.

The test runs a simulation on an input audience, outputting the results to a database table, and analyzing the data through built-in simulation reports. These reports help assess the distribution and prioritization of Actions within a Strategy, ensuring optimal decision-making outcomes.

When to run

Run the distribution test simulation before implementing any Next Best Action logic changes in a production environment. This test ensures that proposed adjustments maintain an appropriate balance across different Actions and Treatments while preventing unintended exclusions.

Expected runtime

The distribution test simulation runs within the same timeframe as a standard Next Best Action batch process. Since it follows the regular batch running pattern, it provides timely insights without introducing significant delays in the deployment cycle.

Sample size

Use a standard sample size of 100,000 records for a distribution test simulation. For the final validation stage, expand the sample to include the entire inbound or outbound dataset for comprehensive coverage. In cases where Policy rules apply to specific scenarios, certain Actions may occur infrequently. Even with a large sample size, some Actions may not always appear in the results due to their inherently limited occurrences.

Visual Business Director integration

Another possible output destination for a distribution test is a Visual Business Director (VBD) data source. This option provides analysis of the impact of Contact Policy changes over time. By running a baseline simulation using the current Engagement Policies and comparing it with a subsequent simulation after making changes, VBD enables a visual comparison of differences at the Action level.

The VBD delta mode highlights variations, such as an increase in gold credit card offers or a reduction in other product offerings. For more information, see Visual Business Director .

Edit online

Running a distribution test simulation

A distribution test helps you validate changes in Engagement Policies or Decision Strategies by running a simulation on a defined customer population. The results provide insights into Action distribution across various channels, average Action Propensity, and the ranking of Actions. This simulation help assess the distribution and prioritization of Actions within a Strategy, ensuring optimal decision-making outcomes.

Defining an audience for testing
  1. In the navigation pane of Pega Customer Decision Hub, click Testing > Simulations.
  2. In the upper-right corner, click Create, and then select Distribution test.
  3. In the Configure inputs section, in the Strategy tile, click Configure to select a Strategy on which you want to run the simulation.

    Configure inputs and define audience to run distribution test

    Configuration of the distribution test
  4. In the Strategy dialog box, in the Trigger_NBA_Grow_Creditcards Strategy, click Add > Apply to select the Strategy.
  5. In the Configure inputs section, in the Audience tile, click Configure to select an audience on which you want to run the simulation.
  6. In the Audience dialog box, in the SampledCustomers_Inbound audience, click Add > Apply.
  7. In the Simulation name field, enter DistributionTest.
  8. Click Done.

Distribution test setting with selected parameters

Selected configuration of distribution test
Configuring output destinations
  1. In the Assign output destinations section, click Create new.
  2. In the Create new output dialog box, configure the following information:
    1. In the Name field, enter SimulationResults.
    2. In the Type list, select Database Table.
    3. Click Done.
  3. In the Assign reports to outputs section, click Add report new to add a new output destination.
  4. In the Assign reports to outputs dialog box, add the following report:
    1. In the Output list, select SimulationResults.
    2. In the Report category list, select Simulations.
    3. In the Report list, select Action distribution.
  5. Click Submit and run.
Reviewing the simulation run results
  1. Once the simulation run is complete, click Review reports.
  2. Open the Action distribution report.
  3. View the volume of individual Actions.

    Distribution test report showing the volume per Action

    Distribution test report
    Note: The Monte Carlo Data Set generates a mock Data Set. Different simulation runs have different results.
  4. Close the report.
Edit online

Analyzing results of a distribution test

Distribution testing evaluates the effectiveness of Engagement Policies and Decision Strategies by simulating their impact on a specific customer population. It provides key insights into Action distribution, Propensity, and prioritization across various channels. By running this test you can refine your Strategies to enhance decision-making and predict the outcomes of Policy changes.

Analyzing results of a distribution test

Two primary reports are included in evaluating simulation results:
Proposition distribution
Proposition distribution hows the volume of each action generated by the Decision Strategy. This report provides a clear view of the total number of actions and their frequency of occurrence.

Proposition distribution report showing the volume of each action

Proposition distribution
Proposition priority
Proposition priority shows the average priority assigned to each Action. Higher-priority Actions, such as the "RewardsPlus Card" offer, are expected to perform better and are more likely to be presented to customers. Lower-priority Actions may have reduced visibility.

Proposition priority report showing the average priority of each action

Proposition priority