Importing data into your application from an external system

To add data from an external source to your application, import a .csv file.

You can import data from a variety of sources, even if they use different field names or types. During the import process, you configure how to process the data, and map the fields in the source .csv file to the fields in your data type.

Importing data from a file

To add data to your data model more quickly, import data from a .csv file.

If your operator ID has Allow rule checkout enabled, import performance might be impacted. Use an operator ID that does not have Allow rule checkout enabled, or disable this option for your operator ID. For more information, see Defining security information for an operator. To view a list of operators that have Allow rule checkout enabled, click View Operators with 'Allow rule checkout' enabled at the bottom of the Migration settings landing page.
Note: You cannot import data by using a .csv file if you are using an Apache Cassandra database.
  1. Upload the file.
    1. In the navigation panel of App Studio, click Data.
    2. Click Data types and integrations.
    3. Click a data type.
    4. On the Records tab, click Import.
    5. In the Purpose list, select Add only.
    6. Click Choose File.
    7. Navigate to your .csv file, and then click Open.
    8. Click Next.
  2. Map the columns in your file to the fields in the data type.
    Tip: To make mapping faster, select a template from the Template type list to use to define the mapping between the fields in your data type and the fields in the .csv file and skip the rest of this step.
    1. In the Target field column, select the fields in your data type that correspond to the fields in the .csv file shown in the Source field column, or click Select to display a dialog box for searching and filtering fields.
      Note: You can select top level and embedded properties as targets for import.
    2. Optional: If you are importing fields from an external system, you can apply business logic, such as lookups, decision trees, and decision tables to translate the external data into fields that are understood by Pega Platform. In the Mapping options column, click the Mapping options icon to select the type of business logic to use for translation, and click Submit. For more information, see Applying business logic when importing data.
    3. Optional: Enter a default value. For new records, the default value is used if the source field is blank. For existing records, the default value is used if both the source and target fields are blank. If you use a lookup, decision tree, or decision table, the source value is the value obtained from the lookup, tree, or table. It is not the value in the .csv file.
    4. Optional: Set defaults for fields that do not have matching source columns in the .csv file.
      1. Click View custom defaults.
      2. Click Add default value.
      3. Enter the target field or click Select to choose the target field from a list of fields.
      4. Enter the default value. For new records, the default value is used if it is provided. For existing records, the default value is used if the target value is blank.
      5. Click Next if you are finished mapping fields, or click Back to mapping to finish mapping fields.
  3. Optional: If you are using a dashboard gadget in your end user portal that displays in-progress data imports and recently completed data imports, enter a short description that describes the import.
  4. Select a validation option.
    • To skip validation, Skip validation step entirely.
    • To perform basic validation, Skip running validate rules.
  5. Click Start validation.
  6. Click Continue import.
    You can close the dialog box for the data import process. The process runs asynchronously and is visible on your worklist.
  7. Click Finish.

Troubleshooting import errors

To improve application quality, resolve the errors about the data that you import.

When you import data into a data type, a .csv file is generated if there are any errors. The .csv file contains the error details for the records for each row in the file. You can fix the errors and reimport the data. You can change the location to which the .csv file is written. For example, for a multi-node system, you can set the destination to a shared location so that a user can access the file from any node in the system.

The following issues can arise when you import data into your data model: