Big Data Management User Guide > Processing Hierarchical Data on the Spark Engine > Type Configuration
  

Type Configuration

A type configuration is a set of complex port properties that specify the data type of the complex data type elements or the schema of the data. After you create a complex port, you specify or change the type configuration for the complex port on the Ports tab.
The type configuration for a complex port depends on the complex data type assigned to the port. You must specify a complex data type definition for a struct port or a struct element in a complex port. Array and map ports have a default type configuration that you can change.
The following table lists the type configuration properties for complex ports:
Complex Port
Type Configuration Properties
array
  • - Type. Data type of the elements of an array. Default is string.
  • - Array Dimension. Number of levels for an array. Default is 1.
  • - Precision. Maximum number of significant digits for numeric data types, or maximum number of characters for string data types.
  • - Scale. Maximum number of digits after the decimal point of numeric values.
map
  • - Type. Data type of the key-value pair. Default is string for key and value.
  • - Precision. Maximum number of significant digits for numeric data types, or maximum number of characters for string data types.
  • - Scale. Maximum number of digits after the decimal point of numeric values.
struct
  • - Complex data type definition. The schema of the data that you created or imported as complex data type definition in the type definition library.

Changing the Type Configuration for an Array Port

The default type configuration for an array is string[], which is a one-dimensional array of strings. You can change the type configuration for the array port. Specify the data type of the array elements and the array dimension.
    1. In the transformation, select the array port for which you want to specify the type configuration.
    2. In the Type Configuration column, click the Open button.
    The Specify Type Configuration for Array dialog box appears.
    The Type Configuration dialog box for an array port shows properties that you can configure. The properties include Type, Array Dimension, Precision, and Scale. The default Type is string. The default Array Dimension is 1. Based on the default Type and Array Dimension, the default Type Configuration is string[].
    3. In the Type drop-down menu, change the data type of the elements in the array.
    For example, if the elements in the array are of type integer, select integer.
    The type configuration value appears as integer [].
    4. Optionally, in the Array Dimension drop-down menu, select a number to create a nested array.
    For example, array dimension 3 creates a nested array of 3 levels. The type configuration value appears as integer [] [] [].
    5. Optionally, configure the following properties:
    6. Click OK.
    On the Ports tab, the specified type configuration appears in the Type Configuration column of the array port.

Changing the Type Configuration for a Map Port

The default type configuration for a map is <string, string>. You can change the type configuration for a map port. Specify the data type of the key-value pair.
    1. In the transformation, select the map port for which you want to set the type configuration.
    2. In the Type Configuration column, click the Open button.
    The Type Configuration dialog box appears:
    The Type Configuration dialog box for a map port shows the properties for the data type, precision, and scale for the key and value. The default key data type is string, and the default value data type is string.
    3. Specify the properties for the key and value of the map data type.
    1. a. In the Type drop-down menu, select the data type.
    2. b. Optionally, in the Precision box, specify the maximum number of significant digits for numeric data types, or maximum number of characters for string data types.
    3. c. Optionally, in the Scale box, specify the maximum number of digits after the decimal point of numeric values.
    4. Click OK.
    On the Ports tab, the specified type configuration appears in the Type Configuration column of the map port.

Specifying the Type Configuration for a Struct Port

The type configuration for a struct port requires a complex data type definition in the type definition library. Specify the complex data type definition that represents the schema of the struct data.
Before you specify the type configuration for the struct port, create or import a complex data type definition. You can also reference an existing complex data type definition in the type definition library.
    1. In the transformation, select the struct port for which you want to specify the type configuration.
    2. In the Type Configuration column, click the Open button.
    The Type Configuration dialog box appears:
    This screenshot shows the Type Configuration dialog box for a struct port. In the left-hand panel, you can expand your type definition library to view a list of your complex data types definitions. When you select the complex data type definition that you want to use, the right-hand panel shows the schema of the data in the definition.
    3. Select a complex data type definition from the list of definitions in the type definition library for the mapping or the mapplet.
    4. Click OK.
    On the Ports tab, the specified type configuration appears in the Type Configuration column of the struct port.