When you select an intelligent structure to use in a Structure Parser transformation, the intelligent structure model output fields appear on the Output Fields tab of the Properties panel.
In a mapping, the output fields are grouped into the following groups:
•Unidentified group. This group contains the data that was not identified by the intelligent structure. You might want to pass this data to a target file for further analysis.
•Output groups. One or more groups that contain the data that the intelligent structure model identifies.
You can select a group or groups to transfer pass-through fields to. Pass-through fields are fields that you don't map in the transformation field mapping and that the transformation transfers to the selected group or groups as is. Use this option if you want to use the pass-through fields later in the mapping, for example, to pass a timestamp field to the next downstream transformation. The Structure Parser transformation passes the pass-through fields to each selected group.
In a mapping in advanced mode, the output fields are put in one group. This group includes pass-through fields if you configure the transformation to enable pass-through fields.
The Output Fields tab displays the first five fields in each output group, including the name, type, precision, scale, and origin of each field. You can click the link at the bottom of the group to display all the fields in the group. The origin of a field shows the path of the respective node in the intelligent structure model. If a name of a node in the intelligent structure model contains special characters, the Secure Agent replaces them with an underscore (_) character, and the Output Fields tab displays the revised name as the output field name. The Output Fields tab doesn't display the origin of fields with revised names.
To edit the precision of an output field, click the precision value and enter the precision you require.
Note: Intelligent Structure Discovery doesn't enforce precision and scale on decimal fields.
You can't edit the transformation output fields. If you want to exclude output fields from the data flow or rename output fields before you pass them to a downstream transformation or target, configure the field rules for the downstream transformation.
Output groups for relational output
If you configure the transformation to generate relational output, the transformation can contain multiple output groups. Each output group contains one or more fields that you can write to a relational target or pass to a downstream transformation for further processing.
In advanced mode, the Structure Parser transformation can't generate relational output.
Output groups for JSON, JSON lines, and XML output
If you configure the transformation to generate JSON, JSON lines, or XML output, the transformation contains one output group. The output group contains one string field which contains the output data. The default precision is 1,000,000 characters. You can pass the data in the string field to a downstream transformation or to a target of the appropriate type. For example, if you configure the transformation to generate XML output, you can connect the output field to a flat file target to save the output to an XML file.
Output groups for Avro, Parquet, and ORC output
If you configure the transformation to generate Avro, Parquet, or ORC output, the transformation contains one output group. The output group contains one binary field which contains the output data. The default precision is 1,000,000 characters. You can pass the data in the binary field to a downstream transformation or to a target of the appropriate type. For example, if you configure the transformation to generate Parquet output, you can connect the output field to a target such as Amazon S3 or Hadoop Files.
In advanced mode, the Structure Parser transformation can't generate Avro, Parquet, or ORC output.