A Python transformation has incoming fields and output fields. Use incoming fields and output fields as variables in the Python code snippets that you define on the Python tab.
Add output fields on the Output Fields tab.
Add output fields for the output data that you want to pass to the downstream transformation. To add a field, click Add Field, and then enter the field name, data type, precision, and scale. You can also create output fields on the Outputs tab of the Python editor by clicking Create New Field.
After you add fields to the transformation, you can use the field names as variables in the Python code.
Use the following guidelines when you create output fields:
•A field name can contain only ASCII characters.
•Don't use a Python keyword for a field name. For example, don't use fields names such as import, global, or class.
•Don't use these reserved terms for a field name: resourceJepFile or resourceFilesArray.
• Set user-defined default values in the Python code.
For example, you can enter output_field = 'value' to set the default value 'value' for the output field output_field.
You can configure one or more input fields as partition keys. Data Integration uses the partition keys to repartition the data before the code runs. If you don't add an incoming fields as a partition key, the data is processed using its default partitioning scheme.
Data type conversion
A Python transformation converts transformation data types to Python data types based on the Python transformation field type.
When a Python transformation reads input rows, it converts incoming field data types to Python data types. When a Python transformation writes output rows, it converts Python data types to output field data types.
For example, the following processing occurs for an input field with the double data type:
1The Python transformation converts the double data type in the incoming field to the Python float data type.
2The Python transformation uses the value in the incoming field as the value for the Python float data type.
3When the transformation generates the output row, it converts the Python float data type to the double data type.
The following table shows how the Python transformation maps transformation data types to Python data types:
Transformation data type
Python data type
Integer
Int
Decimal
Float
Double
Float
Timestamp
Datetime
String
Str
text
Text
Data types in input and output fields
In the Python transformation, the data types in corresponding incoming fields and output fields must be the same. If the data types are not the same, convert them in the Python code.
For example, you create an incoming field with the integer data type and an output field with the string data type. You define the Python code to process the data in the incoming field and write the data to the output field. In the Python code, you can use the str() function to convert the integer data type in the incoming field and write the output as a string data type in the output field.
Partition keys
You can use partition keys to define how to group data into partitions before the Python code runs.
You can configure one or more input fields as partition keys. The Python transformation uses the partition keys to repartition the data before the code runs. If you do not add an incoming field as a partition key, the data is processed using its default partitioning scheme.