Java Transformation Overview
Use the Java transformation to extend Developer tool functionality.
The Java transformation provides a simple native programming interface to define transformation functionality with the Java programming language. You can use the Java transformation to define simple or moderately complex transformation functionality without advanced knowledge of the Java programming language or an external Java development environment. The Java transformation is an active or passive transformation.
The Developer tool uses the Java Development Kit (JDK) to compile the Java code and generate byte code for the transformation. The Developer tool stores the byte code in the Model repository.
The Data Integration Service uses the Java Runtime Environment (JRE) to run generated byte code at run time. When the Data Integration Service runs a mapping with a Java transformation, the Data Integration Service uses the JRE to run the byte code and process input rows and generate output rows.
Create Java transformations by writing Java code snippets that define transformation logic. Define transformation behavior for a Java transformation based on the following events:
- •The transformation receives an input row.
- •The transformation processed all input rows.
In mappings that run on the Spark engine, you can use complex data types in Java transformations to process hierarchical data. With complex data types, the Spark engine directly reads, processes, and writes hierarchical data in Avro, Parquet, and JSON complex files.
Reusable and Non-Reusable Java Transformations
You can create a reusable or non-reusable Java transformation.
Reusable transformations can exist in multiple mappings. Non-reusable transformations exist within a single mapping.
The views in the editor where you define properties and create Java code differ based on whether you are creating a reusable or non-reusable Java transformation.
Active and Passive Java Transformations
When you create a Java transformation, you define its type as active or passive.
After you set the transformation type, you cannot change it.
A Java transformation runs the Java code that you define on the On Input tab one time for each row of input data.
A Java transformation handles output rows based on the transformation type as follows:
- •A passive Java transformation generates one output row for each input row in the transformation after processing each input row.
- •An active Java transformation generates multiple output rows for each input row in the transformation.
Use the generateRow method to generate each output row. For example, if the transformation contains two input ports that represent a start date and an end date, you can use the generateRow method to generate an output row for each date between the start date and the end date.
Data Type Conversion
A Java transformation converts Developer tool data types to Java data types, based on the Java transformation port type.
When a Java transformation reads input rows, it converts input port data types to Java data types.
When a Java transformation writes output rows, it converts Java data types to output port data types.
For example, the following processing occurs for an input port with the integer data type in a Java transformation:
- 1. The Java transformation converts the integer data type of the input port to the Java primitive int data type.
- 2. In the transformation, the transformation treats the value of the input port as the Java primitive int data type.
- 3. When the transformation generates the output row, it converts the Java primitive int data type to the integer data type.
The following table shows how the Java transformation maps Developer tool data types to Java primitive and complex data types:
Developer Tool Data Type | Java Data Type |
|---|
array* | java.util.List |
bigint | long |
binary | byte[] |
date/time | With nanoseconds processing enabled, BigDecimal with nanosecond precision With nanoseconds processing disabled, long with millisecond precision (the number of milliseconds since January 1, 1970 00:00:00.000 GMT) |
decimal | With high precision processing disabled, double with precision 15 With high precision processing enabled, BigDecimal |
double | double |
integer | int |
map* | java.util.Map |
string | String |
struct* | Customized JavaBean class with getters and setters for the struct field elements |
text | String |
* Supported only on the Spark engine. |
In Java, the java.util.List, java.util.Map, String, byte[], and BigDecimal data types are complex data types. The double, int, and long data types are primitive data types.
In the Developer tool, array, struct, and map data types are complex data types.
Complex Data Type Conversion on the Spark Engine
You can add complex ports to the Java transformation in mappings that run on the Spark engine. A complex port is a port that is assigned a complex data type. The Java transformation converts complex data types to comparable Java complex data types. It also converts elements of complex data types to comparable Java data types, which are a boxed version of primitive data types.
Array Data Type
When a Java transformation reads input rows, it converts array data type to Java List data type. The transformation converts the data type of array elements to a boxed version of Java data types.
For example, the transformation converts the Developer tool complex data type array<integer> to a Java data type List<Integer>.
When a Java transformation writes output rows, it converts Java List data type to array data type. The transformation converts the data type of List elements to an unboxed version of the Developer tool data types.
Struct Data Type
When a Java transformation reads input rows of struct data type, it generates an equivalent Java bean class. The name of the Java bean class is the same as the name of the struct data type. The transformation converts the data type of struct elements to a boxed version of Java data types.
If the struct data type uses special characters or Java reserved keywords, such as final or private, the Java transformation replaces the special characters with an underscore (_) in the class name. You can open the full code in the Java view of the transformation properties tab to see the generated classes.
The Java transformation generates class member field names prefixed with an underscore (_). It also generates getters and setters for the member fields.
The generated Java bean class has the namespace of the type definition library name as an outer class. The name of the outer class is the same as the name of the type definition library. The Java data type name of the struct port is of the following format:
type_library_name.struct_type_name
For example, the type library name is m_Type_Definition_Library. The complex data type definition for the struct port is:
Customer {
name string
age integer
}
The Java transformation generates the Java bean classes with getters and setters for the member fields. The following code snippet shows the outer class and inner class:
public static final class m_Type_Definition_Library {
public static final class Customer implements Serializable {
private String _name;
private Integer _age;
public Customer() {}
The following code snippet shows the getter and setter for the struct element name of type string:
public String get_name() {
return _name;
}
public void set_name(String _name) {
this._name = _name;
}
When a Java transformation writes output rows of struct data type, it converts Java bean class to struct data type. The transformation converts the member fields of the class to struct elements. The data type of member fields are converted to an unboxed version of the Developer tool data types.
Map Data Type
When a Java transformation reads input rows, it converts map data type to Java Map data type. The transformation converts the data type of map elements to a boxed version of Java data types.
For example, the Developer tool complex data type map<string, bigint> is converted to a Java data type Map<String, long>.
When a Java transformation writes output rows, it converts Java Map data type to map data type. The transformation converts the data type of Map elements to an unboxed version of the Developer tool data types.