Creating your own import functions

It is possible to contribute user defined functions using the extension point com.heiler.ppm.function.core.functions. The contributed functions can be used within mapping formulas in the same way as the build-in functions.

A function can have zero, one or more parameters and returns a single value. Parameters can be mandatory or optional, optional parameters have to be the last parameters. The parameters as well as the returned value are typed whereby any Java class can be used as type.

A function might throw an exception or log one or more messages into the import protocol (Logging into import protocol is available since HPM 6.0.) If the function throws an exception, a log entry containing the exception is log into the import protocol. The ID of the entity item as well as location information of the function arguments are added automatically to the log entry.

A function consists of two parts: The contribution to the extension point com.heiler.ppm.function.core.functions and a Java class performing the actual work.

The contribution

Each contribution contains the following information:

the name of the function (language independent)
a description (language dependent)
the parameters
the return type
the context in which the function should be active.
the executor class which performs the actual work. The executor has to extend the abstract class com.heiler.ppm.function.core.AbstractFunctionExecutor.

Each parameter entry contains the following information:

the id (language independent), used to access the parameter in the executor
the name (language independent), only used when printing the list of available functions in the formula viewer. Should be set to the same value as id to avoid confusion.
the position of the parameter
whether the parameter is mandatory or optional
the type

The extension point com.heiler.ppm.function.core.functions is used for several purposes within the HPM (for example, also for validation functions in the export module). Therefore, many available fields do not have to be set if the function is only used in the import module. Only the fields mentioned in the example are considered by the import module.

The parameter types and the return type has to be specified in the field value-class / fix-class and not in the field class.

The executor

The executor is a class extending extending the abstract class com.heiler.ppm.function.core.AbstractFunctionExecutor. The executor gets a arguments an execution context and the the parameters.

The context can be used to write an entry into the import protocol (using the method logStatus, available since HPM 6 ). Note that the method isLogEnabled is only relevant for the export module and returns always false when used within the context of the import module.

The parameters can be retrieved using the id specified in the parameter definition. The return value is either

an object of the specified class
null if the referred cell in the import file is empty (Excel, CSV) or the XML tag resp. attribute does not exist
an object of the class com.heiler.ppm.mapper.core.functions.DeleteContentObject if the referred cell in the import file is empty and the option Empty cells delete field content is active is activated (Excel, CSV) or the the XML-tag does not contain any text.

If the value provided in the import file cannot be converted into the specified class, an error is written into the import protocol and the function is not called at all.

The class com.heiler.ppm.function.core.FunctionUtils contains some utility methods that are useful for function executor, like checking if the value of a parameter is not null null or is within a certain range.

Example: addDays

Assume that we need a function that adds to a specified date (argument 1) a number of specified days (argument 2). An example how the function might be used is:

  addDays( [Date], 7)

whereby Date is a column in a CSV sheet containing dates. To show how logging works, each time the function is called, an entry is added to the import protocol.

The contribution for the extension point looks like:

Extension for addDays

<extension 
    point="com.heiler.ppm.function.core.functions">
  <function 
      name="addDays"
      description="Adds a number of days to the specified date" >
 
    <!-- Return type of the function -->
    <value-class 
        fix-class="java.sql.Date">
    </value-class>
 
    <!-- Executor that does the actual work (written in Java) -->
    <executor
        class="AddDaysExecutor">
    </executor>
 
    <!-- First parameter with id and name "date"  -->
    <parameter
        id="date"  
        name="date"
        mandatory="true"
        order="1">
      <!-- Type of the first parameter -->
      <value-class
          fix-class="java.sql.Date">
      </value-class>
    </parameter>
 
    <!-- Second parameter with id and name "days"  -->
    <parameter
        id="days"
        name="days"
        mandatory="true"
        order="2" >
      <!-- Type of the second parameter -->
      <value-class
          fix-class="java.lang.Integer">
      </value-class>
    </parameter>
 
    <!-- This function should be available only in the import module -->
    <context
        context="IMPORT">
    </context>
  </function>
</extension>

An implementation of the executor for the function addDays might look like:

Executor for function addDays

import java.sql.Date;
import com.heiler.ppm.function.core.AbstractFunctionExecutor;
import com.heiler.ppm.function.core.ExecutionContext;
import com.heiler.ppm.function.core.ExecutionParameter;
import com.heiler.ppm.function.core.FunctionException;
import com.heiler.ppm.function.core.FunctionUtils;
 
class AddDays extends AbstractFunctionExecutor
{
    @Override
    public Object doExecute( ExecutionContext context, ExecutionParameter parameters ) throws FunctionException
    {
      Date date = ( Date ) parameters.getValue( "date" );
      FunctionUtils.assertNotNull( date, "date" );
 
      Integer days = ( Integer ) parameters.getValue( "days" );
      FunctionUtils.assertNotNull( days, "days" );
 
      return new Date( date.getTime() + days * 24 * 60 * 60 * 1000 );
    }
}

Activate custom entities

The following steps are necessary to activate a repository entity for import:

In the repository, the property purpose or importPurpose has to be set to 1 for this entity and for each sub entity, each logical key and each field that should be importable. For historical reasons, logical keys allow only the property purpose is. Note that an entity or a field is importable if either purpose or importPurpose is set to 1.
A contribution to the extension point com.heiler.ppm.importer5.core.entityImportConfiguration has to be provided.
Optionally, a contribution to the extension point com.heiler.ppm.importer5.ui.importDataContribution can be provided.

Contributing to the extension point com.heiler.ppm.importer5.core.entitySpecificImportConfiguration

This extension point specifies:

a name (only used the debugging resp. documentation purposes)
the field that contains the external identifier of the item
a class that implements the interface EntitySpecificImportConfiguration. This class handles import specific operations like loading and storing entity items.
the position which the entity should be shown at in the import GUI. This field allows a float as value, therefore it is possible to insert the entity between two existing entities.
whether the entity supports multi-threaded import or not. For example, structure groups do not support multi-threaded import.
optionally a class extending the abstract class com.heiler.ppm.importer5.core.api.entityspecific.EntitySpecificImportData and stores additional information for one import like the catalog or the structure systems the entity items should be imported in.
optionally a priority. If a standard contribution should be overwritten, the priority has to be set to 1.

Example 1: New entity CustomEntity based on entity type EGDType

Assume a new entity named CustomEntity which bases on the entity type EGDType should be made importable.

The contribution to the extension point com.heiler.ppm.importer5.core.entitySpecificImportConfiguration would look like

Contribution example

<extension
    point="com.heiler.ppm.importer5.core.entitySpecificImportConfiguration">
  <entitySpecificImportConfiguration
      name="CustomEntityImportConfiguration"
      uniqueKey="/CustomEntity/CustomEntity.Identifier"
      class="com.heiler.ppm.article.dataimport.core.EGDImportConfiguration"
      order="1.5"/>
</extension>

In versions before 6.0.02.00, the class com.heiler.ppm.article.dataimport.core.EGDImportConfiguration cannot be used as it contains a hard coded reference to the field Product.Identifier. You have to make a copy of the class EGDImportConfiguration, specify this class in the extension and replace the field identifier.

Example 2: Change default behavior when importing items

Assume you want to perform an action after an item is saved by the import. This behavior can be achieved by replacing the default ArticleImportConfigurationby an own implementation CustomArticleImportConfiguration.

The contribution to the extension point com.heiler.ppm.importer5.core.entitySpecificImportConfiguration could look like

 <extension
     point="com.heiler.ppm.importer5.core.entitySpecificImportConfiguration">
   <!-- priority and 
   <entitySpecificImportConfiguration
       name="ArticleImportConfiguration"
       class="com.heiler.ppm.custom.core.CustomArticleImportConfiguration"
       importData="com.heiler.ppm.article.dataimport.core.ArticleImportData"
       order="1"
       supportsMultithreadedImport="true"
       uniqueKey="Article/Article.SupplierAID"
       priority="1" />
   </extension>

The customized class could look like:

CustomArticleImportConfiguration

package com.heiler.ppm.custom.core;
import org.eclipse.core.runtime.CoreException;
import com.heiler.ppm.importer5.core.api.entityspecific.DetailModelWrapper;
import com.heiler.ppm.std.core.entity.EntityProxy;
 
public class CustomArticleImportConfiguration extends ArticleImportConfiguration
{
  public CustomArticleImportConfiguration()
  {
    super();
  }
  @Override
  public EntityProxy saveDetailModelWrapper( DetailModelWrapper detailModelWrapper ) throws CoreException
  {
    EntityProxy entityProxy = super.saveDetailModelWrapper( detailModelWrapper );
    // do some post processing
    return entityProxy;
  }
}

Executing or scheduling an import programmatically

The central interface for executing or scheduling an import is com.heiler.ppm.importer5.core.api.Importer. An object of this interface can be retrieved by using the factory com.heiler.ppm.importer5.core.api.ImporterFactory.

An example for executing an import immediately using a import profile stored in a file:

Executing an import

Importer importer = ImporterFactory.getInstance()
                                   .createImporter();
 
// Load the import profile from a file (Other methods exists to load the import profile from the server or a string)
ImportProfile profile = importer.loadImportProfile( new File( "myImportProfile.him" ) );
 
// Set the import file which should be imported
profile.getDataSources()[0]
       .setFile( new File("MyFileToBeImported.csv") );
 
// Perform the import. The method blocks until the import has been finished.
ImportResult result = importer.performImport( profile, null, null );
 
// Retrieve some information about the performed import
int importedItemsWithErrors = result.getImportCounter( )
                                    .getImported(ReturnMode.WITH_ERRORS);
 
// Clear the problem log
result.getProblemLog()
      .clear( new NullProgressMonitor() );

An example for scheduling an import in an hour using an import profile stored in the database (method loadImportProfileFromDatabase is provided below).

Scheduling an import profile

Importer importer = ImporterFactory.getInstance()
                                   .createImporter();
 
//Load the import profile from the database 
ImportProfile profile = loadImportProfileFromDatabase( "MyMapping" );
 
//Set the import file
profile.getDataSources()[ 0 ].setFile( new File( "MyFileToBeImported.csv" ) );
 
//Set execution date (in one hour)
Date date = new Date( new Date().getTime() + 60 * 60 * 1000 );
 
//Schedule the import. The method returns immediately after the import has been scheduled.
ImportJobToken jobToken = importer.scheduleImport( profile, date, null, null );
 
// Wait until the import has been finished 
if ( jobToken.waitUntilFinished() )
{
    //Retrieve some information about the import (in case the import has not been canceled)
    ImportResult result = jobToken.getImportResult();
    int importedItemsWithErrors = result.getImportCounter()
                                        .getImported( ReturnMode.WITH_ERRORS );
}
    
//Clear the problem log
jobToken.getProblemLog()
        .clear( new NullProgressMonitor() );

An example for a method loading an import profile from the database. In case the import profile could not be found, a runtime exception is thrown.

Load an import profile from the database

private ImportProfile loadImportProfileFromDatabase( final String mappingName ) throws CoreException
{
    Importer importer = ImporterFactory.getInstance()
                                       .createImporter();
 
    // Retrieve all import profiles with the given mapping name
    List< ImportProfileProxy > profileProxies = importer.getAllImportProfiles( new Predicate< ImportProfileProxy >()
    {
        @Override
        public boolean evaluate( ImportProfileProxy profileProxy )
        {
            return mappingName.equals( profileProxy.getName() );
        }
    } );
 
    // Throw a runtime exception if import profile could not be found
    if ( profileProxies.size() == 0 )
    {
      throw new RuntimeException( "Mapping '" + mappingName + "' not found" );
    }
 
    // Returns the found import profile (import profile names are unique )
    return  profileProxies.get( 0 )
                          .getImportProfile();
}