Audit Trail Configuration
This page describes the configuration of the Audit Trail in Product 360
Prerequisite
Before you can start with this chapter, you need to have finished the following parts:
Configure Audit Trail in the Product 360 Application
The Audit Trail feature is delivered as part of the Server and Desktop Client package. By default the Audit Trail functionality is disabled. To enable this feature you have to configure the following properties in <PIM_SERVER_INSTALLATION_ROOT>\configuration\HPM\server.properties file:
Enable Audit Trail
General settings |
Mandatory |
|||||||||||||
system.name |
|
This is a mandatory property. It specifies the name of the system, e.g. Test System /Productive System / Demo / Poad, etc.
|
||||||||||||
audittrail.rest.url |
|
URL path to the Elasticsearch REST server. E.g. audittrail.rest.url = http://localhost:9200,http://localhost:9201 |
||||||||||||
audittrail.rest.user |
|
Login name of the Elasticsearch REST server. |
||||||||||||
audittrail.rest.password |
|
Login password of the Elasticsearch REST server. |
||||||||||||
audittrail.rest.allow.self-signed.certificate |
|
Allows self-signed certificate only if you use https. |
||||||||||||
audittrail.mode |
|
The Audit Trail can be set up with the below modes
|
||||||||||||
audittrail.installation.type |
|
The Audit Trail can have below installation types
|
||||||||||||
audittrail.threadpool.size |
|
This property should have a value that is the same as db.default.pool.maxPoolSize |
||||||||||||
audittrail.backup.restoration.mode |
|
This setting will synchronize the Product 360 records in the relational database and their corresponding Audit Trail data in Elasticsearch. Default: false NOTE: Set to "true" only when Product 360 - Server starts after recovering from a disaster. |
For AWS managed Elasticsearch
policy_id in template is deprecated in the AWS Elasticsearch update of April 2021 which is used by Product 360 for Index Lifecycle Management. To make the application compatiable, the below flag is introduced in the plugin_cusotmization.ini file
com.heiler.ppm.audittrail.server/auditTrail.aws.policyIdDeprecated=true
This should only be made false in case April 2021 update is not applied to AWS Elasticsearch.
Note : This flag is introduced from Product 360 versions 10.1.0.00.12, 10.1.0.01.05, 10.1.0.02.01
Configure Audit Trail view
To limit the amount of data displayed, there are the following properties in <PIM_SERVER_INSTALLATION_ROOT>\configuration\HPM\plugin_customization.ini file.
Property |
Value |
com.heiler.ppm.audittrail.server/historyDetailTab.pageSize |
The count of documents to be displayed initially in the history detail tab for single/multi selected objects or audit trail search results, default is 5 |
com.heiler.ppm.audittrail.server/historyDetailTab.content.maxChildren |
The maximum number of child nodes within a parent node in the history detail tab, default is 70. |
Start Product 360 Server
In case of successful Audit Trail setup you should see the following log messages for each root entity which is configured for Audit Trail:
Lifecycle for entity '<entity identifier>' is running in Elasticsearch
Repository Based Configuration
Short Identifier
In order to generate and store the changes in the entity item change document, it is important to provide the Short Identifier for entities and fields.
Audit Trail Settings
Audit Trail can be activated for any root entity in the repository. An activation/deactivation on the sub-entity level is not supported. The setting of the root entity will always be used, independent of the setting for the sub-entities.
Open the repository editor,
Go to Custom section and select a root entity
If "Audit trail settings" is not already available for that entity then right-click → select "New child" → click "Audit trail settings"
Audit trail settings have the below properties
Property
Value
Datasource
Datasource of Entity, which can be any of the following - MAIN, MASTER, or SUPPLIER
Documentation
Retention Policy
There are 3 types of retentions can be chosen from the drop-down
Retention type
NO_RETENTION
If no auditing is needed for the entity. No change information will be recorded in the Elasticsearch for this entity
LONG_RETENTION
If auditing is needed to be stored for the entity for a considerable amount of time, e.g. a few years.
The longterm_policy and longterm_template from the audittrail configuration folder will be used to create policy and index
SHORT_RETENTION
If auditing is needed to be stored for the entity for a relatively short amount of time, e.g. a few months.
The shortterm_policy and shortterm_template from the audittrail configuration folder will be used to create policy and index
Revision setting
Below revision settings are available to choose from for auditing
Revision
ALL_REVISIONS
All revisions changes will be recorded in the Audit Trail
HEAD_REVISION_ONLY
Only head revisions changes will be recorded in the Audit Trail
NON_HEAD_REVISION_ONLY
All the revisions changes except head revision will be recorded in the Audit Trail
Use Own Index
Below values can be set for this property
Own index
true
This will create a separate index in Elasticsearch for this particular entity
false
This will use a shared index in Elasticsearch for multiple entities.
Configuring Audit Trail for entities is a manual process.
Customers migrating to this version must manually configure the "Audit trail settings" for each entity.
Supports Audit Trail Property
If this flag is set to false for a field/entity then any changes done on the field/entity are not recorded in the Audit Trail. However a corresponding change document is stored recording the transaction but with empty change summary.
Entity level : If set at an entity level then all it's sub-entities and fields also inherit this behavior.
Field level : If set at a field level then only that field is affected. Any change done with respect to this field is not recorded in change summary of the document.
Index policies and templates
Elasticsearch provides OOB functionality to handle index lifecycle management which helps indices to automatically rollover in different phases after defined durations. This lifecycle management makes it easier to manage indices in hot-warm-cold architectures.
You can read more about lifecycle management -
Audit Trail policies and templates are available at the following location in <PIM_SERVER_INSTALLATION_ROOT>\configuration\HPM\audittrail.
There are folders available for each audittrail.installation.type configured in server.properties. E.g. Folder elastic-standalone will have all the configurations for installation type elastic-standalone.
Based on the repository configuration for each entity, policy and template will be applied to particular indices.
Lifecycle Policies
There are shortterm_policy and longterm_policy in Product 360.
Each has a different configuration for the number of days an index will be in one phase. These can be configured based on customer needs and Product 360 will take care of applying these for indices in Elasticsearch after Product 360 server restart.
Each phase is explained below -
Phases |
|
Hot |
This phase is used for both creation and search of data by the Audit Trail. The index will be in the hot phase once created and stay in this phase until rollover. Only hot indices are writable indices. When indices rollover, there is a potential chance that indices might be in an inconsistent state. To make sure indices never go to an inconsistent state, Product 360 has a job running to rollover indices from the hot phase to the warm phase. Other phases are taken care of by Elasticsearch lifecycle management. This rollover job is deprecated from Product 360 versions 10.1.0.00.08, 10.1.0.01.03, 10.1.0.02.00 Rollover of indices from the hot phase to warm phase using the below settings -
These rollover settings are deprecated from Product 360 versions 10.1.0.00.08, 10.1.0.01.03, 10.1.0.02.00 Policy rollover time must be more than double of rollover time given in plugin_customization.ini so that Product 360 has sufficient time to attempt a proper rollover of the index. If Product 360 is unable to perform the rollover, then the default rollover defined in the Elasticsearch policy will execute. |
Warm |
This phase is used only for search by the Audit Trail. The index will be in a warm phase once the rollover is complete. We can define actions to be taken in the warm phase for changing shards and replicas and also to shrink the index. |
Delete |
Delete phase deletes the index once it reaches the time defined in min_age. Customers should set the min_age of this phase based on how long they want to retain Audit Trail data of a particular entity. |
For on-prem Elasticsearch installation, in every phase, the min_age means the index will wait for that amount of time in the previous phase before entering the current phase.
For example, if the min_age in the delete phase has 10d then the index will be in the warm phase for at least 10 days and then it will be deleted.
Read more at https://www.elastic.co/guide/en/elasticsearch/reference/6.8/_timing.html
Index Templates
There are 3 index templates in the Audit Trail configuration folder. Each has index settings JSON which contains configurations like number_of_shards, number_of_replicas, etc.
template |
|
longterm_template |
This defines the index settings for the LONG_RETENTION entity based on the repository's "Audit trail settings". |
shortterm_template |
This defines the index settings for the SHORT_RETENTION entity based on the repository's "Audit trail settings". |
migration_template |
If the customer migrates data from the old Audit Trail to the new Audit Trail, a new index will be created using this template configuration. |
Configure Audit Trail logs in Product 360
Logs for different components of the Audit Trail can be enabled in log4j2.xml located at <PIM_SERVER_INSTALLATION_ROOT>\configuration\HPM\log4j2.xml
All components can be configured to have different logging levels based on need. Below are the components.
Audit Trail modules |
|
AUDIT_TRAIL |
Any logging level set to this component will apply to all the other components of the Audit Trail. By default, this is set to INFO. |
AUDIT_TRAIL.LIFECYCLE |
This component will show logs of all the activities related to the lifecycle like index creation, index rollover, etc. |
AUDIT_TRAIL.WATCHDOG |
This component logs activity related to internal Audit Trail jobs which synchronizes data between Elasticsearch and RDBMS. |
AUDIT_TRAIL.PERSISTENCE |
This component logs activity related to the persistence of data in the Audit Trail |
AUDIT_TRAIL.SEARCH |
This component logs activity related to the search of data from the Audit Trail |
<
Logger
name
=
"AUDIT_TRAIL.LIFECYCLE"
level
=
"DEBUG"
additivity
=
"false"
>
<
AppenderRef
ref
=
"AuditTrailAppender"
/>
<
AppenderRef
ref
=
"StdFileAppender"
/>
</
Logger
>