DAM Hotfolder
Cloud Edition only
This project is designed for Product 360 Cloud Edition customers only. (On premise customers can use this project and modify it to their needs, but it is not under Informatica's support! )
Preface
This BPM ( ActiveVOS ) project enables Product 360 Cloud Edition customer's to ingest Digital Assets via uploading the files into the blob storage. This hotfolder functionality works only for the Asset Provider HMM, which is the default in the Cloud Edition settings.
Table of Contents
Prerequisites
The DAM hotfolder also supports the auto assignment functionality over the message queue communication. Therefor the messaging service of ActiveVOS must be configured. Please reach out to the Installation and Operation guide for the following areas:
Installation of ActiveVOS with ActiveMQ support and configuration of queues in ActiveVOS (Note: ActiveVOS version 9.2.4.6 is a prerequisite for queue based interactions between Product 360 and ActiveVOS)
Installation
Before you can use the DAM hotfolder you have to deploy these resources:
InfaResources.bpr
DAM_Hotfolder.bpr
with the BPM (ActiveVOS) console.
Also it is possible that you deploy the workflows with your BPM designer. The resources of the projects are in the zip files:
InfaResources.zip
DAM_Hotfolder.zip
Configuration
The base configuration of the DAM hotfolder is defined in the BPM URN mappings. Therefore you have to define the following mappings in the BPM console.
These settings will be done from Informatica OPS team when the customer request it.
URN mappings for AWS
|
URN |
URL |
Mand. |
Description |
|
urn:p360.api.username |
RestServiceUser |
Yes |
Name of the Product 360 user in which context the requests are fired |
|
urn:p360.api.password |
******** |
Yes |
Password of the user above |
|
urn:p360.rest.url |
https://p360server.p360.internal:1543 |
Yes |
Url of the Product 360 rest services |
|
urn:p360.aws.s3.bucket.name |
pim-example-bucket |
Yes |
S3 bucket name of the customer's environment |
|
urn:p360.dam.aws.sqs.rest.url |
https://sqs.us-west-2.amazonaws.com/1234/pim-example-qa-sqs_queue |
Yes |
Url to the SQS queue, where new objects will be notified |
|
urn:p360.dam.default.category.id |
00010000000 ........ 000000000000000000000 |
Yes |
Identifier of the category where new assets will be automatically assigned to (must be 120 characters!) |
|
urn:p360.dam.hotfolder.path |
/data/opt/informatica/shares/efs/P360Server/daminbox/ |
Yes |
Path where the files will be copied from the bucket before creating the asset within Product 360 |
|
urn:p360.bpm.rest.url |
https://p360bpm.p360.internal:443/active-bpel/services/REST |
Yes |
Url to the ActiveVOS cluster |
|
urn:p360.dam.s3.error.path |
/dev/P360BPM/hotfolder/Error/ |
Yes |
Path to the folder of the S3 bucket where files will be copied when the creation process run into an error |
|
urn:p360.dam.s3.processed.path |
/dev/P360BPM/hotfolder/Archived/ |
Yes |
Path to the folder of the S3 bucket where files will be copied when the creation process was successful. It is recommended to leave this entry empty or remove the key, because otherwise a lot of space on the blob storage can be allocated, which cab create additional costs) |
|
urn:p360.dam.upload.path |
/data/opt/informatica/shares/efs/P360Server/upload/ |
Yes |
Path to the folder where the assets will be uploaded during the "/manage/file" Rest call |
|
urn:p360.dam.max.threads |
2 |
Yes |
Maximum number of parallel AciveVOS threads while creating new assets. Value has to be between 1 and 10. |
|
urn:p360.dam.filename.pattern |
.+\.(tif|jpg|pdf) |
Yes |
Regular expression for allowed filenames. (.* ==> all files are supported) |
|
urn:p360.dam.assigndoc.mode |
0 |
Yes |
0 = No AssignDocMessage 1 = AssignDocMessage (only for Master Catalog) Note: only together with the corresponding settings in hmm.properties of P360 server, the auto assignment can be correctly triggered by a message whose sending mode is defined with this parameter. |
|
urn:p360.dam.update.old.documents |
0 |
Yes |
Deactivated! This is an experimental setting. It is not supported with version 10.5 |
URN mappings for Azure
|
URN |
URL |
Mand. |
Description |
|
urn:p360.api.username |
RestServiceUser |
Yes |
Name of the Product 360 user in which context the requests are fired |
|
urn:p360.api.password |
******** |
Yes |
Password of the user above |
|
urn:p360.rest.url |
https://p360server.p360.internal:1543 |
Yes |
Url of the Product 360 rest services |
|
urn:p360.dam.default.category.id |
00010000000 ........ 000000000000000000000 |
Yes |
Identifier of the category where new assets will be automatically assigned to (must be 120 characters!) |
|
urn:p360.dam.error.path |
/opt/informatica/shares/S3_CUSTOMER/dev/P360Server/assethotfolder/Error/ |
Yes |
Path to the folder of the blob storage where files will be copied when the creation process run into an error |
|
urn:p360.dam.processed.path |
/opt/informatica/shares/S3_CUSTOMER/dev/P360Server/assethotfolder/Archived/ |
Yes |
Path to the folder of the blob storage where files will be copied when the creation process was successful. It is recommended to leave this entry empty or remove the key, because otherwise a lot of space on the blob storage can be allocated, which cab create additional costs) |
|
urn:p360.dam.upload.path |
/data/opt/informatica/shares/efs/P360Server/upload/ |
Yes |
Path to the folder where the assets will be uploaded during the "/manage/file" Rest call |
|
urn:p360.dam.max.threads |
2 |
Yes |
Maximum number of parallel AciveVOS threads while creating new assets. Value has to be between 1 and 10. |
|
urn:p360.dam.filename.pattern |
.+\.(tif|jpg|pdf) |
Yes |
Regular expression for allowed filenames. (.* ==> all files are supported) |
|
urn:p360.dam.assigndoc.mode |
0 |
Yes |
0 = No AssignDocMessage 1 = AssignDocMessage (only for Master Catalog) Note: only together with the corresponding settings in hmm.properties of P360 server, the auto assignment can be correctly triggered by a message whose sending mode is defined with this parameter. |
|
urn:p360.dam.update.old.documents |
0 |
Yes |
Deactivated! This is an experimental setting. It is not supported with version 10.5 |
Context properties for the messaging service
These 3 required context properties in the messaging service ActiveMQ have to be created. You can navigate to the service in the ActiveVOS console.
|
Name |
Value |
|
queue.JNDI_P360_DAM_ASSIGNMENT |
heiler.hmm.backend.event.assignment |
|
queue.JNDI_P360_DAM |
heiler.hmm.backend.event |
|
queue.JNDI_P360_DAM_MODIFIED |
heiler.hmm.backend.event.assetModified |
Schedule the maintenance job
The DAM hotfolder is implemented as an endless loop. The workflow has to be started one time via creating a scheduled maintenance job via the ActiveVOS console.
Job settings for AWS
|
Key |
Value |
|
Name |
DAM Hotfolder |
|
Frequency |
Once |
|
Start Date on Server |
Do not modify |
|
Start Time on Server |
Do not modify |
|
Run Options |
Skip if running |
|
Service Name |
Hotfolder_aws |
|
Input Document |
<hot:startDAMHotfolderRequest xmlns:hot="http://p360.dam/activevos/process/Hotfolder" xmlns:soap="http://www.w3.org/2003/05/soap-envelope"> |
Job settings for Azure
|
Key |
Value |
|
Name |
DAM Hotfolder |
|
Frequency |
Once |
|
Start Date on Server |
Do not modify |
|
Start Time on Server |
Do not modify |
|
Run Options |
Skip if running |
|
Service Name |
Hotfolder_azure |
|
Input Document |
<hot:startDAMHotfolderRequest xmlns:hot="http://p360.dam/activevos/process/Hotfolder" xmlns:soap="http://www.w3.org/2003/05/soap-envelope"> |
Start the DAM hotfolder
The DAM hotfolder can be started in the ActiveVOS console.
Now the process should run. You can check this as well in the ActiveVOS console.
Throughput
On AWS a throughput of about 40.000 files per 24h is observed. This has been measured with files of 3.3 MB average file size and ActiveVOS is running on a m5.large instance.
The throughput depends on the load of ActiveVOS. If other workflows requires resources, the file throughput will be lower.
Limitations
Upload of huge files can lead to an internal timeout while processing.
10 GB files are processed successfully.
Verification
To verify whether the DAM hotfolder is working as expected you have to upload files into your blob storage.
AWS
Upload a file into your blob storage hotfolder. I.e. /qa/P360Server/assethotfolder/. (You can do this for example with the S3 Browser)
Check the processes in you ActiveVOS console
You can also check whether you see the new DAM asset in your Product 360 clients
Azure
Upload a file into your blob storage hotfolder. I.e. /qa/P360Server/assethotfolder/ (You can do this for example with the Microsoft Azure Storage Explorer)
Check the processes in you ActiveVOS console
You can also check whether you see the new DAM asset in your Product 360 clients