Secure Agent Services > Mass Ingestion (Files)
  

Mass Ingestion (Files)

To change or optimize the behavior of Mass Ingestion Files that your Secure Agent group uses, configure Mass Ingestion properties for your runtime environment in Administrator.
You can configure the following properties:
Type
Name
Description
AGENT_RUNTIME_SETTINGS
file-listener-snapshot-dir
A directory where the snapshots of a new file listener components are added. You can add the following directory paths:
  • - A path relative to the MassIngestionRuntime directory. For example, ../data/monitor.
  • - The absolute path. For example,
    • <Secure agent installation directory>/apps/MassIngestionRuntime/data/monitor
    where Secure agent installation directory is the name of the directory where the secure agent is installed.
Note: Use the snapshot directory shared with all agents when multiple Secure Agents are present in a group.
AGENT_RUNTIME_SETTINGS
mi-task-workspace-dir
A directory in the agent that file ingestion tasks use as an intermediate staging area when transferring files to a target. The directory is a custom location in the agent. The path can be a shared location, mounted location, or a location apart from the default location in the agent.
AGENT_RUNTIME_SETTINGS
mi-task-project-dir
A directory where the file ingestion task stores the project files. The directory is a custom location in the agent. The path can be a shared location, mounted location, or a location apart from the default location in the agent.
AGENT_RUNTIME_SETTINGS
mi-task-logs-dir
A directory where the file ingestion task stores the task logs files. The directory is a custom location in the agent. The path can be a shared location, mounted location, or a location apart from the default location in the agent.
AGENT_RUNTIME_SETTINGS
mi-task-quarantine-dir
A directory where the file ingestion task stores the infected files detected when you run a virus scan. The directory is a custom location in the agent. The path can be a shared location, mounted location, or a location apart from the default location in the agent.
For example, userdata\quarantine
Note: To automatically clean up the quarantine directory, set the agent property for the quarantine location to a system temporary files location such as /tmp/informatica/fmi/quarantine.
AGENT_RUNTIME_SETTINGS
agent-dedup-repository
The information about skipped duplicate files is saved in Informatica Intelligent Cloud Services (IICS). To save the skipped duplicate files information in the Secure Agent, set the property to true.
Default is false.
For more information about saving the skipped duplicate information, see the Mass Ingestion Files guide.
AGENT_RUNTIME_SETTINGS
mi-dedup-snapshot-dir
Enter the path to store the information about skipped duplicate files in the Secure Agent.
Applies only when the agent-dedup-repository property is set to true.
AGENT_RUNTIME_SETTINGS
file-listener-max-pool-size
The maximum number of threads to execute the file listener.
Default is 20.
AGENT_RUNTIME_SETTINGS
file-listener-core-pool-size
The total number of threads.
Default is 20.
AGENT_RUNTIME_SETTINGS
fmi-task-max-pool-size
The maximum number of threads to execute the file ingestion task.
Default is 50.
AGENT_RUNTIME_SETTINGS
fmi-task-core-pool-size
The initial or minimum number of threads.
Default is 20.
AGENT_RUNTIME_SETTINGS
ftp-receive-socket-buffer-size
The buffer size for FTP inbound packets.
Default is 16 bytes.
AGENT_RUNTIME_SETTINGS
ftp-send-socket-buffer-size
The buffer size for FTP outbound packets.
Default is 16 bytes.
AGENT_RUNTIME_SETTINGS
http-client-timeout
The timeout duration in seconds for Agent requests to Informatica Intelligent Cloud Services.
Default is 30 seconds.
PGP_SETTINGS
public-keyring-path
The directory to store the public key ring. You can add the following directory paths:
  • - A path relative to the directory where mass ingestion is installed. For example,
    • ../data/pubring.pkr
    where pubring.pkr is the name of the file where you store the public key ring.
  • - The absolute path. For example,
    • <Secure agent installation directory>/apps/MassIngestionRuntime/data/pubring.pkr
    where pubring.pkr is the name of the file where you store the public key ring and Secure agent installation directory is the name of the directory where the agent is installed.
PGP_SETTINGS
secret-keyring-path
The directory to store the secret key ring. You can add the following directory paths:
  • - A path relative to the directory where mass ingestion is installed. For example,
    • ../data/secring.pkr
    where secring.pkr is the name of the file where you store the secret key ring.
  • - The absolute path. For example,
    • <Secure agent installation directory>/apps/MassIngestionRuntime/data/secring.pkr
    where secring.pkr is the name of the file where you store the secret key ring and Secure Agent installation directory is the name of the directory where the agent is installed.
JVM_SETTINGS
app-heap-size
The minimum and maximum heap sizes of the Mass Ingestion Files application.
Default is -Xms256m -Xmx2048m.
JVM_SETTINGS
lcm-heap-size
The minimum and maximum heap sizes of life-cycle management scripts.
Default is -Xms32m -Xmx128m.
You can configure the following properties in the Custom Configuration Details area when you edit a Secure Agent:
Type
Name
Description
AGENT_RUNTIME_SETTINGS
ComplexFileDisableWriteChecksum
Set the value to True to ignore the crc file. The job runs successfully with Hadoop Files V2 as source and Snowflake Cloud Data Warehouse V2 as the target.
Guidelines to specify the folder path
A folder path can be a shared location, mounted location, or a location apart from the default location in the Secure Agent.
The following table lists the use of slashes around the source folder path:
Source
Folder Path
Windows
<folder path>
For example, C:\temp
Linux
/<folder path>/
For example, /root/path
Windows shared location
<folder path> with additional slashes (\)
For example, the path \\INV12B2B01\Shared\path, is specified as \\\\INV12B2B01\\Shared\\path