Monitoring Data Ingestion and Replication Jobs > Monitoring Data Ingestion and Replication jobs > Viewing details for an ingestion and replication job
  

Viewing details for an ingestion and replication job

You can drill down on a specific Data Ingestion and Replication job from any of the following locations:
To view job details, click the job name in the jobs list. A page for the job appears. The details vary by type of ingestion job.

Application ingestion and replication job details

You can drill down on a job to see job details from the My Jobs page accessed from the Data Integration Home page, from the All Jobs and Running Jobs pages in Monitor, and from the Overview and All Jobs tabs on the Data Ingestion and Replication page in Operational Insights.
After you click a job name, job information is displayed in the Task Summary, Overview, Object Detail, and Alerts panes. The Alerts pane is displayed only for incremental load and combined initial and incremental load jobs.
The following image displays the panes:
Note: Clicking the expander arrow next to the pane name expands or collapses the Task Summary, Object Detail, and Alerts panes.

Task Summary

View a diagram that shows the source and target connection types and connection names. It also shows the calculated data throughput, in rows per second if the job has successfully replicated data to the target, regardless of the job's current status. If the calculated value is 0, indicating no data has flowed to the target, the throughput is not displayed.
In the upper right corner, the Status field displays the job status, which can be one of the following values:
Also, for incremental load jobs and combined initial and incremental load jobs, you can download the job execution log for the entire job run. In the Select log list under the diagram, select one of the following log types:
Note: For initial load jobs, you can get the job log for a specific source object from the Object Detail tab.

Overview

View information about the job, including the runtime environment, task name, load type, task type, task definition location, user who started the job, records read, records written, number of subtasks, and schedule name, if a schedule was used.
The circle image displays the number of subtasks on source tables by status. The color of the circle's rim corresponds to the status.
The following table describes the summary job properties and statistics:
Property
Description
Duration (on left)
The amount of time, in the hh:mm:ss format, that the job ran before it ended.
Start Time (on left)
The date and time when the job was deployed.
End Time (on left)
The date and time when the job ended because it completed processing, was stopped, or failed. This field is not displayed for running jobs
Runtime Environment
The name of the runtime environment in which the job runs.
Agent Name
For incremental load and combined initial and incremental load jobs, the name of the Secure Agent on which the job runs.
If a failover of a Secure Agent to another Secure Agent occurs, Application Ingestion and Replication displays the latest Secure Agent assigned to the job.
Note: For initial load jobs, the Agent Name is displayed in the Object Detail pane.
Task Name
The name of the associated application ingestion and replication task. You can click the task-name link to view or edit task details in Data Ingestion and Replication, if necessary. If you edit the task, you must redeploy it for the updated task definition to be used for a job.
Load Type
The type of load operation that the job performs. Options are:
  • - Initial Load. Loads a snapshot of source data read at a specific point-in-time to a target.
  • - Incremental Load. Loads incremental data changes to a target on a continuous basis, until the job is stopped or ends.
  • - Initial and Incremental Load. Performs an initial load and then automatically switches to an incremental load.
Task Type
The type of task, which is Application Ingestion and ReplicationTask.
Task Location
The project or project folder that contains the application ingestion and replication task definition.
Started By
The name of the user who started the job.
Records Read
The number of records that were read from the source.
Note: For a combined initial and incremental load job, the Records Read count might be greater than the total number of object-level DML change records written. This behavior occurs because the initial load or resync processing always starts after change data capture has begun. As a result, some change records are included in the Records Read count and then discarded before initial load or resync processing starts. These discarded records cause the Records Write count to be less than the Records Read count.
Records Written
The number of records that were successfully replicated to the target.
Note: The Records Written value might be different from the Records Read value if source records are discarded. For example, in a combined initial and incremental load job, change records read from the source before the initial unload phase completes are discarded because they're not yet needed.
Subtasks
The number of subtasks that the application ingestion and replication job used to replicate data from source tables to the target. When a job runs, it uses a separate subtask to process each source table.
Schedule
For initial load jobs, the name of the schedule that is used to run the job or "No schedule" if you run the job manually.

Object Detail

On the Object Detail pane lists subtasks on the source tables. You can view statistics and status information by source table from the last run of a application ingestion and replication job. When you click the expander arrow next to an object name, counts of processed inserts, updates, deletes, and LOB changes are shown for the table.
The following image shows a sample Object Detail pane for a combined initial and incremental load job:
Object Detail pane
Note: This pane shows information for the latest job run. This tab is blank for jobs that have not run or are resuming.
The following table describes the Object Detail fields that are displayed for each table, depending on the load type and status:
Column
Description
Object
The name of the source table or view for which data was propagated to the target.
For an incremental load job or a combined initial and incremental load job, click the arrow icon to the left of the object name to display detailed counts of Inserts, Deletes, Updates, LOBs, and DDL statements processed. For a combined initial and incremental load job, the Unload Count field is also displayed to show the number of records that the initial load portion of processing read from the source. The following usage notes apply to the detailed CDC counts:
  • - The counts are only for the current job run. If you stop and restart the job, the counts start over from zero. Do not use these counts to identify the number of rows written to the target.
  • - The counts are based on rows read from the source and do not reflect the records written to the target. Target write operations might be optimized by combining operations and reducing the number of physical writes. In this case, the counts might not match the number of write operations.
  • - The value N/A means that the count value is not applicable for the count type or the value has not yet been calculated.
  • - The Unload Count might not reflect the number of source records at the time the job is started or resynchronized because of a delay in the start of unload processing. Between the time of the unload request and start of unload processing, rows might be added to or deleted from the source table.
Target Object
The name of the target object that is mapped to the source object.
Agent Name
For initial load jobs, the name of the Secure Agent on which the job ran.
For incremental load and combined initial and incremental load jobs, see the Overview section of the Task Summary pane for the job-level agent name.
Unload Agent Name
For combined initial and incremenal load jobs, the name of the Secure Agent on which the initial load phase of the job ran.
Records Read
For an initial load job, the number of records that were read from the source. For other load types, this information is available only at the job-level on the Job Overview tab.
Records Written
For an initial load job, the number of records that were successfully written to the target. For other load types, this information is available only at the job-level on the Job Overview tab.
Note: The Records Read value can be greater than the Records Written value if some records read from the source were discarded. For example, in a combined initial and incremental job, any source change records read before the initial unload phase of the job has completed are discarded.
Task Duration
For an initial load job, the amount of time the subtask that processed the source table ran before it completed or was stopped. For other load types, this information is available only at the job-level on the Job Overview tab.
When a job runs, it uses a separate subtask to process each source table.
Stage
For a combined initial and incremental load job, this column shows the stage in the transition from initial load processing to CDC processing for a subtask that's specific to a source object. This column does not appear for other load types.
The stage can be one of the following values:
  • - Not Started. Initial load processing has not yet started for the source object, or if an error occurred and the source object is in the Error on Retry state, the next attempt to process the object has not yet started.
  • - Started. Initial load processing has started.
  • - Unloading. The subtask is unloading data from the source object as part of initial load processing.
  • - Unloaded. The subtask has finished unloading data from the source objects as part of initial load processing.
  • - Backlog. The subtask is applying any change data that was captured from the source object during the initial unload phase to the target.
  • - Completed. The subtask completed initial load processing of the source object.
  • - Normal. The subtask completed initial load and backlog processing of the source object and has started normal processing of the change data stream from the source through to the target.
  • - Cancelled. Initial load processing was cancelled or stopped.
  • - Error. The subtask detected an error in the source object.
Actions menu > Resync
For a subtask in a combined initial and incremental load job, if the subtask stage is Normal and the subtask status is any status other than Queued or Starting, the Actions menu is displayed on the right end of the subtask row. From the Actions menu, you can select Resync to resynchronize the source and target objects. For more information, see "Resynchronizing source and target objects" in Data Ingestion and Replication help.
Status
The status of the job subtask for the source object.
Note: If the job stops running, the subtask status reflects the status last collected before the job ended. For example, the job might be aborted but the subtask is in a Running status.
The status can be one of the following values:
  • - Queued. The subtask has not yet started running.
  • - Starting. The subtask is starting.
  • - Started. For a combined initial and incremental load job, the subtask has started.
  • - Running. The subtask is running.
  • - On Hold. The subtask, as well as the job, is in a paused state while the Database Ingestion and Replication (DBMI) agent is being updated.
  • - Completed. The subtask completed processing successfully.
  • - Stopping. The subtask is stopping in response to a Stop request.
  • - Stopped. The subtask has stopped.
  • - Aborting. The subtask is ending immediately in response to an Abort request.
  • - Aborted. The subtask has been aborted.
  • - Failed. The subtask ended unexpectedly.
  • - Error. The subtask is in error and no longer writing data to the target table. For a combined initial and incremental load job, the subtask might be running and processing incremental change data but no data is being sent to the target.
  • - Error on Retry. An error occurred on the last retry of subtask processing, and now the subtask is waiting to retry processing again.
Note: If a DDL change occurs on a source table and then you resume the job, the table subtask state might not change as expected until the first DML operation occurs on the source table.
Log
You can download a job execution log for a source object. The type and availability of the log depends on the load type and status. Options are:
  • - Complete. The complete log for an object subtask from job execution. This log type is available for a Completed, Failed, or Aborted subtask in an initial load job.
  • - Error. The log that contains error messages. This log type is available only for a Failed or Error subtask in an initial load or incremental load job.
  • - Stage Log. The log that covers the transition from the initial processing phase to the incremental processing phase in a combined initial and incremental load job for a source object.
To download a log, click the Download icon.
Note: If you undeployed the job, you can download the log for a table only if the associated task has not been deleted.
For incremental load jobs, you can get the complete log and error log for the entire job run from the Task Summary pane.
Note: To control the line spacing in the list, click the Settings icon next to the Find box.

Alerts

The Alerts pane appears for a selected incremental load or combined initial and incremental load job. On the Alerts pane, you can view alert messages that appear for certain events, such as source schema changes, during incremental load or combined initial and incremental load processing.
Note: The Alerts pane displays alert messages for all detected schema changes even if you set the schema drift options for the associated task to Ignore.
You can filter the list of alerts based on severity or a date range. To specify a date range, enter one of the following types of values in the Filter field:
The list of alerts includes the following columns:
Column
Description
Level
Severity level of the alert message, such as Critical or Warning.
Code
Alphanumeric code that identifies the alert type followed by the date and time when the event occurred.
Click the expander arrow to display a description of the event.
You can configure alert notifications for application ingestion and replication jobs from the Alerts > Data Ingestion and Replication Alerts page in Operational Insights. Operational Insights then sends Data Ingestion and Replication alert notifications to the users and user groups you select, whenever an ingestion and replication job acquires one of the configured statuses, detects a DDL change, or a delay in the change data capture processing occurs.

Database ingestion and replication job details

You can drill down on a job to see job details from the My Jobs page accessed from the Data Integration Home page, from the All Jobs and Running Jobs pages in Monitor, and from the Overview and All Jobs tabs on the Data Ingestion and Replication page in Operational Insights.
After you click a job name, job information is displayed in the Task Summary, Overview, Apply Jobs, Object Detail, and Alerts panes. The Apply Jobs tab is displayed only for CDC Staging Tasks. The Alerts pane is displayed only for incremental load and combined initial and incremental load jobs.
The following image displays the panes:
Monitor Task Summary, Overview, Apply Jobs, and Alerts panesMonitor Object Detail pane
Note: Clicking the expander arrow next to the pane name expands or collapses the Task Summary, Object Detail, and Alerts panes.

Task Summary

View a diagram that shows the source and target connector types and connection names. It also shows the calculated data throughput, in rows per second if the job has successfully replicated data to the target, regardless of the job's current status. If the calculated value is 0, indicating no data has flowed to the target, the throughput is not displayed.
In the upper right corner, the Status field displays the job status, which can be one of the following values:
Also, for incremental load jobs and combined initial and incremental load jobs, you can download the job execution log for the entire job run. In the Select log list under the diagram, select one of the following log types:

Overview

View information about the job, including the runtime environment, task name, load type, task type, task definition location, user who started the job, records read, records written, number of subtasks, and schedule name, if a schedule was used.
The circle image displays the number of subtasks on source tables by status. The color of the circle's rim corresponds to the status.
The following table describes the summary job properties and statistics:
Property
Description
Duration (on left)
The amount of time, in the hh:mm:ss format, that the job ran before it ended.
Start Time (on left)
The date and time when the job was deployed.
End Time (on left)
The date and time when the job ended because it completed processing, was stopped, or failed. This field is not displayed for running jobs
Runtime Environment
The name of the runtime environment in which the job runs.
Agent Name
For incremental load and combined initial and incremental load jobs, the name of the Secure Agent on which the job runs.
If a failover of one Secure Agent to another Secure Agent occurs, Database Ingestion and Replication displays the latest Secure Agent assigned to the job.
Note: For initial load jobs, the Agent Name is displayed in the Object Detail pane.
Task Name
The name of the associated database ingestion and replication task or CDC Staging Task. You can click the task-name link to view or edit task details in Data Ingestion and Replication, if necessary. If you edit the task, you must redeploy it for the updated task definition to be used for a job.
Load Type
The type of load operation that the job performs. Options are:
  • - Initial Load. Loads a snapshot of source data read at a specific point-in-time to a target.
  • - Incremental Load. Loads incremental data changes to a target on a continuous basis, until the job is stopped or ends.
  • - Initial and Incremental Load. Performs an initial load and then automatically switches to an incremental load.
  • - CDC Staging Group. Appears only if you selected a CDC Staging Task.
Task Type
The type of task, which is Database Ingestion and ReplicationTask or CDC Staging Task.
Task Location
The project or project folder that contains the database ingestion and replication task definition.
Started By
The name of the user who started the job.
Records Read
The number of records that were read from the source.
Note: For a combined initial and incremental load job, the Records Read count might be greater than the total number of object-level DML change records written. This behavior occurs because the initial load or resync processing always starts after change data capture has begun. As a result, some change records are included in the Records Read count and then discarded before initial load or resync processing starts. These discarded records cause the Records Write count to be less than the Records Read count.
Records Written
The number of records that were successfully replicated to the target.
Note: The Records Written value might be different from the Records Read value if source records are discarded. For example, in a combined initial and incremental load job, change records read from the source before the initial unload phase completes are discarded because they're not yet needed.
Subtasks
The number of subtasks that the database ingestion and replication job used to replicate data from source tables to the target. When a job runs, it uses a separate subtask to process each source table.
Capture Progress
For incremental load and combined initial and increment load jobs, the date and time in the source change stream to which capture processing has progressed, as shown in the time zone of the user profile.
CDC Staging Job
For an apply job that's part of a CDC staging group, the job name of the parent CDC Staging Task.
Schedule
For initial load jobs, the name of the schedule that is used to run the job or "No schedule" if you run the job manually.

Apply Jobs

For a CDC Staging Task, view a list of the apply jobs that are associated with the staging group. Any apply jobs that have been undeployed or that were created before the May 2025 release are not listed. Click an apply job name to display job details, including the job status.

Object Detail

On the Object Detail pane lists subtasks on the source tables. You can view statistics and status information by source table from the last run of a database ingestion and replication job. When you click the expander arrow next to an object name, counts of processed inserts, updates, deletes, and LOB changes are shown for the table.
The following image shows a sample expanded object row for a combined initial and incremental load job:
Note: This pane shows information for the latest job run. This tab is blank for jobs that have not run or are resuming.
The following table describes the Object Detail fields that are displayed for each table, depending on the load type and status:
Column
Description
Object
The name of the source table or view for which data was propagated to the target.
For an incremental load job or a combined initial and incremental load job, click the arrow icon to the left of the object name to display detailed counts of Inserts, Deletes, Updates, LOBs, and DDL statements processed. For a combined initial and incremental load job, the Unload Count field is also displayed to show the number of records that the initial load portion of processing read from the source. The following usage notes apply to the detailed CDC counts:
  • - The counts are only for the current job run. If you stop and restart the job, the counts start over from zero. Do not use these counts to identify the number of rows written to the target.
  • - The counts are based on rows read from the source and do not reflect the records written to the target. Target write operations might be optimized by combining operations and reducing the number of physical writes. In this case, the counts might not match the number of write operations.
  • - The value N/A means that the count value is not applicable for the count type or the value has not yet been calculated.
  • - The Unload Count might not reflect the number of source records at the time the job is started or resynchronized because of a delay in the start of unload processing. Between the time of the unload request and start of unload processing, rows might be added to or deleted from the source table.
Target Object
The name of the target object that is mapped to the source object.
Agent Name
For initial load jobs, the name of the Secure Agent on which the job ran.
For incremental load and combined initial and incremental load jobs, the Agent Name is displayed in the Overview section of the Task Summary pane.
Unload Agent Name
For combined initial and incremental load jobs, the name of the Secure Agent on which the initial unload phase of the job ran.
Records Read
For an initial load job, the number of records that were read from the source. For other load types, this information is available only at the job-level on the Job Overview tab.
Records Written
For an initial load job, the number of records that were successfully written to the target. For other load types, this information is available only at the job-level on the Job Overview tab.
Note: The Records Read value can be greater than the Records Written value if some records read from the source were discarded. For example, in a combined initial and incremental job, any source change records read before the initial unload phase of the job has completed are discarded.
Task Duration
For an initial load job, the amount of time the subtask that processed the source table ran before it completed or was stopped. For other load types, this information is available only at the job-level on the Job Overview tab.
When a job runs, it uses a separate subtask to process each source table.
Stage
For a combined initial and incremental load job, this column shows the stage in the transition from initial load processing to CDC processing for the table-specific job subtask. This column does not appear for other load types.
The stage can be one of the following values:
  • - Not Started. Initial load processing has not yet started for the table, or if an error occurred and the table is in the Error on Retry state, the next attempt to process the table has not yet started.
  • - Started. Initial load processing has started.
  • - Unloading. The subtask is unloading data from the table as part of initial load processing.
  • - Unloaded. The subtask has finished unloading data from the table as part of initial load processing.
  • - Backlog. The subtask is applying any change data that was captured from the source table during the initial unload phase to the target.
  • - Completed. The subtask completed initial load processing of the table.
  • - Normal. The subtask completed initial load and backlog processing of the table and has started normal processing of the change data stream from the table through to the target.
  • - Cancelled. Initial load processing was cancelled or stopped.
  • - Error. The subtask detected an error in the source table.
Actions menu > Resync
For a subtask in a combined initial and incremental load job, if the subtask stage is Normal and the subtask status is any status other than Queued or Starting, the Actions menu is displayed on the right end of the subtask row. From the Actions menu, you can select Resync to resynchronize the source and target objects. For more information, see "Resynchronizing source and target objects" in Data Ingestion and Replication help.
Status
The status of the job subtask for the source object.
Note: If the job stops running, the subtask status reflects the status last collected before the job ended. For example, the job might be aborted but the subtask is in a Running status.
The state can be one of the following values:
  • - Queued. The subtask has not yet started running.
  • - Starting. The subtask is starting.
  • - Started. For a combined initial and incremental load job, the subtask has started.
  • - Running. The subtask is running.
  • - On Hold. The subtask, as well as the job, is in a paused state while the Database Ingestion and Replication (DBMI) agent is being updated.
  • - Completed. The subtask completed processing successfully.
  • - Stopping. The subtask is stopping in response to a Stop request.
  • - Stopped. The subtask has stopped.
  • - Aborting. The subtask is ending immediately in response to an Abort request.
  • - Aborted. The subtask has been aborted.
  • - Failed. The subtask ended unexpectedly.
  • - Error. The subtask is in error and no longer writing data to the target table. For a combined initial and incremental load job, the subtask might be running and processing incremental change data but no data is being sent to the target.
  • - Error on Retry. An error occurred on the last retry of subtask processing, and now the subtask is waiting to retry processing again.
Note: If a DDL change occurs on a source table and then you resume the job, the table subtask state might not change as expected until the first DML operation occurs on the source table.
Data Validation
For initial load jobs that have an Oracle or a SQL Server source and a Snowflake target, if the task status is Completed, you can run data validation to compare the source and target data. This column shows the status of the data validation task. This column does not appear for other load types and other source and target combinations, or if the task is in any other status than Completed.
From the Actions menu displayed on the right end of the subtask row, you can select Run Data Validation to start the validation task. For more information, see "Running data validation for a database ingestion and replication job" in Data Ingestion and Replication help.
The data validation status can be one of the following values:
  • - N/A. The data validation has not been run.
  • - Processing. The data validation is running.
  • - Success. The data validation completed successfully. Click on the Success status to see the data validation summary. To download the data validation report, click the Download icon.
  • - Error. An error occurred during the data validation execution. To view the error message, click the Download icon.
  • - Report Pending. The data validation has been requested but the Data Validation service on the Secure Agent that runs the task is down.
  • - No Comparable Data. The source or target table does not contain any data.
CDV Last Run
If data validation is available for the task, this column shows the date and time when the data validation was last performed for the given subtask. If the data validation has not been run, the field value is N/A.
This column appears in initial load jobs that have an Oracle or a SQL Server source and a Snowflake target, only if the task status is Completed.
Log
You can download a job execution log for a source object. The type and availability of the log depends on the load type and status. Options are:
  • - Complete. The complete log for an object subtask from job execution. This log type is available for a Completed, Failed, or Aborted subtask in an initial load job.
  • - Error. The log that contains error messages. This log type is available only for a Failed or Error subtask in an initial load or incremental load job.
  • - Stage Log. The log that covers the transition from the initial processing phase to the incremental processing phase in a combined initial and incremental load job for a source object.
To download a log, click the Download icon.
Note: If you undeployed the job, you can download the log for a table only if the associated task has not been deleted.
For incremental load jobs, you can get the complete log and error log for the entire job run from the Task Summary pane.
Note: To control the line spacing in the list, click the Settings icon next to the Find box.

Alerts

The Alerts pane appears for the selected incremental load or combined initial and incremental load job. On the Alerts pane, you can view alert messages that appear for certain events, such as source schema changes, during incremental load or combined initial and incremental load processing.
The following image shows this pane with an alert example:
Note: The Alerts pane displays alert messages for all detected schema changes even if you set the schema drift options for the associated task to Ignore.
You can filter the list of alerts based on severity or a date range. To specify a date range, enter one of the following types of values in the Filter field:
The list of alerts includes the following columns:
Column
Description
Level
Severity level of the alert message, such as Critical or Warning.
Code
Alphanumeric code that identifies the alert type followed by the date and time when the event occurred.
Click the expander arrow to display a description of the event.
You can configure alert notifications for database ingestion and replication jobs from the Alerts > Data Ingestion and Replication Alerts page in Operational Insights. Operational Insights then sends Data Ingestion and Replication alert notifications to the users and user groups you select, whenever a database ingestion and replication job acquires one of the configured statuses, detects a DDL change, or a delay in the change data capture processing occurs.

File ingestion and replication job details

You can view job details and results for each file ingestion and replication job instance, including the status of the job and success and error statistics.
To view detailed information about a file ingestion and replication task, click the task name on the My Jobs page that's accessed from the navigation bar in Data Integration Home page, the All Jobs and Running Jobs pages in Monitor, or from either tab of the Data Ingestion and Replication page in Operational Insights. You can also download the session log.
The following image shows the details of a file ingestion and replication job in Operational Insights:
Job details of file ingestion jobs

Job Properties

The job properties for a file ingestion and replication job instance include general information about the job, when it started and ended, and its duration.
The following table describes the job properties:
Property
Description
Task Name
The name of the associated file ingestion and replication task. You can click the task-name link to view or edit task details in Data Ingestion and Replication.
Task Type
Task type. In this case, file ingestion and replication task.
Started By
Name of the user or schedule that started the job.
Start Time
Date and time when the job was started.
End Time
Date and time when the job completed or stopped.
Duration
The amount of time the job ran before it completed or was stopped.

Results

The job results for a file ingestion and replication job instance include the job status and error statistics.
The following table describes the job results:
Property
Description
State
Job status. A job can have one of the following statuses:
  • - Running. The job is still running.
  • - Success. The job completed successfully.
  • - Failed. The job did not complete because it encountered errors
  • - Aborted. The job was aborted.
    • Note: When the Secure Agent is unavailable while running a file ingestion job, the job remains in an unresponsive state, and after 200 minutes, its status changes to Aborted.
Session Log
Allows you to download the session log file.
By default, the session log files are written to the following directory:
<Secure Agent installation directory>/apps/MassIngestionRuntime/data/taskLogs
Success Files
Number of files that are successfully transferred, downloaded, and uploaded to the target.
Error Files
Number of files that were not transferred to the target.
Duplicate Files
Number of files that were identified as duplicates.
Error Message
Error message, if any, that is associated with the job.

File Events

This section shows the total number of files that the file ingestion and replication job transferred with information about each file.
The File Events section is updated each time the file ingestion and replication job transfers a file. The state of the file is updated throughout the file transfer process. You can track the progress of a file transfer based on the state of the file.
The File Events section displays the following properties for each file:
Property
Description
Name
The name of the file.
File size
The size of the file in bytes.
Status
The status of the file transfer. A file can have one of the following status:
  • - Success. The file transfer completed successfully.
  • - Failed. The file transfer did not complete because it encountered errors.
  • - Processing. The file transfer is still running.
  • - Duplicate. The task previously transferred a file with the same name, directory location, and size.
  • - Interrupted. The file transfer is interrupted because of network issues or changed server credentials during the file transfer. Run the file ingestion and replication job to resume the transfer of the interrupted files.
    • Note: The status is applicable when the file ingestion and replication job transfers file from or to the advanced FTP, advanced SFTP, or advanced FTPS sources and targets.
  • - In Doubt. The previous task instance encountered errors while transferring the file. Applicable for tasks where the source is configured to skip duplicate files.
  • - Quarantined. The job marks any infected file it detects from a source as quarantined.
You can monitor the Status property to track the progress of the file transfer of each file.
Transfer Type
The type of file transfer. A file can have one of the following transfer types:
  • - <source_name>Download. The file is downloaded from source. <source_name> is the name of the source.
  • - <target_name>Upload. The file is uploaded to the target.<target_name> is the name of the target.
  • - Copy from Source. The file ingestion and replication task is performing file processing actions.
  • - Copy to Target. The file is transferred from a local directory to a local directory.
Start time
Date and time when the file transfer started.
Duration
The length of time to transfer the file, in milliseconds.
Remarks
Applies to file events in Failed status. The message includes the reason for failure of the event based on the file transfer type.
Note: If you click the instance name of a file ingestion and replication job on the My Jobs page, accessed from the Data Integration Home page, the job details page appears. On this page, you can click the name of the associated task to open the task definition in edit mode if you need to change the task and re-run the job.

Streaming ingestion and replication job details

You can drill down on a job to see job details from the My Jobs page accessed from the Data Integration Home page, from the All Jobs and Running Jobs pages in Monitor, and from the Overview and All Jobs tabs on the Data Ingestion and Replication page in Operational Insights.
After you click a job name, job information is displayed on the Overview tab with a summary source-to-target processing diagram at the top. Information is also displayed on the Alert, Performance, and Past Run tabs if available.
The following image shows the diagram and Overview tab:

Overview tab

The Overview tab displays general properties of the job. You can download the job log too.
The following table describes the job overview properties:
Property
Description
Job Name
The name of the job.
Version
The version number of the job.
Task Type
The task type of streaming ingestion and replication task.
Task Location
The project or project folder that contains the streaming ingestion and replication task.
Started By
The name of the user who deployed the job.
Secure Agent
The location where the Secure Agent is running.
A warning symbol near the Secure Agent indicates that the Secure Agent is either offline or not reachable.
Status
The status of the job.
A job can have one of the following statuses:
  • - Deploying. The job is being deployed.
  • - Up and Running. The job is running.
  • - Running with Warning. The job is running with warnings.
  • - Running with Error. The job is running with error. If a job continuously runs with warnings for seven minutes or for the time specified in the runtime option, the state of the job changes to Running with Error.
  • - Undeployed. The job is undeployed.
  • - Stopped. The job was intentionally stopped.
Duration
Total time the job ran before it is undeployed. The total time is shown in hh:mm:ss format.
Start Time
The date and time when the job was deployed.
Runtime Environment
Name of the runtime environment that the job uses to run.
Download Log
You can download one of the following types of logs:
  • - Complete. The entire log, including all types of messages. It is available for any job that ran, regardless of its status.
  • - Latest. Latest version of the log.
To download a log to your local system, click the Download icon.

Alert tab

The Alert tab displays the alert messages when an event occurs.
The following image shows the Alert tab for a streaming ingestion and replication job:
Alert tab details
The following table describes the job alert properties:
Property
Description
Alert
The messages or a group of messages that the job returns when a deployed job encounters a warning.
Time
The date and time when the event occurred.

Performance tab

The Performance tab displays graphs of throughput information for the source and target of the job.
The following image shows the Performance tab:
The following table describes the job performance properties:
Property
Description
Total messages
The average number of messages streamed per second.
Total kb
The average kilobits of messages streamed per second.

Past Run tab

The Past Run tab displays the statistics and status information for previous runs of a streaming ingestion and replication job.
The following table describes the properties displayed for each run:
Column
Description
Version
The version number of the job.
Start Time
The date and time when the job was deployed.
End Time
The date and time when the job was undeployed.
Duration
Total time the job ran before it is undeployed. The total time is shown in hh:mm:ss format.