The May 2024 release of Data Ingestion and Replication includes the following changed behaviors.
Monitoring Mass Ingestion jobs in Data Integration and Monitor services
The Data Integration and Monitor services include the following changes for monitoring Mass Ingestion jobs:
•The My Jobs page in Data Integration and the All Jobs page in Monitor display all of your jobs or all jobs, respectively, but the retention time for undeployed jobs is based on the Job Log Service (JLS) purge policy.
Both pages display only the new jobs created from the current release. The undeployed jobs from the previous release are excluded.
Previously, the My Jobs page in Data Ingestion and Replication used to display all jobs without purging. Therefore, the number of jobs listed in the Data Ingestion and Replication service could be greater than in the Data Integration service.
• When you deploy an application ingestion task or database ingestion task, the newest job doesn't automatically appear at the top of the jobs list on the My Jobs page in Data Integration and the All Jobs page in the Monitor. You can sort jobs by using the Start Time property or use the find and filter features to search for a specific job.
Previously, you could sort jobs from newest to oldest on the My Jobs page in the Data Ingestion and Replication service.
•When you click the instance name of a file ingestion job on the My Jobs page in Data Integration, the job details page appears. When you click the task name from the job details page, it opens in the edit mode.
Previously, when you clicked the task name from the job details page, the details opened only in the view mode for file ingestion tasks in Mass Ingestion.
•When you undeploy a streaming ingestion and replication job, the history of the undeployed tasks appears on both the My Jobs page in Data Integration and the All Jobs page in Monitor.
Previously, only the latest tasks were listed.
The generated name of the job instance contains a hyphen (-) instead of an underscore (_) before the job instance number
The generated name of the job instance now contains a hyphen (-) instead of an underscore (_) between the task name and job instance number on the My Jobs page, on the Running Jobs and All Jobs pages in Monitor, and in the Recent jobs panel of the unified Home page:
<taskname>-<job_instance_number>
In the Cloud Data Ingestion and Replication Command Line Interface (CLI) and in Operational Insights, the format of the job instance name continues to use the underscore (_) before the job instance number:
<taskname>_<job_instance_number>
In the CLI, you must use the <taskname>_<job_instance_number naming convention.
Default age-out period for incremental load processing has increased
Database ingestion incremental load and combined initial and incremental load jobs stop tracking transactions for restart purposes when those transactions have no DML for tables of interest after an age-out period. The age-out period has been increased from 60 minutes to 24 hours.
The restart point where Data Ingestion and Replication starts reading change data might include recapturing empty transactions that are up to 24 hours old.