Database Ingestion and Replication > Database Ingestion and Replication sources - preparation and usage > Db2 for z/OS sources
  

Db2 for z/OS sources

To use Db2 for z/OS sources in database ingestion and replication tasks, first prepare the source database and review the usage considerations.

Source preparation

Usage considerations

Installing and configuring the stored procedure for Db2 for z/OS CDC

To perform Db2 for z/OS CDC processing for incremental load jobs, Database Ingestion and Replication provides a stored procedure that runs on the z/OS source system. The stored procedure calls the Db2 Instrumentation Facility Interface (IFI) to collect change data from the Db2 logs.

z/OS system requirements

Before you begin, verify that the Db2 for z/OS source system meets the following requirements:

Install the Stored Procedure Libraries and Customize the JCL

On your client machine, perform the following steps:
  1. 1Verify that the Db2 for zOS Database Ingestion connector is available.
  2. The Db2 for zOS Database Ingestion connector package contains the Db2 for z/OS stored procedure libraries. When the connector is enabled for the Secure Agent Group (runtime environment), the connector package .zip file is downloaded to the downloads folder in the installation location. The package name has the format package-DB2ZMI.nnnnn, where nnnnn is an incremented package version number. If multiple package versions exist, use the latest one.
  3. 2Unzip the package-DB2ZMI.nnnnn.zip file. The stored procedure files are added to the Db2WLMStoredProcedure folder under the package name.
  4. 3Use FTP to transfer the #STPINST file in the Db2WLMStoredProcedure folder to a sequential file, PDS, or PDSE on the z/OS system.
  5. Notes:
  6. 4Use FTP or another file transfer method to transfer each of the following files to a separate data set on the z/OS system:
  7. Notes:
On the z/OS system, use TSO to receive the transmit (XMI) data sets into an APF-authorized library and then edit the stored procedure JCL member. Also, set Db2 user privileges and add a row to the resource limit table if used.
  1. 1Receive the DBRMLIB transmit data set:
  2. RECEIVE INDATASET(DBMI.ZOS.DBRMLIB.XMI)
    Notes:
  3. 2Receive the LOADLIB transmit data set:
  4. RECEIVE INDATASET(DBMI.ZOS.LOADLIB.XMI)
    See the Notes in Step 1.
  5. 3Receive the USERJCL transmit data set:
  6. RECEIVE INDATASET(DBMI.ZOS.USERJCL.XMI)
    See the Notes in Step 1.
  7. 4Customize the #STPINST file that contains the stored procedure JCL for your environment.
  8. The JCL creates the stored procedure and a global temporary table that holds the results of the requests to the Db2 IFI for data. It also binds the stored procedure package.
    Notes:
  9. 5Customize the JCL in members of the received USERJCL data set. For more information, see Db2 for z/OS USERJCL Data Set.
  10. 6Ensure that the required Db2 privileges have been granted before you run the stored procedure JCL job. For more information, see Db2 for z/OS privileges.
  11. 7Add a row in the Db2 DSNRLSTxx resource limit table for database ingestion packages to ensure processor resources are adequate for stored procedure processing. Otherwise, incremental load jobs might abend. Add the row with the following columns:
  12. Then issue the Db2 -START RLIMIT command for For the changes to the resource limit table to take effect.

Db2 for z/OS USERJCL Data Set

The downloaded USERJCL data set is a partitioned data set (PDS) or extended partitioned data set (PDSE) that contains a JCL member for running a job that collects information that is not readily available otherwise when using Informatica-supplied WLM Db2 stored procedure.
You can fully customize member JCL in the USERJCL PDS or PDSE. Informatica recommends that you create a copy of a delivered JCL member under another name and then customize the copy so that original member is retained intact for reference.
If you install multiple stored procedures on multiple Db2 subsystems, you can install only one USERJCL library and create members in it that are tailored for each Db2 subsystem. Alternatively, you can create a separate library for a specific Db2 subsystem. Make sure that the database ingestion and replication job contains the correct library and member information for the Db2 subsystem.
The USERJCL PDS or PDSE contains the following member:
LOGINV member
LOGINV contains JCL for a job that obtains a Db2 log inventory list. If you set the Initial Start Point for Incremental Load property to Specific Date and Time for an ingestion task in the task wizard, the inventory list is used to determine the start point in the logs when the database ingestion and replication job runs for the first time or restarts. The log inventory list provides Database Ingestion and Replication with a starting RBA or LSN and an ending RBA or LRSN, along with the starting and ending timestamps of all active and archive logs for the Db2 subsystem. Database Ingestion and Replication uses this information to select the appropriate log archive to search for the requested start point. If the Initial Start Point for Incremental Load property is set to any option other than Specific Date and Time, you do not need to install the USERJCL library.
Contents of the LOGINV member:
//<USERID>I JOB 'LOG INVENTORY',MSGLEVEL=(1,1),MSGCLASS=X,
// NOTIFY=&SYSUID,CLASS=A,REGION=0M
//* -------------------------------------------------------
//*
//* PLEASE CHANGE DSN? TO THE DB2 SUBSYSTEM OF YOUR CHOICE.
//*
//* THIS JCL MEMBER CAN BE USED TO RUN A LOG INVENTORY
//* LIST. DBMI WILL REQUEST THIS IF A DBMI JOB IS TRYING TO
//* RETART USING TIME. THE LOG INVERNTORY GIVES DBMI THE
//* ABILITY TO CORRELATE RBA/LSRN TO ACTUAL LOG RESTART
//* POSITIONS. DBMI PARSES THE SYSPRINT OUTPUT TO GET THE
//* REQUIRED INFORMATION.
//*
//*
//* SUBSTITUTION TAGS
//* _________________
//*
//* SUBSTITUTION TAGS ARE USED TO INSERT DATA INTO THE JOB
//* BY DBMI BEFORE IT IS SUBMITTED. YOU MAY COPY THIS JCL
//* INTO ANOTHER MEMBER AND MODIFY ITS CONTENTS. SUBSTITUTION
//* TAGS MAY ALSO BE REMOVED AND HARD CODED FOR INDIVIDUAL
//* JOB NEEDS.
//*
//*
//* <USERID> WILL BE REPLACED WITH THE USER ID.
//*
//* -------------------------------------------------------
//LOGINV EXEC PGM=DSNJU004
//STEPLIB DD DISP=SHR,DSN=DSN?10.SDSNLOAD
// DD DISP=SHR,DSN=DSN?10.DSNC.RUNLIB.LOAD
//SYSUT1 DD DISP=SHR,DSN=DSN?.BSDS01
//SYSPRINT DD SYSOUT=*
//SYSSUMRY DD SYSOUT=*
//
To submit USERJCL member jobs and retrieve output from submitted jobs, Database Ingestion and Replication executes Db2-supplied stored procedures in batch as required. Make sure that the Database Ingestion and Replication user has the required privileges to execute these stored procedures. For more information about Db2-supplied procedures, see https://www.ibm.com/docs/en/db2-for-zos/12?topic=sql-procedures-that-are-supplied-db2.
A USERJCL member is processed in the following manner:
  1. 1For the LOGINV member, if the Initial Start Point for Incremental Load advanced property is set to Specific Date and Time on the Source page of the database ingestion and replication task wizard, Database Ingestion and Replication checks whether the following custom properties are specified for the source:
  2. 2Database Ingestion and Replication reads the LOGINV member by using the Db2-supplied stored procedure ADMIN_DS_BROWSE.
  3. 3Database Ingestion and Replication substitutes any tags required for the job execution.
  4. 4Database Ingestion and Replication submits the job to the Db2 for z/OS database by using the Db2-supplied stored procedure ADMIN_SUBMIT_JOB.
  5. 5Database Ingestion and Replication queries the status of the submitted jobs by using the Db2-supplied stored procedure ADMIN_JOB_QUERY.
  6. 6After the job completes, Database Ingestion and Replication obtains the job output by using the Db2-supplied stored procedure ADMIN_JOB_FETCH.

Db2 for z/OS privileges

To deploy and run a database ingestion and replication task that has a Db2 for z/OS source, the source connection must specify a Database Ingestion and Replication user who has the privileges required for the ingestion load type.
Privileges for initial load processing
Privileges for incremental load processing
For incremental load jobs, grant the following privileges.
Note: If you do not have the authority required to issue these grants, ask a Db2 administrator who has SYSDBA authority or a higher authority level to issue them.
Note: To make the change data for tables in the Db2 logs available to the stored procedure, issue the following SQL for each source table:
ALTER TABLE schema.table DATA CAPTURE CHANGES <--for each source table
COMMIT;
To issue this statement, you'll need ALTER authority. If you don't have it, ask the Db2 DBA to issue this SQL or to grant ALTER authority to you. For more more information, see Enabling Db2 DATA CAPTURE CHANGES.
Permissions Required for Executing the Stored Procedure on z/OS
Ensure that the following Db2 permissions are granted before you run the stored procedure JCL job:
Additionally, grant INSERT and DELETE authority on the schema and stored procedure name that are specified in the JCL for the global temporary table:
GRANT INSERT, DELETE ON schema.stored_procedure_name_RS_TBL TO user

Enabling Db2 DATA CAPTURE CHANGES

You must enable Db2 DATA CAPTURE CHANGES on some system catalog tables and the Db2 source tables to log SQL change events in an expanded format that is required for change data replication.

Change capture mechanism for Db2 for z/OS sources

Database Ingestion and Replication provides a single change capture mechanism and architecture for capturing change data from a Db2 source on z/OS and applying the data to a target. This architecture uses multithreaded processing to optimize the performance of collecting data and of parsing and transforming the data into a format the target accepts.
The Secure Agent must run on a Linux or Windows box, separate from the Db2 z/OS source system. The network bandwidth between the z/OS system and the Secure Agent box must be robust. Informatica recommends network transfer rates that can handle100s of gigabytes or one or more gigabits of log data. If the network transfer rate is not capable of delivering the log data to the Secure Agent at speeds equal to or greater than the rate at which Db2 produces log data of CDC interest, the database ingestion and replication job will not be able to provide the data to the target in a timely manner. If the data throughput does not meet your SLA, consider changing the hardware to increase the ethernet bandwidth between the z/OS system and the Secure Agent box.
The following image shows the Db2 for z/OS change capture components and data flow:
  1. 1The multithreaded log collector, which runs in the DBMI Agent service under the Secure Agent, issues multiple concurrent requests for Db2 log data to the Db2 stored procedure.
  2. Each request includes a starting RBA, or a starting LRSN if the logs are in a data-sharing environment, along with a list of tables of CDC interest for database ingestion.
    While processing a sequence of log data, the log collector can request the next sequence of log data.
  3. 2The Db2 Instrumentation Facility Interface ( IFI) API extracts data for the selected source tables of CDC interest from active and archive logs. The IFI then transfers the data, in raw, native form, to the Informatica Db2 stored procedure in z/OS Workload Manager (WLM).
  4. 3The Db2 stored procedure returns the UOWs with captured data to the UOW Manager.
  5. 4The UOW Manager sends the UOWs in commit order to the log parser.
  6. 5The multithreaded log parser concurrently parses DML changes from the committed UOWs. The result is raw native Db2 log data in the format that the target type expects.
  7. Note: Because this resource-intensive activity occurs on the Secure Agent box, CPU consumption on the z/OS system is minimized.
  8. 6The CDC writer applies the formatted data to the target.