Catalog Source Configuration > Oracle SQL Loader > Before you begin
  

Before you begin

Before you create a catalog source, ensure that you have the information required to connect to the source system.
Perform the following tasks:

Copy the required files to the Secure Agent machine

The Oracle SQL Loader catalog source job processes control files and loader files to extract metadata. The catalog source job reads these files from the paths that you specify when you configure the catalog source. Additionally, to perform connection assignment, it needs access to data files.
You can choose to provide paths for both control files and loader files, or to provide a path only for the control files. The catalog source job processes control files and loader files differently based on the input you provide.
When you enter only the control file path, the catalog source job extracts all necessary information from the control file. It parses the control file to identify the source data file location, target table, column mappings, data transformations, and to load the required settings. It then uses this information to determine the data flow, validate the connection, and to generate lineage. Because the control file contains the complete configuration for the Oracle SQL Loader task, the catalog source jobs run without issues.
When you enter both the required control file path and the optional loader file path, the catalog source job uses information from both files to extract lineage. The control file defines the data structure, target table mappings, and transformation rules, and the loader file provides the command line parameters, such as file paths, connection details, and options to load data. Together, the files give the catalog source job a complete view of the data loading process. This allows it to accurately map the source to the target.
If you have both control files and loader files, enter a value for the optional loader file path. For example, the control file directory contains multiple files and you want to extract metadata from specific control files or for a specific control file pattern. You can then provide the path to the loader file directory. The loader file provides instructions on which control files or patterns to extract.
Perform the following tasks to ensure that the catalog source job can access the required files:
  1. 1Create or identify a directory on the Secure Agent machine to copy the control files.
  2. 2Copy the control files from the Oracle SQL Loader source system to the directory.
  3. 3Create or identify a directory on the Secure Agent machine to copy the loader files.
  4. 4Copy the loader files from the Oracle SQL Loader source system to the directory.
  5. 5Optional. Perform the following steps if you want to perform connection assignment:
    1. aCreate or identify a directory on the Secure Agent machine to copy the data files.
    2. bCopy the data files from the Oracle SQL Loader source system to the directory.

Verify permissions

To extract metadata and to configure other capabilities that a catalog source might include, you need account access and permissions on the source system. The permissions required might vary depending on the capability.

Permissions for metadata extraction

Ensure that you have the required permissions to enable metadata extraction.
Grant the following permissions:

Create a connection

Before you configure the Oracle SQL Loader catalog source, create a connection object in Administrator.
    1In Administrator, select Connections.
    2Click Add Connection.
    3Search for and select Oracle and then click Next.
    4Enter the following connection details:
    Property
    Description
    Connection Name
    Name of the connection.
    Each connection name must be unique within the organization. Connection names can contain alphanumeric characters, spaces, and the following special characters: _ . + -,
    Maximum length is 255 characters.
    Description
    Description of the connection. Maximum length is 4000 characters.
    Use Secret Vault
    Stores sensitive credentials for this connection in the secrets manager that is configured for your organization.
    This property appears only if secrets manager is set up for your organization.
    When you enable the secret vault in the connection, you can select which credentials that the Secure Agent retrieves from the secrets manager. If you don't enable this option, the credentials are stored in the repository or on a local Secure Agent, depending on how your organization is configured.
    Note: If you’re using this connection to apply data access policies through pushdown or proxy services, you cannot use the Secret Vault configuration option.
    For information about how to configure and use a secrets manager, see Secrets manager configuration.
    Runtime Environment
    The name of the runtime environment where you want to run the tasks.
    Select a Secure Agent, Hosted Agent, or serverless runtime environment.
    Select a Secure Agent, Hosted Agent, serverless, or elastic runtime environment.
    Oracle Subtype
    The Oracle connection subtype that you can use to connect to Oracle on-premises or Oracle Autonomous Database.
    Select one of the following options:
    • - Oracle ADB. Connects to Oracle Autonomous Database.
    • - Oracle On-premise. Connects to Oracle on-premises.
    5Select Oracle Database Authentication as the Authentication Mode.
    6Enter properties specific to the Oracle connection:
    Property
    Description
    User Name
    User name for the database login. The user name can't contain a semicolon.
    Password
    Password for the database login. The password can't contain a semicolon.
    Host
    Name of the machine that hosts the database server.
    Port
    Network port number used to connect to the database server.
    Default is 1521.
    Service Name
    Service name or System ID (SID) that uniquely identifies the Oracle database.
    Specify the SID in the following format to connect to Oracle databases:
    SID:<ORACLE_SID>
    Schema
    The schema name to select tables during object selection in a mapping.
    If you want to select tables from multiple schemas, leave the field blank. When left blank, all schemas you have access to are displayed and you can select tables from the available schemas.
    Code Page
    The code page of the database server.
    7Enter the Advanced Settings:
    Property
    Description
    Encryption Method
    The method that the Secure Agent uses to encrypt the data exchanged between the Secure Agent and the database server.
    Default is No Encryption.
    This property doesn't apply if you use the Hosted Agent.
    Crypto Protocol Version
    Cryptographic protocols to use when you enable SSL encryption.
    Not applicable when you use the Hosted Agent or the serverless runtime environment.
    Validate Server Certificate
    Validates the certificate that is sent by the database server. If you specify the HostNameInCertificate parameter, the Secure Agent also validates the host name in the certificate.
    Trust Store
    The location and name of the truststore file.
    For the serverless runtime environment, specify the following certificate path in the serverless agent directory:
    /home/cldagnt/SystemAgent/serverless/configurations/ssl_store/<TrustStore_filename>
    Trust Store Password
    The password to access the contents of the truststore file.
    Host Name in Certificate
    Host name of the machine that hosts the secure database.
    If you specify a host name, the Secure Agent validates the host name included in the connection with the host name in the SSL certificate.
    Key Store
    The location and the file name of the keystore.
    For the serverless runtime environment, specify the following certificate path in the serverless agent directory:
    /home/cldagnt/SystemAgent/serverless/configurations/ssl_store/<KeyStore_filename>
    Key Store Password
    The password for the keystore file required for secure communication.
    Metadata Advanced Connection Properties
    Additional properties for the JDBC driver to fetch the metadata. Enter properties in the following format:
    <parameter name>=<parameter value>
    If you enter more than one property, separate each key-value pair with a semicolon.
    For example, enter the following property to configure the connection timeout when you test a connection:
    LoginTimeout=<value_in_seconds>
    Note: The default connection timeout is 270 seconds.
    To connect to an Oracle database enabled for advanced security, you can specify the Oracle advanced security options for the JDBC driver.
    For example, EncryptionTypes=AES256;
    EncryptionLevel=accepted;DataIntegrityLevel=accepted;
    DataIntegrityTypes=SHA1
    Runtime Advanced Connection Properties
    Additional properties for the ODBC driver to run mappings.
    If you specify more than one property, separate each key-value pair with a semicolon.
    For example, charset=sjis;
    readtimeout=180
    To connect to an Oracle database enabled for advanced security, you can specify the Oracle advanced security options for the ODBC driver.
    For example, EncryptionTypes=AES256;EncryptionLevel=1;
    DataIntegrityLevel=1;DataIntegrityTypes=SHA1;
    DataIntegrityTypes=SHA1
    8Click Test.
    9Click Save.

Create endpoint catalog sources for connection assignment

An endpoint catalog source represents a source system that the catalog source references. Before you perform connection assignment, create endpoint catalog sources and run the catalog source jobs.
You can then perform connection assignment to reference source systems and run connection-aware scans to view complete lineage with source system objects.