Intelligent Data Lake Service Properties
To view the Intelligent Data Lake Service properties, select the service in the Domain Navigator and click the Properties View. You can edit the properties by clicking the pencil icon in the respective area, while the service is running, but you must restart the service for the properties to take effect. You can configure the following Intelligent Data Lake Service properties:
- •General Properties
- •Model Repository Service Options
- •Data Preparation Service Options
- •Data Integration Service Options
- •Catalog Service Options
- •Data Lake Security Options
- •Data Lake Options
- •Event Logging Options
- •Export Options
- •Data Asset Recommendation Options
- •Sampling Options
- •Logging Options
- •Custom Options
General Properties
General properties for the Intelligent Data Lake Service include the name, description, license, and the node in the Informatica domain that the Intelligent Data Lake Service runs on.
To edit the general properties, click the pencil icon in the general properties area. In the Edit General Properties window, edit the required fields.
The following table describes the general properties for the service:
Property | Description |
|---|
Name | Name of the Intelligent Data Lake Service. The name is not case sensitive and must be unique within the domain. It cannot exceed 128 characters or begin with @. It also cannot contain spaces or the following special characters: ` ~ % ^ * + = { } \ ; : ' " / ? . , < > | ! ( ) ] [ |
Description | Description of the Intelligent Data Lake Service. The description cannot exceed 765 characters. |
License | License object with the data lake option that allows the use of the Intelligent Data Lake Service. |
Node Assignment | Type of node in the Informatica domain on which the Intelligent Data Lake Service runs. Select Single Node if a single service process runs on the node or Primary and Backup Nodes if a service process is enabled on each node for high availability. However, only a single process runs at any given time, and the other processes maintain standby status. The Primary and Backup Nodes option will be available for selection based on the license configuration. Default is Single Node. |
Node | Name of the node on which the Intelligent Data Lake Service runs. |
Backup Nodes | If your license includes high availability, nodes on which the service can run if the primary node is unavailable. |
Model Repository Service Options
The Model Repository Service is an application service that manages the Model repository. When an analyst creates projects, the Model Repository Service connects to the Model repository to store the project metadata. When you create the Intelligent Data Lake Service, you must associate it with a Model Repository Service using the Model Repository Service Options properties.
To edit the Model Repository Service options, click the pencil icon. In the Edit Model Repository Service Optionswindow, edit the required fields.
The following table describes the Model Repository Service options:
Property | Description |
|---|
Model Repository Service | Name of the Model Repository Service associated with the Intelligent Data Lake Service. |
Model Repository Service User Name | User account to use to log in to the Model Repository Service. |
Model Repository Service Password | Password for the Model Repository Service user account. |
Modify Repository Password | Select the checkbox to modify the Model Repository Service user password. |
Data Preparation Service Options
The Data Preparation Service is an application service that manages data preparation within the Intelligent Data Lake application. When you create the Intelligent Data Lake Service, you must associate it with a Data Preparation Service using the Data Preparation Service options.
To edit the Data Preparation Service options, click the pencil icon. In the Edit Data Preparation Service Optionswindow, edit the required fields.
The following table describes the Data Preparation Service options:
Property | Description |
|---|
Data Preparation Service | Name of the Data Preparation Service associated with the Intelligent Data Lake Service. |
Data Integration Service Options
The Data Integration Service is an application service that performs data integration tasks for Intelligent Data Lake. When you create the Intelligent Data Lake Service, you must associate it with a Data Integration Service using the Data Integration Service options.
To edit the Data Integration Service options, click the pencil icon. In the Edit Data Integration Service Options window, edit the required fields.
The following table describes the Data Integration Service options:
Property | Description |
|---|
Data Integration Service | Name of the Data Integration Service associated with the Intelligent Data Lake Service. |
Catalog Service Options
The catalog represents an indexed inventory of all the configured assets in an enterprise. You can find metadata and statistical information, such as profile statistics, data asset ratings, data domains, and data relationships, in the catalog. The catalog options will be based on the Catalog Service configuration you have set up when you installed Informatica Live Data Map.
To edit the catalog service options, click the pencil icon in the Catalog Service Options area. In the Edit Catalog Service Options window, edit the required fields.
The following table describes the catalog service options:
Property | Description |
|---|
Catalog Service | Name of the Catalog Service associated with the Intelligent Data Lake Service. |
Catalog Service User Name | User account to use to log in to the Catalog Service. |
Catalog Service User Password | Password for the Catalog Service user account. |
Modify Catalog Service User Password | Select this checkbox to modify the Catalog Service user password. |
Data Lake Resource Name | Hive resource for the data lake. You configure the resource in Live Data Map Administrator. |
Data Lake Security Options
Select the security mode and specify the related details using the Data Lake Security Options.
To edit the data lake security options, click the pencil icon in the Data Lake Security Options area. In the Edit Data Lake Security Options window, edit the required fields.
The following table describes the data lake security options:
Property | Description |
|---|
Hadoop Authentication Mode | Security mode of the Hadoop cluster for the data lake. If the Hadoop cluster uses Kerberos authentication, you must set the required Hadoop security properties for the cluster. |
HDFS Service Principal Name | Service principal name (SPN) of the data lake Hadoop cluster. |
Principal Name for User Impersonation | Service principal name (SPN) of the user account to impersonate when connecting to the data lake Hadoop cluster. The user account for impersonation must be set in the core-site.xml file. |
SPN Keytab File for User Impersonation | Path and file name of the SPN keytab file for the user account to impersonate when connecting to the Hadoop cluster. The keytab file must be in a directory on the machine where the Intelligent Data Lake Service runs. |
HBase Master Service Principal Name | Service principal name (SPN) of the HBase Master Service. Use the value set in this file: /etc/hbase/conf/hbase-site.xml. |
HBase RegionServer Service Principal Name | Service principal name (SPN) of the HBase Region Server service. Use the value set in this file: /etc/hbase/conf/hbase-site.xml. |
HBase User Name | User name with permissions to access the HBase database. |
Data Lake Options
The data lake options include the Hive, HDFS, and Hadoop configuration details.
To edit the data lake options, click the pencil icon in the Data Lake Options area. In the Edit Data Lake Options window, edit the required fields.
The following table describes the data lake options:
Property | Description |
|---|
HDFS Connection | HDFS connection for the data lake. |
HDFS Working Directory | HDFS directory where the Intelligent Data Lake Service copies temporary data and files necessary for the service to run. |
Hadoop Distribution Directory | Directory that contains Hadoop distribution files on the machine where Intelligent Data Lake Service runs. The directory must be within the Informatica directory. The default directory is <Informatica installation directory>/services/shared/hadoop/<hadoop distribution name>. |
Hive Connection | Hive connection for the data lake. |
Hive Table Storage Format | Data storage format for the Hive tables. Select from the following options: - - DefaultFormat
- - Parquet
- - ORC
|
Export Options
After the project is published to the data lake, you can export the data to a .csv file and save it on the local drive.
To edit the export options, click the pencil icon in the Export Options area. In the Edit Export Options window, edit the required fields.
The following table describes the export options:
Property | Description |
|---|
Number of Rows to Export | Number of rows to export to a .csv file. You can specify a maximum of 2,000,000,000 rows. Enter a value of -1 to export all rows. |
Data Asset Recommendation Options
The recommendation options are used to define the number of recommended data assets that can be displayed on the Projects page in the Intelligent Data Lake application.
To edit the data asset recommendation options, click the pencil icon in the Data Asset Recommendation Options area. In the Edit Data Asset Recommendation Options window, edit the required fields.
The following table describes the data asset recommendation options:
Property | Description |
|---|
Number of Recommendations to Display | The number of recommended data assets to display on the Projects page. You can specify a maximum of 50 recommendations. A value of 0 means no recommendations will be displayed. You can use recommended alternate or additional data assets to improve productivity. |
Sampling Options
You can specify the sample size to be retrieved in the Intelligent Data Lake application in the Intelligent Data Lake Service properties.
To edit the sampling options, click the pencil icon in the Sampling Options area. In the Edit Sampling Options window, edit the required fields.
The following table describes the sampling options:
Property | Description |
|---|
Data Preparation Sample Size | Number of sample rows to fetch for data preparation. You can specify a maximum number of 1,000,000 rows and a minimum of 1,000 rows. |
Logging Options
Logging options include properties for the severity level for service logs. Configure the Log Severity property to set the logging level.
To edit the logging options, click the pencil icon in the Logging Options area. In the Edit Logging Options window, edit the required fields.
The following table describes the logging options:
Property | Description |
|---|
Log Severity | Severity of messages to include in the logs. Select from one of the following values: - - FATAL. Writes FATAL messages to the log. FATAL messages include nonrecoverable system failures that cause the service to shut down or become unavailable.
- - ERROR. Writes FATAL and ERROR code messages to the log. ERROR messages include connection failures, failures to save or retrieve metadata, service errors.
- - WARNING. Writes FATAL, WARNING, and ERROR messages to the log. WARNING errors include recoverable system failures or warnings.
- - INFO. Writes FATAL, INFO, WARNING, and ERROR messages to the log. INFO messages include system and service change messages.
- - TRACE. Write FATAL, TRACE, INFO, WARNING, and ERROR code messages to the log. TRACE messages log user request failures.
- - DEBUG. Write FATAL, DEBUG, TRACE, INFO, WARNING, and ERROR messages to the log. DEBUG messages are user request logs.
|
Custom Options
Configure custom properties that are unique to specific environments. You might need to apply custom properties in special cases.
When you define a custom property, enter the property name and an initial value. Define custom properties only at the request of Informatica Global Customer Support.
To view the custom options, select the service in the Domain Navigator and click the Properties view. You can change the properties while the service is running, but you must restart the service for the properties to take effect.
To edit the custom options, click the pencil icon in the Custom Options area. In the Edit Custom Options window, edit the required fields.