Enterprise Data Preparation
This section describes new Enterprise Data Preparation features in version 10.4.0.
Data Lake Access Management
Effective in version 10.4.0, you can manage access to the data lake by assigning permissions on the Enterprise Data Catalog resources that represent Hive schemas and HDFS locations in the data lake to Enterprise Data Preparation users and groups of users.
When you grant permissions on specific schemas or locations to a user or group of users, the application displays only those schemas and locations to which a user has permissions when the user performs an import, publish, or upload operation.
For more information, see the Enterprise Data Preparation 10.4.0 Administrator Guide.
Microsoft Azure Data Lake Storage as a Data Source
Effective in version 10.4.0, you can use a Microsoft Azure Data Lake Storage Gen2 location as a data source.
Publish Files to the Data Lake
Effective in version 10.4.0, you can publish prepared data in a file to an HDFS location in the data lake.
When you publish data, you can select the file type to write the data to in the data lake. For example, if choose to publish data as a comma separated value file, the application writes the data to the data lake as a .csv file.
For more information, see the Enterprise Data Preparation 10.4.0 User Guide.
Upload Files to the Data Lake
Effective in version 10.4.0, you can upload several file formats from your local drive to the data lake.
You can choose from the following upload options:
- Upload a file directly to the data lake.
You can upload a comma delimited file, an Avro file, a JSON file, or a Parquet file in UTF-8 format directly from your local drive to the data lake, without previewing the data. You might choose this option if you want to upload a file without previewing the data.
- Allow CLAIRE to determine the file structure, and then upload the file to the data lake.
You can upload the data in a comma delimited file or a Microsoft Excel spreadsheet to the data lake. When you upload the file, Enterprise Data Preparation uses the CLAIRE embedded discovery engine to determine the structure of the file and display a preview of the data.
When you use this option to upload an Excel spreadsheet, the CLAIRE engine discovers the sheets and tables in the spreadsheet. You can select the sheet and table you want to preview.
Note: Uploading an Excel spreadsheet is a preview feature.
- Define the file structure, and then upload the file to the data lake.
You can upload the data in a comma delimited file from your local drive to the data lake. When you upload the file, you can preview the data, specify the structure of the file, and configure column attributes to meet your requirements. You might choose this option if you need to modify column attributes before you upload the file.
For more information, see the Enterprise Data Preparation 10.4.0 User Guide.