Microsoft Azure Data Lake Storage Gen2 Connector > Mappings for Microsoft Azure Data Lake Storage Gen2 > File formatting options
  

File formatting options

Select the format of the Microsoft Azure Data Lake Storage Gen2 file and configure the formatting options.
The following table describes the formatting options for Avro, Parquet, Delta, JSON, ORC, and delimited flat files:
Property
Description
Schema Source
The schema of the source file.
Select one of the following options to specify a schema:
  • - Read from data file. Imports the schema from a file in Microsoft Azure Data Lake Storage Gen2.
  • - Import from schema file. Imports the schema from a schema definition file in the agent machine.
Schema File
The schema definition file in the agent machine from where you want to upload the schema.
The following table describes the formatting options for flat files:
Property
Description
Flat File Type
The type of flat file.
Select one of the following options:
  • - Delimited. Reads a flat file that contains column delimiters.
  • - Fixed Width. Reads a flat file with fields that have a fixed length.
    • You must select the file format in the Fixed Width File Format option.
    If you do not have a fixed-width file format, click New > Components > Fixed Width File Format to create one.
Delimiter
Character used to separate columns of data in a delimited flat file. You can set values as comma, tab, colon, semicolon, or others.
You cannot set a tab as a delimiter directly in the Delimiter field. To set a tab as a delimiter, you must type the tab character in any text editor. Then, copy and paste the tab character in the Delimiter field.
EscapeChar
Character immediately preceding a column delimiter character embedded in an unquoted string, or immediately preceding the quote character in a quoted string data in a delimited flat file.
Qualifier
Quote character that defines the boundaries of data in a delimited flat file. You can set qualifier as single quote or double quote.
Qualifier Mode
Not applicable.
Code Page
Select the code page that the Secure Agent must use to read data from a delimited flat file.
Select UTF-8 for mappings.
Header Line Number
Specify the line number that you want to use as the header when you read data from a delimited flat file.
Specify the value as 0 or 1.
To read data from a file with no header, specify the value as 0.
First Data Row
Specify the line number from where you want the Secure Agent to read data in a delimited flat file. You must enter a value that is greater or equal to one.
To read data from the header, the value of the Header Line Number and the First Data Row fields should be the same. Default is 1.
Target Header
Not applicable.
Distribution Column
Not applicable.
Max Rows To Preview
Not applicable.
Row Delimiter
Not applicable.
The following table describes the formatting options for JSON files:
Property
Description
Data elements to sample
Not applicable.
Memory available to process data
Not applicable.
Read multiple-line JSON files
Not applicable.

Rules and guidelines for Delta file format

Consider the following rules and guidelines when you use the Delta file format to read from a Delta Lake:
Mappings