Microsoft Fabric OneLake Connector > Mappings for Microsoft Fabric OneLake > Microsoft Fabric OneLake sources in mappings
  

Microsoft Fabric OneLake sources in mappings

In a mapping, you can configure a Source transformation to represent a Microsoft Fabric OneLake object.
The following table describes the Microsoft Fabric OneLake source properties that you can configure in a source transformation:
Property
Description
Connection
Name of the source connection. Select a source connection or click New Parameter to define a new parameter for the source connection.
If you want to overwrite the parameter at runtime, select the Allow parameter to be overridden at run time option when you create a parameter. When the task runs, the agent uses the parameters from the file that you specify in the task advanced session properties. Ensure that the parameter file is in the correct format.
When you switch between a non-parameterized and a parameterized Microsoft Fabric OneLake connection, the advanced property values are retained.
Source Type
Select Single Object or Parameter.
Object
Name of the source object.
Ensure that the headers or file data does not contain special characters.
Parameter
Select an existing parameter for the source object or click New Parameter to define a new parameter for the source object. The Parameter property appears only if you select Parameter as the source type.
When you parameterize the source object, specify the complete object path including the file system in the default value of the parameter.
If you want to overwrite the parameter at runtime, select the Allow parameter to be overridden at run time option when you create a parameter. When the task runs, the agent uses the parameters from the file that you specify in the task advanced session properties. Ensure that the parameter file is in the correct format.
Format
The format of the file that you want to read from Microsoft Fabric OneLake.
Select one of the following file formats:
  • - Avro
  • - JSON
  • - ORC
  • - Flat
  • - Parquet
  • - Discover Structure1
  • - Document to read PDF files1
Default is None. If you select None as the format type, Microsoft Fabric OneLake Connector reads data from Microsoft Fabric OneLake files in binary format.
For more information, see File formatting options.
Input Type
Determines the type of data that you can read when you select the Document file format.
You can only read from PDF files with document file format.
Intelligent Structure Model1
Determines the underlying patterns in a sample file and autogenerates a model for files with the same data and structure.
This property applies to the discover structure format type.
You can either select an existing model to associate with the Source transformation, or create a new model.
For more information about intelligent structure models, see Components in the Data Integration documentation.
1Applies only to mappings in advanced mode.
The following table describes the Microsoft Fabric OneLake source advance properties:
Property
Description
Workspace Override
Overrides the workspace specified in the connection.
Allow Wildcard Characters
Indicates whether you want to use wildcard characters for the directory source type.
If you select this option, you can use the question mark (?) and asterisk (*) wildcard characters to define one or more characters in a search.
For more information, see Wildcard characters.
Lakehouse Override
Overrides the lakehouse specified in the connection.
File Name Override
Overrides the source object at run time.
Source Type
The type of source from which you want to read data.
Select the following source types:
  • - File
  • - Directory
  • - Directory - Recursive
Select Directory - Recursive to recursively read from flat and complex files.
Default is File.
Compression Format
Reads compressed data from the source.
Select one of the following options:
  • - None. To read uncompressed and Parquet files.
  • - Gzip. To read flat files and Parquet files that use Gzip compression. The compressed files must have the .gz extension.
You cannot preview data for a compressed flat file.
Tracing Level
Sets the amount of detail that appears in the log file. You can choose terse, normal, verbose initialization or verbose data. Default is normal.

Directory source in Microsoft Fabric OneLake sources

You can select the type of source from which you want to read data.
You can select the following type of sources from the Source Type option under the advanced source properties:
Use the following rules and guidelines to select Directory as the source type:

Wildcard characters

When you read an Avro, JSON, ORC, Parquet, flat file, or PDF, you can use the ? and * wildcard characters to define one or more characters in a search.
To use wildcard characters for the source file name, select the source type as Directory and enable the Allow Wildcard Characters option in the advanced source properties.
You can use the following wildcard characters:
? (Question mark)
The question mark character (?) allows one occurrence of any character. For example, if you enter the source file name as a?b.txt, the Secure Agent reads data from files with the following names:
* (Asterisk)
The asterisk mark character (*) allows zero or more than one occurrence of any character. If you enter the source file name as a*b.txt, the Secure Agent reads data from files with the following names:

Rules and guidelines for wildcard characters

Consider the following rules and guidelines when you use wildcard characters:
Mappings
Mappings in advanced mode

Reading files from subdirectories

You can read objects stored in subdirectories in Microsoft Fabric OneLake in mappings.
You can use recursive read for flat files and complex files in mappings. When you create a mapping in advanced mode, you cannot use recursive read for flat files.
To read objects stored in subdirectories, select the source type as Directory - Recursive in the advanced source properties.

Rules and guidelines for reading from subdirectories

Consider the following rules and guidelines when you read objects stored in subdirectories:
Mappings
Mappings in advanced mode