Rules and guidelines for personal staging location
When you select the personal staging location as a staging environment, the data is first staged in a java temporary location and then copied to a personal staging location of the unity catalog. Both the staged files will be deleted after the task runs successfully.
However, to stage the data in a different directory, configure the DTM property -Djava.io.tmpdir=/my/dir/path in the JVM options in the system configuration settings of the Administrator service.
To enable data staging in a different directory, you should have read and write permission and enough disk space to stage the data in the directory.
When you specify a personal staging location in the Databricks Delta connection properties for staging, consider the following rules and guidelines:
•You can only specify unity enabled catalog in the SQL warehouse JDBC URL.
•All mappings that are configured run without SQL ELT optimization.
•The data is staged in the folder stage://tmp/<user_name> where the <user_name> is picked from the Databricks token provided in the connection and requires read and write access to the personal staging location in root location of AWS and Azure.