Rules and guidelines for personal staging location
When you select the personal staging location as a staging environment, the data is first staged in a java temporary location and then copied to a personal staging location of the unity catalog. Both the staged files will be deleted after the task runs successfully.
However, to stage the data in a different directory, configure the DTM property -Djava.io.tmpdir=/my/dir/path in the JVM options in the system configuration settings of the Administrator service.
To enable data staging in a different directory, you should have read and write permission and enough disk space to stage the data in the directory.
When you specify a personal staging location in the Databricks Delta connection properties for staging, consider the following rules and guidelines:
•You can only specify unity enabled catalog in the SQL warehouse JDBC URL.
•The data is staged in the folder stage://tmp/<user_name> where the <user_name> is picked from the Databricks token provided in the connection and requires read and write access to the personal staging location in root location of AWS and Azure.