Transformations > Python transformation > Install and configure Python
  

Install and configure Python

Install and configure your Python code to run on a Secure Agent machine. Configure tasks such as setting the advanced session property and installing the Python packages.
Note: You can't create a custom Python installation when you use a serverless runtime environment, so these steps don't apply.
To install and configure Python, complete the following tasks:
  1. 1To allow the Python transformation to use third-party libraries, make sure that the runtime environment has access to an installation of Python and the referenced resource files.
  2. 2Install the prerequisite packages on the Secure Agent machine.
  3. 3Download the version of Python that you want to install on the Secure Agent machine and extract the files.
  4. 4Prepare the installation directory.
  5. 5Compile and build the installation directory.
  6. 6Set environment variables to allow the Secure Agent machine access to the installation.
  7. 7Verify the installation contains the correct folders.
  8. 8For custom installations, install the str2bool library.
  9. 9If your installation uses third-party libraries, install them.
  10. 10Copy the installation to the Secure Agent machine.
  11. 11If your installation uses third-party libraries, verify their installation.

Step 1. Enable access to third-party libraries

To use the Python transformation with third-party libraries, the runtime environment must have access to an installation of Python and the resource files that you reference in the Python code.
When you install Python, you can include any third-party libraries you want to access in the Python code.
In the mapping task, use an advanced session property to specify the locations of the Python installation directory and the Python executable:
  1. 1In the mapping task advanced session properties, click Add.
  2. 2Select advanced.custom.property as the session property name.
  3. 3For the session property value, enter the following text:
    4. infaspark.pythontx.executorEnv.PYTHONHOME=$INFA_HOME/python&:infaspark.pythontx.exec=$INFA_HOME/python/bin/python3
  4. 4Click Finish.

Step 2. Install prerequisite packages

Before you install Python, make sure that the Secure Agent machine has the required packages.
The following table lists the commands to install a package on the Secure Agent machine:
Package
Command
OpenSSL
sudo yum install openssl-devel
zlib
sudo yum install zlib-devel
libffi
sudo yum install libffi-devel

Step 3. Download the Python distribution

Find the version of Python that you want to install on the Secure Agent machine and extract the files.
  1. 1Navigate to the home directory.
  2. 2Download the Python distribution onto the Secure Agent machine. For example, you can run the following command to get an installation of Python 3.6.5 from the internet:
    3. wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz
    This command downloads the Python distribution in the home directory.
  3. 3To untar the file, run the following command :
    4. tar -xvf Python-3.6.5.tgz
    The extracted files appear in the directory ~/Python-3.6.5.

Step 4. Prepare a directory to install Python

Specify a location to contain the Python installation. To prepare an installation directory, run commands to create a directory and set it as the working directory.
  1. 1To create a Python3 directory under the home directory, run the following command:
    2. mkdir Python3
  2. 2To create an environment variable called $workingdir and set the value to the present working directory, run the following command :
    3. export workingdir=`pwd`
  3. 3To verify the $workingdir environment variable, run the following command:
    4. echo $workingdir

Step 5. Install Python

Run commands to compile and build the Python installation.
  1. 1Navigate to the ~/Python-3.6.5 directory where you extracted the Python distribution:
    2. cd Python-3.6.5
  2. 2To compile Python under $workingdir/Python3, run the following command:
    3. ./configure --enable-shared --prefix=$workingdir/Python3
  3. 3To build the Python installation, run the following command:
    4. make
    make install

Step 6. Set environment variables

Configure the Secure Agent machine to access the Python installation.
  1. 1To set the PYTHONHOME environment variable, run the following command:
    2. export PYTHONHOME=$workingdir/Python3
    Notice that $PYTHONHOME is the directory where we installed Python.
  2. 2To set the LD_LIBRARY_PATH environment variable, run the following command:
    3. export LD_LIBRARY_PATH=$PYTHONHOME/lib
  3. 3To set the PATH environment variable, run the following command:
    4. export PATH=$PYTHONHOME/bin:$PATH

Step 7. Verify the Python installation

Confirm the location of the installation folders.
Verify that the Python installation directory contains the correct folders:
  1. 1To change the directory to $PYTHONHOME, run the following command:
    2. cd $PYTHONHOME
  2. 2To verify that $PYTHONHOME contains the following folders, run the ls command :
    3. bin
    include
    lib
    share

Step 8. Install str2bool library

Custom Python installations require the str2bool library.
Use the following pip command to install the str2bool library in $PYTHONHOME:
bin/pip3 install str2bool

Step 9. Optionally, install third-party libraries

Identify and install any additional libraries that you need in your Python installation.
If your Python installation uses third-party libraries, use the pip command to install them in $PYTHONHOME.
For example, you can run the following commands to install numpy and scikit-learn:
bin/pip3 install numpy
bin/pip3 install scikit-learn

Step 10. Copy Python to the Secure Agent machine

Copy the Python installation to the Secure Agent machine.
To complete the Python installation, run the following commands:
  1. 1Create the following directory on the Secure Agent machine if it doesn't exist:
    2. <Secure Agent installation directory>/ext/python/
  2. 2To copy the Python installation to the Secure Agent machine, run the following command:
    3. cp -r $PYTHONHOME/* <Secure Agent installation directory>/ext/python/
  3. 3To check the contents of the Informatica domain, run the following command:
    4. ll <Secure Agent installation directory>/ext/python/
  4. 4Verify that the contents include the following folders:
    5. bin
    include
    lib
    share

Step 11. Optionally, verify third-party library installations

If your Python installation uses third-party libraries, make sure they are installed on the Secure Agent machine.
  1. 1To change directories to the Secure Agent machine, run the following command:
    2. cd <Secure Agent installation directory>/ext/python/
  2. 2To invoke the Python interpreter, run the following command:
    3. bin/python3
  3. 3Run the following Python code:
    4. import numpy
    import sklearn
  4. 4Verify that the import statement is successful for each Python third-party library that you installed.
    5. Press Ctrl+D to exit the Python interpreter.