Execute Python (Python Scripting)

Synopsis

Executes a Python script.

Description

Before using this operator you may need to specify the path to your Python installation under Settings -> Preferences menu (on Mac OS choose RapidMiner Studio -> Preferences). In the appearing settings panel select the Python Scripting tab. Your Python installation must include the pandas module since example sets get converted to pandas.DataFrames. By unchecking the use default python checkbox you can configure an individual Python binary for this operator instead of using the global settings.

This operator executes either the script provided through the script file port or parameter or the script specified in the script parameter. The arguments of the script correspond to the input ports, where example sets are converted to pandas.DataFrames. Analogously, the values returned by the script are delivered at the output ports of the operator, where pandas.DataFrames are converted to example sets.

The operator supports conda (anaconda) virtual environments, virtualenvwrapper virtual environments and you can select a Python binary, by specifying the full file system path to it as well. For more information on how to select the required Python, see the Parameters section of this help page. Note, that you may need to configure the extension. For this go to Settings -> Preferences menu (on Mac OS choose RapidMiner Studio -> Preferences). In the appearing settings panel select the Python Scripting tab. Edit the settings here, if required.

Using conda: if you installed the conda Python distribution to a non default location, you may need to add the installation directory and some subdirectories in the global settings of the Python Scripting Extension. For this go to Settings -> Preferences menu (on Mac OS choose RapidMiner Studio -> Preferences). In the appearing settings panel select the Python Scripting tab. Add the installation directory of your conda installation to the list of search paths. On Windows you need to add the conda_install_dir\Scripts subdirectory and on Linux and Mac OS the conda_install_dir/bin subdirectory as well.

Accessing macros: you can access and modify the macros defined in RapidMiner from the Python code. You can call a macro by enclosing the name of the macro inside the %{} marks. Before interpreting the Python code, these values will be substituted with actual macro values. For a more fine grained control over macros, set the use macros parameter. For more information see the parameter description below.

The console output of Python is shown in the Log View (View -> Show View -> Log).

Input

script file (File)
A file containing a python script to be executed. The file has to comply with the script parameter rules. This port is optional, a file can also be provided through the script file parameter.
input
The Script operator can have multiple inputs. An input must be either an example set, a file object or a Python object which was generated by an 'Execute Python' operator.

Output

output
The Script operator can have multiple outputs. An output can be either an example set, a file object or a Python object generated by this operator.

Parameters

script
The Python script to execute. Define a method with name 'rm_main' with as many arguments as connected input ports or alternatively a *args argument to use a dynamic number of attributes. The return values of the method 'rm_main' are delivered to the connected output ports. If the method returns a tuple then the single entries of the tuple are delivered to the output ports. Entries from the data type 'pandas.DataFrames' are converted to example sets; files are converted to File Objects, other Python objects are serialized and can be used by other 'Execute Python' operators or stored in the repository. Serialized Python objects have to be smaller than 2 GB.

If you pass an example set to your script through an input port, the meta data of the example set (types and roles) is available in the script. You can access it by reading the attribute rm_metadata of the associated pandas.DataFrame, in our example data. data.rm_metadata is a dictionary from attribute names to a tuple of attribute type and attribute role.

You can influence the meta data of an example set that you return as a pandas.DataFrame by setting the attribute rm_metadata. If you don't specify attribute types in this dictionary, they will be determined using the data types in Python. You can specify your own roles or use the standard roles of RapidMiner like 'label'.

For more information about the meta data handling in a Python operator check the tutorial process 'Meta data handling' below.

If a script file is provided either through the script file port or parameter (port takes precedence), that script will be used instead of the value of this parameter.
Range: text
script_file A file containing a python script to be executed. The file has to comply with the script parameter rules. This parameter is optional. Range: filename
use_default_python
Use the Python binary or environment defined in the RapidMiner Studio global settings. The global settings can be accessed from the Settings -> Preferences menu (on Mac OS choose RapidMiner Studio -> Preferences). In the appearing settings panel select the Python Scripting tab. Here you can define the defaults.
Range: boolean
package_manager
This parameter only available if use default python is set to false. This parameter specifies the package manager used by the operator. Currently Conda/Anaconda/Miniconda and Virtualenvwrapper is supported, or you can define the full path to your preferred python binary as well.
Range: selection
conda_environment
This parameter only available if use default python is set to false and package manager is set to conda (anaconda). This parameter specifies the conda virtual environment used by this operator.
Range: selection
venvw_environment
This parameter only available if use default python is set to false and package manager is set to virtualenvwrapper. This parameter specifies the virtualenvwrapper virtual environment used by this operator.
Range: selection
python_binary
This parameter only available if use default python is set to false and package manager is set to specific python binaries .This parameter specifies the path to the python binary, used by this operator.
Range: string
use_macros
Use an additional named parameter macros for the rm_main method (NOTE, that you will need to modify the script and add the parameter manually). This way all the macro values will be passed as an additional parameter of the rm_main method and you can access the macro values via the macros dictionary. Each dictionary value will be a Python string. You can also modify values of the dictionary or add new elements. The changes will be reflected in RapidMiner after the execution of the operator.
Range: boolean

Tutorial Processes

Clustering using Python

Random data is generated and then fed to the Python script. The script clusters the data in Python using as many clusters as are specified in the macro. The resulting ExampleSet contains the cluster in the 'cluster' attribute.

Building a model and applying it using Python

This tutorial process uses the 'Execute Python' operators to first build a decision tree model using the 'Deals' data and then applying it to the 'Deals Testset' data. Before using the data, it the nominal values are converted to unique integers. The first Python scripting operator 'build model' builds the model and delivers it to its output port. The second Python scripting operator 'apply model' applies this model to the testset, adding a column called prediction. After specifying the 'label' and 'prediction' columns with 'Set Role', the result can be viewed.

Creating a plot using Python and storing it in your repository

This tutorial process uses the 'Execute Python' operator to first fetch example data, then create a plot and return both to the output ports. Please store the process in your repository. The data are shown as example set and the plot is stored in the repository as image.

Reading an example set from a file using Python

This tutorial process uses the 'Execute Python' operator to save example data in a csv file. The second 'Execute Python' operator receives this file, reads the data and returns a part of the data to the output port. The result is an example set.

Meta data handling

This tutorial process shows how to access the meta data of incoming example sets inside a 'Execute Python' operator. It also explains how to set the meta data for the outcoming example sets.

Categories

Versions