Synchronization of resources
Job Agents and Web API Agents used to execute processes require the same resources, e.g., custom Java libraries, extensions or context.
To enable correct execution of processes, they uses various external resources like extensions, custom Java libraries,
and RapidMiner Server licenses.
These resources are stored within the {homeDir}/resources/ folder of the Job Agent or Web API Agent.
Central resource management
Such eternal resources are centrally managed and automatically synchronized from the AI Hub Server instance agents are connected to.
For administrators, there's also a web interface showing the currently indexed resourced which are going to be synchronized.
Resource management
Centrally managed resources for Job Agents or Web API Agents are stored in the resource/ folder of
the AI Hub Server home folder.
Both AI Hub Server and all connected agents use the same set of resources.
To install a new or manage an existing centrally managed resources do the following:
- Select the resource type you want to update (Extensions, Custom libraries)
- From the table below locate the path of the resource type you want to update
- Update the resource type by adding or removing content from the selected folder
- Restart Altair AI Hub. All connected Job Agents and Web API Agents will automatically synchronize the new resource configuration from AI Hub Server.
| Type | Path |
|---|---|
| Extensions | <rapidminer-server-home>/resources/extensions/ |
| Python SDK Extensions | <rapidminer-server-home>/resources/python-sdk-extensions/ |
| Custom libraries | <rapidminer-server-home>/resources/libs/ |
| Licenses | In case licensing is set to RAPIDMINER, licenses are automatically synced to connected agents |
| Execution context | <rapidminer-server-home>/.RapidMiner/ |
Automatic synchronization
By default, all resources and the execution context are automatically synchronized from the AI Hub Server instance after the Job Agent or Web API Agent has been started. Also, the resources are synchronized after a restart of the AI Hub Server instance.
Connected agents download the resources to the type specific resource folder and overwrites all potential existing files. These type specific resource folders are:
| Type | Path |
|---|---|
| Extensions | <app-home>/resources/extensions/ |
| Python SDK Extensions | <app-home>/resources/python-sdk-extensions/ |
| Custom libraries | <app-home>/resources/libs/ |
| Licenses | <app-home>/resources/licenses/ |
| Execution context | <app-home>/config/rapidminer/.RapidMiner |
Individual resource management
Individual resources for Job Agent / Web API Agent which differ from the centrally managed resources, can be used by
setting {jobagent|scoring-agent}.sync.enabled = false (see settings table for these applications).
After a restart, it will only use the resources that are already available in the respective resource folders.
If you manage resources (existing ones or adding a new one), you need to:
- Shutdown the application
- Locate the path of the resource type you want to update
- Update the resource type by adding or removing content from the selected folder
- Restart the application
Resource types
Extensions
Extensions are provided from {homeDir}/resources/extensions/ directory.
Python SDK Extensions
Python SDK extensions are provided from {homeDir}/resources/python-sdk-extensions/ directory.
Please see the Python Extension Loader
and how to leverage Python SDK extensions in AI Hub components for more information.
Custom libraries
Custom libraries are Java libraries which you can be used in a process, for example within the
Execute Script operator. You can add these libraries to the {homeDir}/resources/libs/ folder and then they are
automatically available for execution.
Don't confuse custom libraries with extensions.
Licenses
An agent's licenses are installed in the {homeDir}/resources/licenses/ directory.
Execution context
The Execution context for each Job Container are copied over from
{homeDir}/config/rapidminer/.RapidMiner during Job Container start.