1. Documentation
2. 10.3 (Latest) 10.2 10.1 10.0 9.10 9.9 9.8 9.7 9.6 9.5 9.4 9.3 9.2 9.1 9.0 8.2 8.1 8.0 7.6
3. Hub
4. Releases
5. 10.3.1

What's New in RapidMiner AI Hub 10.3.1?

Released: 11th January 2024

The following describes the enhancements and bug fixes in AI Hub 10.3.1:

AI Hub

The following describes the enhancements and bug fixes in AI Hub.

Enhancements

• Improved messaging around Queued Extensions
• Improved temporary file management
• Increased Job Agent's max memory usage by setting Xmx JVM option to 256M
• Provided sorters for date fields in the job list dashboard
• Disabled the Gateway retry mechanics by default to improve throughput
• Logging of stderr and stdout can be enabled for the Job Agent and thus Job Containers with
  • JOBAGENT_PLATFORM_STD_REDIRECT_ERROR_ENABLED=true
  • JOBAGENT_PLATFORM_STD_REDIRECT_OUT_ENABLED=true
  • This redirects all logs produced, including all stdout and stderr also invoked by 3rd party class loaders including all RapidMiner extensions
  • The default path for those logs is home/log/
• Logging of stderr and stdout can be enabled for Scoring Agent and Web API Agents
  • SCORING_AGENT_STD_REDIRECT_ERROR_ENABLED=true
  • SCORING_AGENT_STD_REDIRECT_OUT_ENABLED=true
  • This redirects all logs produced, including all stdout and stderr also invoked by 3rd party class loaders including all RapidMiner extensions
  • The default path for those logs is home/log/
• Switched default file size for incoming requests to 25G for AI Hub Server, Scoring Agent and Web API Agent
• Switched default value for WEBAPI_REST_CLIENT_READ_TIMEOUT from 120000 to 300000
• Switched default value for WEBAPI_REST_CLIENT_CONNECTION_TIMEOUT from 120000 to 300000
• Bumped License Agent version from 1.0.5 to 1.0.7
• Bumped License Core version from 4.5.2 to 4.5.3
• Bumped Studio Core version from 10.3.0 to 10.3.1
• Upgraded all frontend dependencies and underlying build system to Node 18, npm9 and Vite
• Moved from Gradle 6 to Gradle 8 as underlying build manager for backend applications

Fixed

• Fixed standalone user with aihub:admin role not being able to see Create Schedule button
• Fixed Create Schedule auto-refresh not being visible for all users
• Fixed ref not provided during context lookup for Endpoints
• Fixed distribution of (native) Scoring Agent referencing wrong OS libraries for Altair Unit licensing
• Fixed distribution of (native) Scoring Agent missing proper JXBrowser OS libraries
• Fixed cleanup of global temporary directory in AI Hub
• Fixed cleanup of Altair Unit temporary license location directory

License Proxy

The following describes the enhancements and bug fixes in License Proxy.

Enhancements

• Increased log level for handling heartbeats

Deployment fixes and enhancements

The following describes the enhancements and bug fixes in AI Hub 10.3.1:

• Implemented Healthchecks in the docker-compose file for the components. For more information about healthchecks click here
• To optimize large language model caching a new cache volume was added to Job Agent
• Added optional SMTP setting for AiHub Backend
• Raised the upload limit to 25Gb to support large language models and applied that limit for aihub-backend repository
• Added the timezone setting (TZ environment variable) to all the containers
• Added Prometheus metrics export for aihub-backend, aihub-job-agent, webapi-agents, scoring-agent, panopticon-vizapp, panopticon-vizapp-python and panopticon-rserve
• Improved support for running the deployment on a custom port
• Added settings to docker templates to support CPU usage limitation
• Refactored Panopticon configuration, licensing and storage
• Panopticon container were upgraded to 2023.2 Panopticon release
• Fixed Panopticon python integration and changed to use the fast_api python backend
• Fixed Panopticon license persistance
• Panopticon Vizapp default memory and cpu limit increased in the kubernetes templates

Release notes

• The docker compose version has been changed from 3.9 to 2.4 in the docker templates
• Panopticon Vizapp default volume layout has been changed, the upgrade may require data migration
• Please note, that Altair Unit licensing with Altair One for Panopticon has limited support in kubernetes. At the moment it can be used only with credentials auth method. Auth code based authentication requires a fixed MAC address for the vizapp container, that is not supported in many kubernetes CNI plugins.

How to upgrade from the previous docker installation

Download the 10.3.1 version of the docker compose templates

To download the helm chart please click here.

You can replace the docker-compose.yml file if you haven't customized it before.

Edit your .env file

Generic changes

Your custom settings are ment to be kept in the .env file. For 10.3.1 the following changes shall be made compared to the previous version.

All 10.3.0 versions shall be replaced to 10.3.1 in the .env file.

The default upload limit is changed from 10GB to 25GB to better support large language model uploads. Please change the following value:

PROXY_DATA_UPLOAD_LIMIT=25GB

You can upgrade the Grafana container by changing the following value from 10.1.4-ubuntu

OFFICIAL_GRAFANA_IMAGE=grafana/grafana:10.1.6-ubuntu

CPUSET configuration

As a new feature you can limit the available CPUs for following containers: * aihub-backend * aihub-job-agent * webapi-agent * scoring-agent * panopticon-vizapp * panopticon-vizapp-python * panopticon-rserve

Altair Unit license consumption is calculated based on the number available CPUs in the container.

Using the CPUSET variables you can set on which CPU cores the particuar container is allowed to be executed.

The following sample sets:

• aihub-backend to be executed on CPU core #0
• aihub-job-agent to be executed on CPU cores #2 and #4
• webapi-agent-1 to be executed on CPU core #6
• webapi-agent-1 to be executed on CPU core #8
• scoring-agent to be executed on CPU cores #10 #11 #12 and #13
• panopticon-vizapp to be executed on CPU core #14
• panopticon-vizapp-python to be executed on CPU core #16
• panopticon-rserve to be executed on CPU core #18

AIHUB_BACKEND_CPUSET=0
JOBAGENT_CPUSET=2,4
WEBAPI_AGENT_CPUSET_1=6
WEBAPI_AGENT_CPUSET_2=8
SCORING_AGENT_CPUSET=10-13
PANOPTICON_VIZAPP_CPUSET=14
PANOPTICON_PYTHON_CPUSET=16
PANOPTICON_RSERVE_CPUSET=18

Please read the following pages carefully

https://docs.docker.com/engine/reference/run/#cpuset-constraint

https://docs.docker.com/config/containers/resource_constraints/#configure-the-default-cfs-scheduler

Please note, that by default these settings are commented in the docker-compose.yml

After you set the values here, please uncomment the relevant properties in the docker-compose.yml as well.

Prometheus metrics

Prometheus metrics exports are turned on by default for several components. This can be disabled if you remove the prometheus from the following variables:

AIHUB_BACKEND_PROFILES_ACTIVE=default,prometheus
JOBAGENT_SPRING_PROFILES_ACTIVE=default,prometheus
WEBAPI_GATEWAY_PROFILES_ACTIVE=default,prometheus
WEBAPI_AGENT_SPRING_PROFILES_ACTIVE=webapi,prometheus

Email settings for Ai Hub Backend

You can set up email sending related configuration with the following variables if required

# SMTP settings
# These settings are commented in docker-compose.yml
SPRING_MAIL_HOST=mx.example.com
SPRING_MAIL_PORT=587
SPRING_MAIL_PROPERTIES_MAIL_SMTP_AUTH=true
SPRING_MAIL_PROPERTIES_MAIL_SMTP_STARTTTLS_ENABLE=true
LOGIN-USER-TO-SMTP-SERVER
SPRING_MAIL_USERNAME=user1@example.com
LOGIN-PASSWORD-TO-SMTP-SERVER
SPRING_MAIL_PASSWORD=yout-smtp-password
# Optional parameters
# Address,where reports is send to
REPORTING_ERROR_MAIL_TO=rapidminer-deployment@@example.com
# Email subject
REPORTING_ERROR_MAIL_SUBJECT_PREFIX="Message from RapidMiner deployment"
# Logical sender address
REPORTING_ERROR_MAIL_FROM_ADDRESS=rapidminer-deployment@@example.com
# Logical sender name
REPORTING_ERROR_MAIL_FROM_NAME="RapidMiner deployment"

Panopticon related changes

A new variable was introduced to set the Panopticon upload limit independently from the proxy:

PANOPTICON_FILE_UPLOAD_SIZE_MAX_BYTES=30000000

Altair Unit based licensing is bond to a non-changing MAC address for Panopticon.

The value can be a random-generated one, but some MAC address ranges are restricted by the standard.

You can generate a random MAC address with the following command:

cat /dev/urandom | tr -d -c '[:digit:]A-F' | fold -w 12 | sed -E -n -e '/^.[26AE]/s/(..)/\1-/gp' | sed -e 's/-$//g' |sed 's/-/:/g'|head -n10

Then the following placeholder can be replaced with the generated values:

PANOPTICON_VIZAPP_CONTAINER_MAC_ADDRESS="<PANOPTICON-MAC-ADDRESS-PLACEHOLDER>"

Deploy the new version

docker compose up -d

How to upgrade from the previous kubernetes installation

Download the 10.3.1 version of the helm chart

To download the helm chart please click here.

Edit custom-values.yaml

Generic changes

If you set the following versions, they should be changed.

Please note, that you can override the versions per component.

mainVersion: "10.3.1"
cesVersion: "10.3.1"

Please note, that the proxy will respond on the httpPort with a redirect to https once configured with https.

If you set a custom value for dataUploadLimit before and would like to support large language models, we suggest to raise the limit to 25Gb.

dataUploadLimit: "25GB"

If you customized the jobagent.configStorageClass, that shall be renamed to jobagent.storageClass.

A new volume was added to the chart to cache large language models. You can provide a custom name for it using the jobagent.huggingfacePvcName property if required.

If you provided grafana.staticVersion, it's value shall be changet from 0.1.4-ubuntu to 10.1.6-ubuntu.

Prometheus metrics

Prometheus metrics exports are turned on by default for several components. It can be disabled with the springProfilesActive variables at the components section.

Email settings for Ai Hub Backend

You can set up email sending related configuration with the following variables if required

smtpEnabled: True
smtpHost: "mx.example.com"
smtpPort: "587"
smtpUserName: "user1@example.com"
smtpPassword: "yout-smtp-password"
smtpAuth: "true"
smtpStartTLS: "true"
reportErrMailTo: "rapidminer-deployment@@example.com"
reportErrMailSubject: "Message from RapidMiner deployment"
reportErrMailFromAddress: "rapidminer-deployment@example.com"
reportErrMailFromName: "RapidMiner deployment"

Panopticon related changes

You can set a custom upload limit for Panopticon vizapp using the panopticonVizapp.file.upload.size.max.bytes property.

You can set the logging level for Panopticon vizapp using the logger.level.file proporty.

Please add panopticonVizapp.license.detached with true value if you would like to set licensing for Panopticon independently from AiHub.

The Panopticon related volume were refactored in the chart. You can provide a custom names for them using the following properties.

panopticonVizapp:
  appDataPvcName: "panopticon-vizapp-appdata-pvc"
  appDataPvcWtorageClass: "<STORAGECLASS-PLACEHOLDER_RWO>"
  appDataPvcAccessMode: "ReadWriteOnce"
  appDataPvcDiskSize: "4Gi"
  sharedPvcName: "panopticon-vizapp-shared-pvc"
  sharedPvcStorageClass: "<STORAGECLASS-PLACEHOLDER_RWO>"
  sharedPvcAccessMode: "ReadWriteOnce"
  sharedPvcDiskSize: 1Gi
  logsPvcName: "panopticon-vizapp-logs-pvc"
  logsPvcStorageClass: "<STORAGECLASS-PLACEHOLDER_RWO>"
  logsPvcAccessMode: "ReadWriteOnce"
  logsPvcDiskSize: 4Gi
  licensePvcName: "panopticon-vizapp-license-pvc"
  licensePvcStorageClass: "<STORAGECLASS-PLACEHOLDER_RWO>"
  licensePvcAccessMode: "ReadWriteOnce"
  licensePvcDiskSize: 100Mi

Deploy the new version

helm upgrade -n <NAMESPACE-PLACEHOLDER> --install rapidminer-aihub --values custom-values.yaml ./rapidminer-aihub-10.3.1.tgz