Purpose
This article describes the technical specifications that have been tested for running Giotto and summarizes the minimum infrastructure requirements for a supported deployment.
Overview
Giotto runs entirely inside Docker containers. Because of this, it is generally resilient to differences in the host operating system, as long as the required container, GPU, and networking prerequisites are correctly installed.
The tested configurations described here should be understood as validated environments, not hard limits. In practice, Giotto is likely to work on additional operating systems and newer CUDA versions as well, provided that the NVIDIA driver, CUDA version, and container runtime remain compatible.
General deployment model
Giotto is deployed as a containerized platform.
This means that the host VM mainly needs to provide:
- a supported Linux environment
- Docker and Docker Compose
- NVIDIA drivers and CUDA support for GPU workloads
- NVIDIA Container Toolkit so containers can access the GPU
- sufficient disk, memory, CPU, and GPU resources
- correct internal networking and firewall configuration
Because the platform runs inside Docker, the host OS is less coupled to the application logic than in a non-containerized deployment.
Tested operating systems
Giotto has been tested on:
- Ubuntu 22.04
- Ubuntu 24.04
These are the operating systems currently validated in testing.
Note
Because the platform runs in Docker, it is likely that other Linux distributions may also work, but they have not been explicitly validated in the same way.
Tested CUDA versions
Giotto has been tested with CUDA versions in the range:
- CUDA 12.5
- CUDA 12.6
- CUDA 12.7
- CUDA 12.8
- CUDA 13.0
- CUDA 13.2
using the corresponding compatible NVIDIA driver versions.
Note
It is likely that newer CUDA versions will also work, provided that:
- the NVIDIA driver is compatible (from 555.xxx.xxx to 580.xxx.xxx)
- the GPU supports the selected version
- the container runtime is configured correctly
This should still be validated in your target environment before production use.
Tested GPUs
Giotto Solo
Giotto Solo is the smartest small model that runs on a single GPU. The following GPUs have been tested successfully with Giotto Solo:
- NVIDIA H100
- NVIDIA H200
- NVIDIA RTX PRO 6000
- NVIDIA B200
These are the GPU models currently validated in testing.
Note
Other NVIDIA GPUs may also work, especially if they are compatible with the tested CUDA and driver stack, but they are not part of the currently documented tested configurations.
We also tested on 4 x L4: the model could only withstand a 5k context windows and was too slow to be actually used. Hence, we discourage the use of such model for older accelerators.
Furthermore, the more rencent NVIDIA GPUs, like the B200 or the RTXPRO6000 have a Blackwell architecture and thus require CUDA > 12.8.
Giotto Alto
Giotto Alto is a larger model and it can only run on (and has been tested on) these GPU nodes:
8 x B300
16 x B200 only if connected via PCIe or Infiniband
Minimum infrastructure requirements
Giotto Solo
For a working deployment, the minimum tested resource envelope is:
- Disk: at least 200 GB
- RAM: at least 150 GB*
- CPU: at least 8 Cores
These should be treated as the minimum baseline for a usable deployment.
Giotto Alto
For a working deployment, the minimum tested resource envelope is:
- Disk: at least 3 TB
- RAM: at least 1.5 TB
- CPU: at least 64 Cores
These should be treated as the minimum baseline for a usable deployment.
Why these requirements matter
Disk
Giotto requires significant disk capacity because it may need to store:
- Docker images
- models
- datasets
- intermediate artifacts
- logs
- evaluation outputs
A smaller disk may fill up quickly in real workflows.
RAM
Giotto benefits from substantial RAM because the platform may run:
- model serving
- orchestration components
- monitoring tools
- logging infrastructure
- distributed compute services
Insufficient RAM can destabilize the deployment even before GPU memory becomes the bottleneck.
CPU
Cores are recommended to support:
- containerized services
- orchestration
- background platform components
- data preparation and system overhead
Even if the main model execution is GPU-heavy, the platform still needs CPU capacity for coordination and infrastructure services.
Required software prerequisites
In addition to the tested OS and hardware envelope, the following prerequisites should be considered required for a practical deployment:
Docker Engine
Giotto runs inside Docker containers, so Docker Engine must be installed and working.
Docker Compose plugin
Giotto deployment also depends on Docker Compose support.
NVIDIA drivers
GPU-enabled deployments require NVIDIA drivers compatible with the chosen CUDA version.
CUDA toolkit
The host must provide a validated CUDA environment for GPU workloads.
NVIDIA Container Toolkit
The NVIDIA Container Toolkit is required so Docker containers can access the host GPU.
Internal networking
If the deployment includes multiple nodes, the nodes must be able to communicate over the private network.
Firewall configuration
Firewall rules should restrict public exposure while still allowing:
- SSH
- reverse-proxy ports
- private node-to-node traffic
- Docker internal network traffic
Deployment credentials
Installation also requires the deployment credentials and version provided by Giotto.ai.
Optional: SMTP configuration
If workflows or tools are expected to send email, SMTP must be configured correctly.
What is likely to work beyond the tested configurations
Because Giotto is containerized, it is reasonable to expect that it may also work in environments beyond those explicitly tested.
That likely includes:
- newer CUDA versions
- other compatible NVIDIA drivers
- other recent Linux distributions
However, these should be treated as probable but not yet formally validated until tested in your environment.
Recommendations for production deployments
For production use, it is recommended to:
- stay close to the tested Ubuntu versions when possible
- use a CUDA/driver pair that is already validated
- provision more than the bare minimum disk, RAM, and CPU if you expect large workloads
- verify GPU access through Docker before installing Giotto
- validate networking and firewall rules before exposing the platform
In practice, production setups should aim for a margin above the minimum values rather than matching them exactly.
Comments
0 comments
Article is closed for comments.