System Requirements

Hardware Requirements

CPU

• Minimum Cores: 8+ cores
• Recommended Cores: 16+ cores for CPU-based inference workloads
• Architecture: x64/amd64 (64-bit)

Memory

System RAM

Deployment	Minimum	Recommended	Notes
Standard	16GB	32GB	Baseline install; limited capacity for apps and tools
Production	32GB	64GB+	Production workloads
GPU Workloads	32GB	64GB+	System RAM alongside GPU vRAM

GPU Memory (vRAM)

• GPU Inference: 16GB+ vRAM required
• Recommended: 32GB+ vRAM for optimal GPU inference performance

GPU (Optional)

Kamiwaza supports multiple GPU and accelerator platforms:

Discrete GPUs:

• NVIDIA GPUs with compute capability 7.0+ (Linux)
• AMD GPUs via ROCm (Linux) — see Software Dependencies

In a namespaced install, which serves local models through tenant-mode inference, this release serves NVIDIA GPUs only: no ROCm engine image ships. Prepare GPU nodes as described in GPU Nodes.

Unified Memory Systems:

• NVIDIA DGX Spark - GB10 Grace Blackwell, 128GB unified memory
• AMD Ryzen AI Max+ 395 - "Strix Halo" platform, up to 128GB unified memory

See Special Considerations for detailed unified memory platform specifications.

Storage

The platform administrator supplies persistent storage before installation. Kamiwaza's volumes consume that storage; Kamiwaza does not install a storage driver. See Block storage in the prerequisites for your configuration: Kubernetes with Istio or OpenShift with GreyMatter.

Storage Performance

• Required: SSD (Solid State Drive)
• Preferred: NVMe SSD for optimal performance
• Minimum: SATA SSD
• Note: Model weights can be on a separate HDD but load times will increase significantly

Storage Capacity

Size each backing filesystem for the peak simultaneous use: downloaded and extracted artifacts, runtime images, model caches and staging, logs, and application volumes located on that filesystem. Replicated storage also needs its configured replica capacity and the provider's reserved free-space margin.

As a planning allowance, give each node that runs Kamiwaza 350 GB free on the filesystem that holds the container runtime's images, usually /var/lib. This covers images, caches, and headroom; it is not a guarantee that Kamiwaza's volumes fit. When the storage provider uses the same disk (for example, local-path or Longhorn), add Kamiwaza's volume requests, about 624 GiB by default (see Block storage for Kubernetes with Istio or OpenShift with GreyMatter), and the provider's replication and reserve. Size storage on separate disks separately.

Capacity Planning

Component	Minimum	Recommended	Notes
Operating System	20GB	50GB	Ubuntu/RHEL base + dependencies
Kamiwaza	50GB	50GB	Python environment, Ray, services
Model Storage	50GB	500GB+	Depends on number and size of models
Database	10GB	50GB	PostgreSQL for metadata
Vector Database	10GB	100GB+	For embeddings (if enabled)
Logs & Metrics	10GB	50GB	Rotated logs, Ray dashboard data
Scratch Space	20GB	100GB	Temporary files, downloads, builds
Total	Workload-dependent	Workload-dependent	Apply the peak-use calculation above; do not double-count shared caches

The rows describe running workloads. Also allow for simultaneous image import, model staging, and storage-provider replication and reservation overhead.

Storage Performance Requirements

Local Storage (Single Node):

• Minimum: SATA SSD (500 MB/s sequential read)
• Recommended: NVMe SSD (2000+ MB/s sequential read)
• Note: HDD is only recommended for non-dynamic model loads and low KV cache usage - model load times can be very long (15+ minutes); models are in memory after load

Performance Targets:

• Sequential Read: 2000+ MB/s (model loading)
• Sequential Write: 1000+ MB/s (model downloads, checkpoints)
• 4K Random Read IOPS: 50,000+ (database, concurrent access)
• 4K Random Write IOPS: 20,000+ (database writes, logs)

Why It Matters:

• 7B model (14GB): Loads in ~7 seconds on NVMe vs ~28 seconds on SATA SSD
• Concurrent model loads across Ray workers stress random read performance
• Database query performance directly tied to IOPS

---

Supported Operating Systems

Kamiwaza runs on Kubernetes nodes with these operating systems:

Operating system	Architecture
Red Hat Enterprise Linux 9	x64/amd64
Ubuntu 24.04 LTS	x64/amd64
Ubuntu 22.04 LTS	x64/amd64

On OpenShift, the node operating system is the one OpenShift provides, and support follows the OpenShift version.

---

Supported Configurations

Configuration	Description	Support
Single-node Kubernetes with Istio	One Kubernetes node with an Istio service mesh and ingress gateway	Preferred
OpenShift with GreyMatter	Customer-operated OpenShift, including managed OpenShift such as ROSA, with GreyMatter routing	Preferred
Air-gapped	No outbound access; images and charts come from your own registry	Preferred
Federation	Two or more Kamiwaza installations joined over the mesh	Preferred
NVIDIA GPU	GPU inference through the NVIDIA GPU Operator or device plugin	Preferred
Multi-node cluster	One Kubernetes cluster spread over several nodes; see Multi-node clusters in the prerequisites for your environment	Supported

Air-gapped and NVIDIA GPU apply on top of a platform configuration; they are not platforms themselves.

Preferred configurations are the ones Kamiwaza recommends. Supported configurations are fully supported and documented.

Operation	Support
Uninstall	Supported. See the uninstall page for your environment.

---

Software Dependencies

What You Provide

Component	Requirement	Notes
License file	A signed license.lic issued by Kamiwaza	See License File. Contact your Kamiwaza representative.
Kubernetes cluster	Prepared to meet Kamiwaza's platform prerequisites	See the prerequisites for your environment
Supported OS	RHEL 9, Ubuntu 24.04, or Ubuntu 22.04 on every node	See Supported Operating Systems
Browser	Chrome 141+ (tested and recommended)	Download Chrome
GPU drivers	For GPU inference only; see below	Install on GPU nodes before installing Kamiwaza

GPU Drivers (Required for GPU Inference)

Install the appropriate driver for your GPU hardware:

NVIDIA GPUs:

Component	Requirement	Installation guide
NVIDIA driver for CUDA 13 engine images (llama.cpp, vLLM) and DGX Spark	580.65.06 (R580) or later	GPU Nodes
NVIDIA driver for CUDA 12 engine images (llama.cpp, diffusion)	525.60.13 or later; R570 or later for Blackwell GPUs and for diffusion	GPU Nodes
NVIDIA Container Toolkit	Required for GPU containers	GPU Nodes

The engines, their CUDA majors, and the GPUs each one serves are listed in Supported GPUs and driver floors.

DGX Spark software releases include the matching R580 driver and CUDA 13 stack. See the DGX Spark release notes for the versions in each release.

AMD GPUs (ROCm):

Component	Requirement	Installation guide
ROCm for Ryzen AI Max+ 395 (gfx1151)	ROCm 7.2.1 or later on Ubuntu 24.04	Ryzen native Linux compatibility
Other AMD GPUs	A ROCm release that lists the GPU and operating system as supported	ROCm compatibility matrix
Container GPU access	/dev/kfd and /dev/dri exposed to the container runtime	ROCm containers guide

AMD lists the Radeon 8060S in Ryzen AI Max+ 395 systems as production-supported on the ROCm 7.2.x native Linux path. Follow the Ryzen native Linux installation guide; the retired ROCm 7.10 preview path is not required.

---

Verifying System Requirements

Use these commands on each node to check GPU access and host resources before installation. They do not replace the checks in the prerequisites for your environment.

NVIDIA GPU (if applicable)

# Check NVIDIA driver
nvidia-smi
# Expected for CUDA 13 engine images and DGX Spark: driver 580.65.06 or later,
#   and "CUDA Version: 13.0" or later in the header
# Expected for CUDA 12 engine images: driver 525.60.13 or later, or 570 or later
#   for Blackwell GPUs and for diffusion
# Should display GPU name, driver version, and CUDA version

# Check NVIDIA Container Toolkit
nvidia-ctk --version
# Expected: Any version indicates toolkit is installed
# Example output: NVIDIA Container Toolkit CLI version 1.17.3

AMD ROCm (if applicable)

# Check ROCm installation
rocm-smi
# Expected: Should display AMD GPU information
# Look for: GPU temperature, utilization, memory usage

# Check ROCm version
cat /opt/rocm/.info/version
# Expected for Ryzen AI Max+ 395 (gfx1151): 7.2.1 or later

# Verify GPU device access
ls -la /dev/kfd /dev/dri
# Expected: Both devices should exist and be accessible

System Resources

# Check available memory
free -h
# Expected: At least 16GB total (32GB+ recommended). free -h shows a little less
# than installed memory, because the kernel and firmware reserve some: about 15Gi
# on a 16 GB machine and about 30Gi on 32 GB.
# Look for "Mem:" row, "total" column

# Check CPU cores
nproc
# Expected: 8 or more cores

# Check free space on the filesystem that holds container images
df -h /var/lib
# Plan for 350GB free; see Storage Capacity above

---

Hardware Recommendation Tiers

Kamiwaza is a distributed AI platform built on Ray that supports both CPU-only and GPU-accelerated inference. Hardware requirements vary significantly based on:

• Model size: From 0.6B to 70B+ parameters
• Deployment scale: Single-node development vs multi-node production
• Inference engine: LlamaCpp (CPU/GPU), VLLM (GPU)
• Workload type: Interactive chat, batch processing, RAG pipelines

GPU Memory Requirements by Model Size

The table below provides real-world GPU memory requirement estimates for representative models at different scales. These estimates assume FP8 and include overhead for context windows and batch processing.

Model Example	Parameters	Minimum vRAM	Notes
GPT-OSS 20B	20B	24GB	Includes weights + 1-batch max context; fits 1x 24GB GPU (e.g., L4/RTX 4090)
GPT-OSS 120B	120B	80GB	~40GB weights + 1-batch max context; 1x H100/H200 or 2x A100 80GB recommended
Qwen 3 235B A22B	235B	150GB	~120GB weights + 1-batch max context; 2x H200 (282GB) or 2x B200 (384GB) ideal for max context
Qwen 3-VL 235B A22B	235B	150GB	Same base minimum (includes 1-batch max context); budget +20-30% vRAM for high-res vision inputs

Key Considerations:

• Minimum vRAM: FP8 weights + 1-batch allocation at your target max context
• Headroom: For longer contexts, larger batch sizes, and concurrency, budget additional vRAM beyond minimums
• Vision Workloads: Image/video processing adds overhead; budget 20-30% more for vision-language models
• Tensor Parallelism: Distributing large models (120B+) across multiple GPUs requires high-bandwidth interconnects (NVLink 3.0+)

Tier 1: Development & Small Models

Use Case: Local development, testing, small to medium model deployment (up to 13B parameters)

Hardware Specifications:

• CPU: 8-16 cores / 16-32 threads
• RAM: 32GB (16GB minimum for development only)
• Storage: 400GB NVMe SSD as an initial development allocation, assuming Kamiwaza's volumes are on separate storage. If the volumes share this disk, size it using Storage Capacity.
• GPU: Optional - Single GPU with 16-24GB VRAM
  • NVIDIA RTX 4090 (24GB)
  • NVIDIA RTX 4080 (16GB)
  • NVIDIA T4 (16GB)
• Network: 1-10 Gbps

Workload Capacity:

• Low-volume workloads: 1-10 concurrent requests (supports dozens of interactive users)
• Development, testing, and proof-of-concept deployments
• Light production workloads

Tier 2: Production - Medium to Large Models

Use Case: Production deployment of medium to large models (13B-70B parameters), high throughput

Hardware Specifications:

• CPU: 32 cores / 64 threads
• RAM: 128-256GB system RAM
• Storage: 1.2–2TB NVMe SSD as a planning range; size the model library, local PVCs, and replication reserves using Capacity Planning.
• GPU: 1-4 GPUs with 40GB+ VRAM each
  • 1-4x NVIDIA B200 (192GB HBM3e)
  • 1-4x NVIDIA H200 (141GB HBM3e)
  • 1-4x NVIDIA RTX 6000 Pro Blackwell (48GB)
  • 1-2x NVIDIA H100 (80GB)
  • 1-4x NVIDIA A100 (40GB or 80GB)
  • 1-2x NVIDIA L40S (48GB)
  • 2-4x NVIDIA A10G (24GB) for tensor parallelism
• Network: 25-40 Gbps

Workload Capacity:

• Medium-scale production: 100s to 1,000+ concurrent requests (supports thousands of interactive users)
• Example: Per-GPU batch size of 32 across 8 GPUs = 256 concurrent requests; batch size of 128 = 1,024 requests
• Production chat applications
• Complex RAG pipelines with embedding generation
• Batch inference

Tier 3: Enterprise Multi-Node Cluster

Use Case: Enterprise deployment with multiple models, high availability, horizontal scaling, 99.9%+ SLA

Cluster Architecture:

Head Node (Control Plane):

• CPU: 16 cores / 32 threads
• RAM: 64GB
• Storage: 500GB NVMe SSD as an initial control-plane allocation; verify local PVC and staging requirements separately.
• GPU: Same class as worker nodes (homogeneous cluster recommended)
• Role: Ray head, API gateway, scheduling, monitoring (head performs minimal extra work; Ray backend load is distributed across nodes)

Worker Nodes (3+ nodes for HA):

• CPU: 32-64 cores / 64-128 threads per node
• RAM: 256-512GB per node
• Storage: 2TB NVMe SSD per node (local cache)
• GPU: 4-8 GPUs per node (same class as head node)
• Network: 40-100 Gbps (InfiniBand for HPC workloads)

Note: For production clusters, avoid non-homogeneous hardware (e.g., GPU-less head nodes). Each node participates in data plane duties (ingress gateway, HTTP proxying, etc.), so matching GPU capabilities simplifies scheduling and maximizes throughput.

Shared Storage:

• High-performance NAS or distributed filesystem (Lustre, CephFS)
• 10TB+ capacity, NVMe-backed
• 10+ GB/s aggregate sequential throughput
• Low-latency access (< 5ms) from all nodes

Workload Capacity:

• Multiple models deployed simultaneously
• High-scale production: 1,000–10,000+ concurrent requests (supports tens of thousands of interactive users)
• Batch sizes scale with GPU count and model size; smaller requests enable higher throughput per GPU
• High availability with automatic failover
• Horizontal auto-scaling based on load
• Production SLAs (99.9% uptime)

---

Cloud Provider Instance Mapping

AWS EC2 Instance Types

Tier	Instance Type	vCPU	RAM	GPU	Storage
Tier 1: CPU-only	m6i.2xlarge	8	32GB	None	400GB gp3
Tier 1: With GPU	g5.xlarge	4	16GB	1x A10G (24GB)	400GB gp3
Tier 1: Alternative	g5.2xlarge	8	32GB	1x A10G (24GB)	400GB gp3
Tier 2: Multi-GPU	g5.12xlarge	48	192GB	4x A10G (96GB)	2TB gp3
Tier 2: Alternative	p4d.24xlarge	96	1152GB	8x A100 (320GB)	2TB gp3
Tier 3: All Nodes	p4d.24xlarge	96	1152GB	8x A100 (320GB)	2TB gp3

Notes:

• Tier 1 disk figures are planning allocations and assume Kamiwaza's volumes are on separate storage. If the volumes share this disk, size it using Storage Capacity.
• Use gp3 SSD volumes (not gp2) for better performance/cost
• For Tier 3 shared storage: Amazon FSx for Lustre or EFS (with Provisioned Throughput)
• Use Placement Groups for low-latency multi-node clusters (Tier 3)
• H100 instances (p5.48xlarge) available in limited regions for highest performance
• Latest options: Emerging p6/p6e families with H200/B200/Grace-Blackwell are rolling out in select regions; map to Tier 2/3 as available.

Google Cloud Platform (GCP) Instance Types

Tier	Machine Type	vCPU	RAM	GPU	Storage
Tier 1: CPU-only	n2-standard-8	8	32GB	None	400GB SSD
Tier 1: With GPU	n1-standard-8 + 1x T4	8	30GB	1x T4 (16GB)	400GB SSD
Tier 1: Alternative	g2-standard-8 + 1x L4	8	32GB	1x L4 (24GB)	400GB SSD
Tier 2: Multi-GPU	a2-highgpu-4g	48	340GB	4x A100 (160GB)	2TB SSD
Tier 2: Alternative	g2-standard-48 + 4x L4	48	192GB	4x L4 (96GB)	2TB SSD
Tier 3: All Nodes	a2-highgpu-8g	96	680GB	8x A100 (320GB)	2TB SSD

Notes:

• Tier 1 disk figures are planning allocations and assume Kamiwaza's volumes are on separate storage. If the volumes share this disk, size it using Storage Capacity.
• Use pd-ssd or pd-balanced persistent disks (not pd-standard)
• For Tier 3 shared storage: Filestore High Scale tier (up to 10 GB/s)
• Use Compact Placement for low-latency multi-node clusters (Tier 3)
• L4 GPUs (24GB) available as cost-effective alternative to A100
• Latest options: Blackwell/H200 classes are entering preview/limited availability; consider AI Hypercomputer offerings as they launch.

Microsoft Azure Instance Types

Tier	VM Size	vCPU	RAM	GPU	Storage
Tier 1: CPU-only	Standard_D8s_v5	8	32GB	None	400GB Premium SSD
Tier 1: With GPU	Standard_NC4as_T4_v3	4	28GB	1x T4 (16GB)	400GB Premium SSD
Tier 1: Alternative	Standard_NC6s_v3	6	112GB	1x V100 (16GB)	400GB Premium SSD
Tier 2: H100 (recommended)	Standard_NC40ads_H100_v5	40	320GB	1x H100 (80GB)	2TB Premium SSD
Tier 2: H100 Multi-GPU	Standard_NC80adis_H100_v5	80	640GB	2x H100 (160GB)	2TB Premium SSD
Tier 2: A100 Multi-GPU	Standard_NC96ads_A100_v4	96	880GB	4x A100 (320GB)	2TB Premium SSD
Tier 2: A100 Alternative	Standard_NC48ads_A100_v4	48	440GB	2x A100 (160GB)	2TB Premium SSD
Tier 3: H100 (recommended)	Standard_ND96isr_H100_v5	96	1900GB	8x H100 (640GB)	2TB Premium SSD
Tier 3: A100 Alternative	Standard_ND96asr_v4	96	900GB	8x A100 (320GB)	2TB Premium SSD

Notes:

• Tier 1 disk figures are planning allocations and assume Kamiwaza's volumes are on separate storage. If the volumes share this disk, size it using Storage Capacity.
• Use Premium SSD (not Standard HDD or Standard SSD)
• For Tier 3 shared storage: Azure NetApp Files Premium or Ultra tier
• Use Proximity Placement Groups for low-latency multi-node clusters (Tier 3)
• NDm A100 v4 series offers InfiniBand networking for HPC workloads
• Latest options: Blackwell/H200-based VM families are announced/rolling out; align Tier 2/3 to those SKUs where available.

---

Network Configuration

Network Bandwidth Requirements

Single Node Deployment

Network Bandwidth:

• Minimum: 1 Gbps (for model downloads, API traffic)
• Recommended: 10 Gbps (for high-throughput inference)

Considerations:

• Internet bandwidth for downloading models from HuggingFace (one-time)
• Client API traffic for inference requests/responses
• Monitoring and logging egress

Multi-Node Cluster

Inter-Node Network:

• Minimum: 10 Gbps Ethernet
• Recommended: 25-40 Gbps Ethernet or InfiniBand
• Latency: < 1ms between nodes (same datacenter/availability zone)

Why It Matters:

• Ray distributed scheduling requires low-latency communication
• Tensor parallelism transfers large model shards between GPUs
• Shared storage access impacts model loading performance

Network Ports

Inbound: users reach Kamiwaza only through the platform's ingress gateway, on 443/tcp (HTTPS). Port 80/tcp is optional, for redirecting to HTTPS. Models and apps are served through the same gateway; they need no ports of their own.

Outbound: nodes and pods need HTTPS (443/tcp) access to:

• ghcr.io and pkg-containers.githubusercontent.com, for Kamiwaza charts and images;
• the model sources you use, for example huggingface.co, or your cloud model provider's endpoint.

Air-gapped installs have no outbound requirement.

---

Special Considerations

NVIDIA DGX Spark

The NVIDIA DGX Spark is a compact AI workstation powered by the GB10 Grace Blackwell Superchip:

• CPU: 20-core ARM (10x Cortex-X925 + 10x Cortex-A725)
• GPU: Blackwell architecture with 6,144 CUDA cores
• Memory: 128GB LPDDR5x unified memory (273 GB/s bandwidth)
• AI Compute: Up to 1 PFLOP FP4 AI performance
• Storage: 4TB NVMe SSD
• Networking: Dual QSFP ports (up to 200 Gbps aggregate)

Capabilities:

• Run models up to 200B parameters locally
• Two interconnected units can handle models up to 405B parameters
• Unified memory architecture eliminates GPU vRAM constraints

AMD Ryzen AI Max+ 395 "Strix Halo"

AMD's Strix Halo platform provides powerful AI inference in a compact form factor:

• CPU: 16-core Zen 5 (up to 5.1 GHz), 80MB cache
• GPU: Radeon 8060S iGPU (40 CUs, RDNA 3.5 architecture)
• NPU: 50 TOPS XDNA 2 neural engine
• Memory: Up to 128GB LPDDR5x unified memory (up to 112GB GPU-allocatable)
• AI Performance: 126 TOPS total
• TDP: 55W (highly power efficient)

Capabilities:

• Run 70B+ parameter models locally
• Available in mini PCs and high-end laptops
• Unified memory architecture (shared CPU/GPU memory)

---

Shared Storage (Multi-Node Clusters)

Network Filesystem Requirements:

• Protocol: NFSv4, Lustre, CephFS, or S3-compatible object storage
• Network Bandwidth: 10 Gbps minimum, 40+ Gbps for production
• Network Latency: < 5ms between nodes and storage
• Sequential Throughput: 5+ GB/s aggregate (10+ GB/s for large clusters)

Object Storage (Alternative):

• S3-compatible API (AWS S3, GCS, MinIO, etc.)
• Local caching layer recommended for frequently accessed models
• Consider bandwidth costs for cloud object storage

Shared Storage Options:

Solution	Use Case	Throughput	Cost Profile
NFS over NVMe	Small clusters (< 5 nodes)	1-5 GB/s	Low (commodity hardware)
AWS FSx for Lustre	AWS multi-node clusters	1-10 GB/s	Medium (pay per GB/month + throughput)
GCP Filestore High Scale	GCP multi-node clusters	Up to 10 GB/s	Medium-High
Azure NetApp Files Ultra	Azure multi-node clusters	Up to 10 GB/s	High
CephFS	On-premises clusters	5-20 GB/s	Medium (requires Ceph cluster)
Object Storage + Cache	Cost-optimized	Varies	Low storage, high egress

Version Compatibility

• NVIDIA driver: 525.60.13 or later for CUDA 12 images (R570 or later for Blackwell GPUs and for diffusion); 580.65.06 (R580) or later for CUDA 13 images and DGX Spark

---

Important Notes

• GPU Support: NVIDIA GPUs (CUDA) and AMD GPUs (ROCm) on Linux nodes
• Platform: Kamiwaza does not install or configure Kubernetes, the service mesh, storage drivers, or GPU drivers; see the prerequisites for your environment
• Network: Users reach Kamiwaza through the ingress gateway on 443/tcp; see Network Ports
• License: A license file is required for every installation; see License File