24–32 GB class
L4, A10, RTX 4090 or RTX 5090. Consider for smaller models only after measuring total memory, including runtime and cache.
AI inference
AI inference
AI inference: model selection, memory requirements, storage and deployment choices without invented performance claims.
Choose a configuration in Console to review its resources and estimated cost.
What to plan for
Inference executes a trained model. Input size, batching and context length determine working memory as well as model weights.
Decide between a GPU VM you operate and a managed endpoint. The public Finder currently checks GPU VM offers, not a managed inference SLA.
Configuration choices
48–80 GB class
L40S, A40, A100 or H100. More memory may accommodate larger working sets; it does not guarantee concurrency or token speed.
141 GB and multi-GPU
H200, B200 or B300 and explicitly sized multi-GPU systems. Confirm exact model, precision, topology and serving framework.
Weight-only memory ≈ parameters × bytes per parameter. Runtime, KV cache, activations and optimizer state are additional. This is a sizing method, not a deployment recommendation.
Smart Infra Finder
Your workload, your requirementsCheck availability: AI inference
Check the current price and configuration in Console. Your choice is saved through registration. The current lookup uses mock adapters; no real capacity is reserved.
Related products and guides
GPU cloud
Compare GPU memory and architectures, from RTX to data-center accelerators, then check configurations in one console.
Explore →AI infrastructure
Plan infrastructure for model hosting, inference, training and fine-tuning with explicit memory and workload requirements.
Explore →Cloud storage
Compare object, block and file storage, backups and snapshots by access pattern, durability needs and restore workflow.
Explore →Cloud backup
Cloud backup: access model, capacity planning and recovery considerations before choosing a configuration.
Explore →Specification sources
Hardware and model documentation; these sources do not confirm rental availability.
Official documentation 1 →