Model

Model profile

Load repository configuration

ModelScope, then Hugging Face fallback

Workload

Peak reservation

Weight format KV-cache format Max context / request Peak concurrency Average live context Output tokens / request Serving runtime Traffic type Target aggregate tok/s0 = capacity only Usable VRAM

Hardware constraint

Optional

GPU Interconnect Preferred maximum GPUsResults still report the actual requirement if exceeded

Current estimate

Deployment analysis

ESTIMATED PROFILE

Required topology

—

Select a model

Capacity and throughput sizing

Usable VRAM consumed—

Total demand: —
Configuration: —

Weights—including metadata

KV cache—at peak reservation

Aggregate decode—roofline estimate

Per request—fair-share estimate

Memory allocation

Cluster total

Throughput limits

Roofline model

Compute—

Memory bandwidth—

Hardware candidates

Ranked by fit, topology, speed, and GPU count

RankConfigurationVRAM reserveEst. decodeScore

Saved scenarios

Save the current estimate to compare models, quantization formats, or GPU options.

Calculation details

Method and limitations

Memory reserves the full configured context for each concurrent request. Decode throughput uses the lower of an active-parameter compute ceiling and a memory-bandwidth ceiling, then applies runtime, batching, and interconnect efficiency. Use this for planning, then benchmark the exact checkpoint and engine before procurement.