AWS Benchmarks Show NVIDIA Blackwell G7 Outperforms G5 and G6 for LLM Inference on SageMaker
Summary
- • AWS benchmarked G7 (Blackwell) vs G5 (A10G) and G6 (L4) for 30B MoE LLM inference on SageMaker
- • G7 achieves better throughput, latency, and cost-per-token using only 2 GPUs vs 4 for prior generations
- • Tests cover Qwen3-Coder-30B (coding) and NVIDIA Nemotron-3-Nano-30B (enterprise assistant)
- • SageMaker AI Inference Recommendations feature automates GPU configuration benchmarking and selection
Details
AWS benchmarked G7, G6, and G5 GPU instances for 30B MoE LLM inference
Published comparison of three NVIDIA GPU instance families for LLM inference on Amazon SageMaker AI, covering throughput, latency, and cost-per-token.
G7 instances use NVIDIA RTX PRO 4500 Blackwell GPUs
AWS G7 12xlarge uses 2 Blackwell GPUs with 64GB aggregate memory, representing NVIDIA's latest GPU generation.
G7 (2 GPUs, 64GB) outperforms G5/G6 (4 GPUs, 96–192GB) on price-performance
Despite fewer GPUs and less total memory, G7 delivers better price-performance for the tested MoE models due to Blackwell architecture efficiency.
Use Case 1: Qwen3-Coder-30B benchmarked for coding assistant
Deployed on ml.g5.12xlarge (A10G), ml.g6.12xlarge (L4), and ml.g7.12xlarge (Blackwell) using SageMaker DJL Large Model Inference container.
Use Case 2: Nemotron-3-Nano-30B benchmarked for enterprise assistant
Deployed on G6 (L4), G6e (L40S), and G7 (Blackwell) using vLLM via SageMaker AI Generative AI Inference Recommendations.
SageMaker AI Inference Recommendations automates optimal GPU config selection
Feature accepts model, expected workload, and optimization goal; evaluates candidate GPU configurations on real infrastructure and returns deployment-ready recommendations.
Real-world LLM inference gains vary by model architecture and workload shape
AWS notes performance improvements from GPU generation jumps depend on model architecture, quantization format, and workload characteristics.
AWS SageMaker GPU benchmarking: G7 vs G5/G6 for 30B MoE LLM inference
What This Means
AWS's benchmarks provide concrete evidence that NVIDIA's Blackwell architecture delivers significant price-performance improvements for LLM inference vs prior GPU generations — G7 achieves better results with fewer GPUs. This gives enterprises and ML engineers actionable data when selecting GPU instances for SageMaker deployments. The automated Inference Recommendations feature also lowers the expertise barrier for optimizing inference configurations, which is relevant for teams scaling generative AI in production.