AI
8.0
Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM
AWS demonstrates deploying Alibaba's Qwen3.8-2.4T-A95B (2.4T parameters, 95B active per token) on SageMaker HyperPod using vLLM on ml.p6-b300 instances with 8x NVIDIA B300 Blackwell GPUs. The post covers cluster provisioning, OpenAI-compatible endpoints, NVFP4 quantization, tool calling, and Multi-Token Prediction speculative decoding for agentic and reasoning workloads supporting up to 262K native context (extensible to 1M tokens).
Read article →