An AWS blog post presents a validated architecture pairing Amazon SageMaker HyperPod with Qumulo's Cloud Native Qumulo and Cloud Data Fabric to train models in one AWS Region while data remains in another. A 1.02B-parameter LLaMA v3 run across us-east-2 to us-west-2 (60 ms latency) matched co-located throughput after NeuralCache warmup. This lets teams avoid petabyte-scale replication or cross-Region latency penalties.
AWS announced a tiered KV cache feature for large language model inference on SageMaker HyperPod, developed with Curvine. It reduces infrastructure costs and improves time-to-first-token by avoiding recomputation of identical prompts, benefiting teams running models like Qwen, Llama, and DeepSeek.
AWS announced a tiered KV cache feature for large language model inference on SageMaker HyperPod, developed with Curvine. It reduces infrastructure costs and improves time-to-first-token by avoiding recomputation of identical prompts, benefiting teams running models like Qwen, Llama, and DeepSeek.
Amazon announced SageMaker HyperPod Inference Gateway, a Kubernetes-native addon that routes LLM inference requests using real-time GPU signals instead of round-robin load balancing. Benchmarks show P95 first-token latency cuts up to 98% and throughput gains up to 50% under mixed or bursty workloads. It deploys as a single EKS addon with no application or SDK changes.
Amazon announced SageMaker HyperPod Inference Gateway, a Kubernetes-native addon that routes LLM inference requests using real-time GPU signals instead of round-robin load balancing. Benchmarks show P95 first-token latency cuts up to 98% and throughput gains up to 50% under mixed or bursty workloads. It deploys as a single EKS addon with no application or SDK changes.
AWS published a walkthrough showing how to run the open-source SkyRL RL framework on SageMaker HyperPod to post-train Qwen3-VL-8B for multi-turn visual maze navigation. GRPO training raises maze solve rate from 43.75% to over 95% on a fixed 64-maze eval set. It matters to practitioners because it shows a reproducible recipe for multi-node RL post-training with colocation, checkpointing, and managed monitoring.
AWS published a walkthrough showing how to run the open-source SkyRL RL framework on SageMaker HyperPod to post-train Qwen3-VL-8B for multi-turn visual maze navigation. GRPO training raises maze solve rate from 43.75% to over 95% on a fixed 64-maze eval set. It matters to practitioners because it shows a reproducible recipe for multi-node RL post-training with colocation, checkpointing, and managed monitoring.
AWS announced new Ray capabilities on SageMaker HyperPod, integrating the open-source Ray framework with HyperPod's purpose-built infrastructure for foundation model training and serving. The integration uses KubeRay to manage Ray clusters on Kubernetes, enabling distributed Python workloads such as Ray Train and Ray Serve.
AWS announced new Ray capabilities on SageMaker HyperPod, integrating the open-source Ray framework with HyperPod's purpose-built infrastructure for foundation model training and serving. The integration uses KubeRay to manage Ray clusters on Kubernetes, enabling distributed Python workloads such as Ray Train and Ray Serve.
AWS published guidance on administering Amazon SageMaker HyperPod clusters shared by multiple teams through SageMaker Unified Studio. It lays out a four-layer control model—organization, project, cluster, and workload—for governing identity, capacity allocation, and observability. The guidance matters to platform teams running shared accelerator clusters for training and fine-tuning.
AWS published guidance on administering Amazon SageMaker HyperPod clusters shared by multiple teams through SageMaker Unified Studio. It lays out a four-layer control model—organization, project, cluster, and workload—for governing identity, capacity allocation, and observability. The guidance matters to platform teams running shared accelerator clusters for training and fine-tuning.