An AWS engineering post walks through using SageMaker AI Inference Recommendations to run automated concurrency sweeps, benchmarking NVIDIA Nemotron-3 Nano 30B on ml.g7e.2xlarge instances to find the highest concurrency that stays within latency SLAs. The feature is built into SageMaker, so teams avoid building custom load-testing infrastructure, and a search recipe can auto-discover the saturation point cheaper than a linear sweep.
Amazon SageMaker AI introduced a user interface for its generative AI inference recommendations feature, which was previously available only via APIs. The UI helps developers quickly find optimal instance types, serving containers, and optimization strategies for deploying generative AI models, reducing the manual benchmarking cycle.
Amazon SageMaker AI introduced a user interface for its generative AI inference recommendations feature, which was previously available only via APIs. The UI helps developers quickly find optimal instance types, serving containers, and optimization strategies for deploying generative AI models, reducing the manual benchmarking cycle.
AWS announced the aws-ai-ml skill, available through the Agent Toolkit for AWS, which gives coding agents like Kiro, Claude Code, and Codex SageMaker AI inference optimization expertise. Agents can benchmark live endpoints, recommend deployment configurations, compare benchmark runs, and generate executable SageMaker Python SDK v3 code. The skill works with any MCP-supporting agent and can also run in a pre-configured SageMaker Studio JupyterLab space.
AWS announced the aws-ai-ml skill, available through the Agent Toolkit for AWS, which gives coding agents like Kiro, Claude Code, and Codex SageMaker AI inference optimization expertise. Agents can benchmark live endpoints, recommend deployment configurations, compare benchmark runs, and generate executable SageMaker Python SDK v3 code. The skill works with any MCP-supporting agent and can also run in a pre-configured SageMaker Studio JupyterLab space.
AWS benchmark compares small LLM inference on SageMaker G7 vs G5 and G6 instances
AWS benchmark compares small LLM inference on SageMaker G7 vs G5 and G6 instances
Amazon SageMaker Python SDK v3 now exposes generative AI inference recommendations directly in notebook workflows, helping developers benchmark endpoints and evaluate instance configurations. The recommendations are also available through the SageMaker AI UI, streamlining deployment optimization for LLMs.
Amazon SageMaker Python SDK v3 now exposes generative AI inference recommendations directly in notebook workflows, helping developers benchmark endpoints and evaluate instance configurations. The recommendations are also available through the SageMaker AI UI, streamlining deployment optimization for LLMs.
An AWS post demonstrates deploying Hugging Face models to SageMaker AI using six open-source agent skills from the Hugging Face Skills repo, working with Kiro and Claude Code. Unguided agents chose outdated TGI containers and produced failed or silently broken deployments; the skills select vLLM from the AWS DLC catalog and add autoscaling, alarms, and teardown.
An AWS post demonstrates deploying Hugging Face models to SageMaker AI using six open-source agent skills from the Hugging Face Skills repo, working with Kiro and Claude Code. Unguided agents chose outdated TGI containers and produced failed or silently broken deployments; the skills select vLLM from the AWS DLC catalog and add autoscaling, alarms, and teardown.