Skip to content
LiveMonitoring 39 sourcesעברית
AI·Newsroom

What’s new in AI: models, tools and techniques, verified

Models & ReleasesDeep Dive2 min read

Amazon SageMaker Adds Support for NVIDIA Multi-Instance GPU Partitioning

Amazon Web Services has introduced GPU partitioning on SageMaker Inference using NVIDIA Multi-Instance GPU technology to allow multiple models to share a single hardware-isolated accelerator.

AI Newsroom Automated DeskPublished Nov 7, 2026, 10:02 UTCConfirmed

In 30 seconds

Amazon SageMaker Inference now supports NVIDIA Multi-Instance GPU (MIG) partitioning, allowing users to divide a single physical GPU into independent partitions. This feature enables organizations to host multiple models on one accelerator with hardware-enforced isolation, improving utilization and reducing inference costs.

Enabling Hardware-Isolated GPU Sharing on SageMaker

Amazon Web Services has announced that Amazon SageMaker Inference now supports GPU partitioning using NVIDIA Multi-Instance GPU (MIG) technology.[1] The capability divides a physical graphics processing unit into independent partitions, each receiving a fixed share of compute units and a dedicated block of memory with hardware-enforced isolation.[1][2]

Previously, deploying a model to a GPU endpoint required requesting at least one whole accelerator.[1] This constraint often left hardware underutilized when smaller models could not consume an entire device, or forced teams to manage complex self-managed Kubernetes stacks to utilize MIG.[1]

How GPU Partitioning Operates

GPU partitioning builds directly on SageMaker inference components, which are used to deploy several models to a single endpoint.[1] Enabling the feature requires adding optional API fields to declare how accelerators are divided on a production variant and to assign individual inference components to specific partition types.[1][2]

The feature does not introduce new API operations, resource types, or IAM actions.[2] Endpoints that omit the partitioning fields continue to operate as they did previously.[1][2]

Supported Hardware and Partition Profiles

GPU partitioning is supported across specific instance families equipped with NVIDIA accelerators, including P4d with A100 GPUs, P5 with H100 GPUs, G7 with RTX PRO 4500 Blackwell GPUs, and G7e with RTX PRO 6000 Blackwell Server Edition GPUs.[1][2] The maximum number of partitions depends on the specific accelerator and does not exceed seven.[1]

According to AWS documentation, carving accelerators during endpoint provisioning adds approximately 17 seconds to the setup process.[1] Partition layouts cannot be modified on a running endpoint; changing the layout requires creating a new endpoint configuration.[2]

Impact on Workload Consolidation and Costs

Salesforce, which participated in defining requirements and validating the initial implementation, stated that the capability would allow organizations to safely consolidate workloads, improve GPU utilization, reduce inference costs, and maintain predictable performance.[1] By providing hardware-isolated slices, teams can host portfolios of fine-tuned variants, embedding models, and safety classifiers on shared accelerators without managing custom infrastructure.[1]

Open Questions and Boundaries

Some statements regarding the benefits of MIG are the organization's own characterizations and have not been independently tested. Financial terms were not disclosed in the sources reviewed, and no independent reporting on this development has been found yet.

Timeline

  1. Amazon Web Services publishes[1]
  2. First detected by AI Newsroom
  3. Verification completed

Sources

  1. [1]
  2. [2]

How this article was produced

This article was researched and written by AI Newsroom’s automated editorial system using a large language model, constrained to cited evidence and checked by an automated quality gate. No human journalist wrote it.

This version was published automatically after passing quality checks, without human review.

AI Newsroom Automated Desk
Language model: gemini / gemini-3.5-flash-lite
Prompt version
article-draft-2026-11-07.2
Editorial standards

Related coverage

Models & Releases