NVIDIA and Microsoft Expand the Full-Stack AI Stack: Vera Rubin on Azure, Nemotron 3.5 in Foundry
Summary:At Microsoft Ignite, NVIDIA and Microsoft announced a broad expansion of their AI stack across Azure, Microsoft Foundry, Azure Local, Windows PCs, and Microsoft Discovery. Azure VMs will offer a preview of Vera Rubin NVL72-based instances, while rack-scale confidential computing and MaxLPS collaboration target multi-accelerator security and power efficiency. Foundry adds Nemotron 3.5 models, programmatic post-training, custom-weight deployment, and retrieval, parsing, and safety components. The
What changed: Azure, Foundry, and the edge execution tier expanded together
This is not a standalone foundation-model launch. NVIDIA and Microsoft connected three delivery layers: Azure supplies rack-scale accelerated compute, Microsoft Foundry handles model selection, post-training, and deployment, while Azure Local, Windows PCs, and Jetson execute workloads closer to the data. The engineering value is reduced friction when moving models from experimentation into production, rather than another headline model score.
Three main changes stand out. First, Azure VMs will offer a preview powered by NVIDIA Vera Rubin NVL72 systems, with confidential computing and MaxLPS collaboration extending toward bare-metal and rack-scale infrastructure. Second, Foundry gains additional Nemotron 3.5 models, programmatic post-training, and custom-weight deployment. Third, the deployment boundary now extends to Azure Local, RTX PRO 6000 Blackwell, Jetson Thor and Orin, RTX Spark, and the scientific-AI toolchain inside Microsoft Discovery.
Vera Rubin NVL72: the rack becomes a compute and security boundary
The significance of NVL72 is not simply a higher GPU count. It treats a rack as a high-bandwidth, low-latency scale-up domain. Once model parallelism spans multiple GPUs, every layer may trigger collective communication. Interconnect topology, routing congestion, node failures, and synchronization jitter become real throughput constraints. A rack-scale design can shorten data paths, but it does not eliminate communication volume, memory pressure, or recovery costs.
For long-context or high-concurrency serving, KV-cache memory can be approximated as:
KV_bytes ≈ 2 × L × H_kv × S × B × b
Here, L is the number of layers, H_kv is the aggregate KV-head dimension, S is sequence length, B is concurrent requests, and b is bytes per numeric element. GQA reduces H_kv; prefix caching and paged scheduling avoid some duplicate storage. Even so, KV cache can reach capacity limits before model weights do as context grows. The announcement does not disclose Rubin's memory capacity, interconnect bandwidth, or context specifications, so NVL72 branding alone is insufficient to calculate realistic concurrency.
Confidential computing and remote attestation form the second engineering thread. The companies plan to extend protection from individual VMs to bare-metal and rack-scale AI infrastructure, allowing customers to verify the execution environment before releasing models, data, or secrets. Production deployment still has to address multi-node attestation chains, key rotation, firmware trust, network transport protection, and endpoint authentication during collective communication. The overhead from memory encryption and attestation validation was not quantified.
The MaxLPS collaboration targets useful throughput under constrained power. The meaningful metric is effective tokens or completed tasks per joule at an identical SLA, not peak FLOPS or a rack's rated power. Otherwise, higher raw performance can hide an energy regression.
Dynamo and reinforcement-learning rollouts: an inference path optimized for throughput
Microsoft AI is already using NVIDIA Dynamo for reinforcement-learning rollout inference during post-training. Rollout engines generate many interaction trajectories in parallel, so the serving system must handle uneven trajectory lengths, variable tool-call durations, and stragglers caused by synchronized batch completion.
In this setting, Dynamo coordinates inference across a GPU fleet, manages requests, handles model updates, and supports failure recovery. Scheduling prefill and decode separately can relieve compute-bound and memory-bandwidth-bound phases, but transferring KV Cache across nodes adds network and serialization overhead. Failure retries may also repeat expensive prefill work. Microsoft has not disclosed its parallel topology, cache policy, or quantization configuration, so Dynamo’s general capabilities should not be converted directly into measured gains for this deployment.
Steady-state decode throughput can be approximated as:
Q ≈ B / t_token
where B is effective concurrent sequence count and t_token is the effective inter-token interval. Queueing, sampling, retries, and request-length variance also affect real service. Frequent weight updates during post-training introduce version-skew risk: older requests may hold KV Cache from an earlier model version while new requests use updated weights. Without a controlled drain or migration mechanism, the system can briefly invalidate caches or create consistency issues. Microsoft is also using RTX PRO 6000 Blackwell GPUs for Bing search models, showing that its internal deployment strategy includes workstation-class inference alongside hyperscale clusters.
Foundry: post-training, model weights, and deployment become one loop
Programmed Training is a programmable post-training service in Microsoft Foundry. Developers can define data processing, training, evaluation, and iteration workflows while Foundry manages the underlying compute. Managed Compute can then deploy custom model weights. Compared with uploading a single fine-tuned checkpoint, this pattern places data governance, behavioral evaluation, and release versions inside one control chain.
The Nemotron 3.5 lineup has a clearer division of labor. Super targets long-running coding, research, and business agents; Lightning targets fast responses and tool use. Embed and Rerank handle retrieval and reranking, Parse extracts structured document information, and Content Safety addresses harmful content. Bolt and Transcribe are still forthcoming and should not be treated as currently available.
EY used Programmed Training to adapt NVIDIA Nemotron open models for enterprise purchase-record classification. NVIDIA reports a 55% accuracy improvement on EY’s evaluation set and a 15-fold cost reduction versus proprietary frontier models. That is an attractive task-level result, but not evidence of a universal advantage: the announcement omits the base model, dataset size, metric definition, proprietary model versions, experimental variance, and end-to-end cost accounting. Production selection still requires testing on proprietary data with identical quality thresholds and workload volume.
Programmable post-training also introduces trade-offs. A managed service lowers cluster-orchestration overhead, but may hide differences in GPU models, parallel strategies, and software versions. If training data crosses a multi-tenant control plane, teams must verify regional residency, encryption boundaries, checkpoint permissions, and deletion guarantees. Reproducibility requires pinning the base model, data snapshot, prompt template, hyperparameters, random seeds, and evaluation set rather than retaining only the final weights.
Azure Local, Jetson, and RTX Spark: a tiered execution model
Azure Local will support RTX PRO 6000 Blackwell and expand to Jetson Thor and Jetson Orin for machine vision, robotics, and autonomous operations. NVIDIA Cosmos physical-AI models will also arrive in Foundry Local. RTX Spark systems will demonstrate local large-language-model applications on Windows PCs.
The first constraint on local deployment is usually memory and thermal budget. Weights occupy roughly parameter count × bytes per parameter, in addition to KV cache, runtime workspaces, vision encoders, context length, and allocator buffers. Nemotron 3.5 Super and Lightning have no published parameter counts, context limits, or quantization profiles, so their hardware fit cannot be inferred from the product names. A sound approach is to compute peak memory from the longest real context, compare FP16, INT8, and lower-bit quantization, then measure sustained throughput, thermal throttling, and cold-start time.
Direct Stream for Windows 365 Cloud PCs addresses a separate path: GPU-accelerated graphics streaming. Its target is two 4K streams at 60 fps with low latency. It is currently a limited preview, with public availability planned for early 2027. This is a graphics and interaction capability, not evidence of local LLM throughput, and should not be conflated with inference on RTX Spark.
Scientific AI: candidate generation is platform-ready, but validation remains decisive
The BioNeMo Agent Toolkit and ALCHEMI Toolkit are joining the Microsoft Discovery catalog for life sciences, drug discovery, materials, and chemical science. BioNeMo lets scientists express computational-biology workflows in natural language while agents coordinate tool and model calls. The first integration uses RFD4-Proteina, developed with the University of Washington’s Institute for Protein Design, to generate candidate protein structures and amino-acid sequences for subsequent evaluation and laboratory validation.
Such systems can broaden the candidate space, but a generated structure is not necessarily a stable conformation, and neither is evidence of biological or drug activity. Production research still requires validation of novelty, sequence-structure consistency, reproducibility, negative-result handling, and wet-lab closure. Agentic orchestration across several models and tools also amplifies error propagation, so tool parameters, model versions, and evidence chains must be retained at every step.
What the public evidence actually proves
| Announcement or result | Evidence type | What it supports | What it does not support | |---|---|---|---| | Azure preview of Vera Rubin NVL72 instances | Official preview announcement | Rubin is entering the Azure delivery path | Performance, cost, capacity, or GA timing for Rubin | | Azure trained DeepSeek-V3 across 10,240 GB200 GPUs | MLPerf Training v6.1 | Azure can operate large-scale training and submit at system scale | Best performance per watt or lowest cost | | Azure’s GB300 led DeepSeek-R1 interactive throughput by over 10% | MLPerf Inference v6.1 and Signal65 analysis | Leading interactive throughput among relevant cloud submissions | Better latency or TCO in every scenario, or equivalent Rubin performance | | EY: 55% accuracy gain and 15-fold cost reduction | Single customer case | Potential for open-model adaptation on a specific task | Generalization to other domains, models, or data distributions | | Direct Stream with two 4K60 streams | Limited-preview specification | A concrete graphics-streaming target | LLM inference performance or public-preview availability |
Deployment guidance for developers
- Treat the Rubin preview as a capacity and compatibility test, not a GA commitment. Confirm regional quota, instance shapes, image support, fault domains, and billing before migrating training or inference.
- Use one measurement vocabulary. Record TTFT, TPOT, p99 latency, effective concurrency, tokens per second, cost per million tokens, and throughput per watt. Results from different MLPerf scenarios or system configurations should not be combined directly.
- Make Foundry post-training reproducible. Pin base models and data snapshots, isolate training and evaluation sets, preserve checkpoints and regression gates, and choose open or proprietary models based on cost at a fixed quality threshold.
- Validate the confidential-computing trust chain. Examine attestation freshness, key lifecycle, multi-rack networking, GPU endpoint authentication, and encryption overhead. Attestation alone is not a substitute for runtime monitoring.
- Budget memory and thermals before edge deployment. Calculate KV-cache peaks from the longest real context, evaluate quantization accuracy and sustained throughput, and measure frequency throttling instead of judging usability from time to first token.
- Keep experimental gates in scientific AI. Treat generated proteins, molecules, and materials as hypotheses that require structural checks, computational reproduction, and wet-lab validation.
Bottom line
The importance of this announcement is the connected path from Vera Rubin infrastructure and Nemotron models to programmatic post-training, custom-weight deployment, and local execution. The strongest public performance data still comes from GB200 and GB300 systems; the Rubin preview has no independently reproducible performance or cost figures. For enterprises, the practical route is to validate customization gains in Foundry first, then use proprietary workloads to measure the latency, cost, data-governance, and energy boundaries across Azure, Azure Local, and edge devices.
— END —Source: NVIDIA AI News (2026-11-19)
Tags: #NVIDIA #Microsoft Azure #Vera Rubin NVL72 #Nemotron 3.5 #Cloud-to-Edge AI
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments