BlockLock Fixes Batch Geometry for Reproducible LLM Inference
Summary:BlockLock addresses batch-dependent inference by fixing the numerical path observed by a protected request while allowing other requests to vary. It offers two mechanisms: BlockLock-Rows preserves batched decoding with fixed-width rows, while BlockLock-Pack uses terminal-anchored packing and isolates the target's KV state for target-only decoding. Across five dense models on NVIDIA A6000, A100, and H100 deployments, the reported experiments preserve target sequences and Top-20 logit bits. At equ
Core idea: move reproducibility from a global constraint to a per-request guarantee
Reproducible LLM inference is not merely a matter of random seeds. The same request can produce different outputs when it is placed in different batches because matrix tiling, reduction order, operator-fusion boundaries, and KV Cache layouts may change. These floating-point variations make output drift difficult to audit or reproduce.
BlockLock does not modify every operator. Instead, it fixes the numerical path observed by protected requests. Ordinary requests can continue sharing batched Prefill and Decode, while the protected request remains stable when its batch companions change.
Two execution modes
BlockLock-Rows
BlockLock-Rows assigns fixed-width rows to a protected request and still allows other requests to participate in batched Decode. The target's matrix shapes, tiling decisions, and reduction order remain independent of batch composition, allowing unused batch capacity to be filled by other requests.
This avoids globally enforcing Batch-Invariant kernels and does not require a separate execution graph for every protected request. Fixed width, however, introduces padding when the request is shorter than the allocated geometry, and longer requests must be divided across fixed-width units.
BlockLock-Pack
BlockLock-Pack uses terminal-anchored packing and creates an isolated dummy segment for the protected request. Prefill can still be batched with other requests, after which the target's KV state is extracted for target-only decoding.
This further reduces batch coupling during Decode, but the isolated target cannot use the same batched parallelism as ordinary requests. KV state extraction also introduces additional data movement and scheduling overhead. The abstract does not provide full cost curves for different context lengths and output-length distributions.
Why fixed geometry stabilizes floating-point results
Floating-point addition is not associative. Reducing the same values in different orders can change the least significant bits. Batch-Invariant kernels constrain tiling and reduction behavior across the entire inference stack. BlockLock instead fixes the shapes and layouts observed by the target request, so changing batch companions no longer changes its numerical path.
The practical distinction is important: Batch-Invariant makes the whole system resistant to batch changes, whereas BlockLock charges the reproducibility cost only where it is needed. Production services can use this as a per-request or per-tenant reliability tier rather than applying the strictest determinism policy to all traffic.
Evaluation results and scope
The paper evaluates Qwen3-8B, Qwen3-14B, Llama-3.1-8B-Instruct, Gemma-3-4B-IT, and Gemma-3-12B-IT across NVIDIA A6000, A100, and H100 single-GPU deployments. It uses tinyBenchmarks and BFCL, with initial tensor-parallel experiments on two A6000 GPUs.
The reported tests preserve target sequences and Top-20 logit bits in both modes. At equal delivered work, BlockLock-Rows reportedly reduces latency by 53.5%–55.0% and improves throughput by 115.2%–122.2% relative to vLLM Batch-Invariant.
The results are promising, but the guarantees are conditional. Bitwise identity holds under the paper's stated deployment contract, which includes the hardware, operator implementations, batching policy, and request geometry. Moving to another GPU, driver, CUDA version, inference framework, or model build should not automatically preserve the same outputs. The abstract also does not report end-to-end tail latency, padding overhead under skewed sequence lengths, KV movement costs, or concurrency fairness.
Engineering trade-offs
BlockLock limits reproducibility overhead to protected requests instead of imposing Batch-Invariant constraints on all traffic. Compared with an isolated fixed-shape Prefill/Decode baseline, it preserves shared Prefill computation; compared with Batch-Invariant kernels, it requires no custom kernel implementations.
Three costs remain. Fixed-width rows create padding, which becomes more expensive as request lengths diverge. BlockLock-Pack's isolated Decode reduces batching efficiency for the target. Finally, bitwise identity is a brittle guarantee: changes to kernel selection, fusion strategy, or parallel partitioning can invalidate it. Production deployments should therefore record hardware fingerprints, framework versions, scheduler policies, and geometry settings alongside results.
Deployment recommendations
For teams that need stable regression testing, model auditing, cross-batch consistency, or replayable inference, BlockLock is worth prototyping in an existing vLLM deployment. The first test should submit the same request with different batch companions and compare the complete token sequence, logit bit patterns, and failure cases rather than relying only on task accuracy.
Teams should also measure P50, P95, and P99 latency, GPU utilization, and throughput for both modes under short outputs, long outputs, highly skewed request lengths, and high concurrency. The serving stack should expose protected requests as an explicit scheduling attribute, pin their row width, packing anchor, and Decode path, and rerun bit-level validation after inference-framework or hardware upgrades.
BlockLock presents a pragmatic compromise for batched inference: rather than enforcing global determinism for every request, it locks the necessary numerical path on a request-by-request basis. For auditable production systems, that is more controllable than relying on random seeds or occasional retries.
— END —Source: ArXiv AI (cs.AI) (2026-11-06)
Tags: #LLM Inference #Batch Determinism #Reproducibility #vLLM #GPU Inference
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments