Thinking Machines Launches Inkling 2 Small, a 594B-Parameter MoE Model with Vision and Variable Compute

Summary:Thinking Machines Lab has introduced Inkling 2 Small, the second generation of its Inkling model family. The Mixture-of-Experts model contains 594B total parameters with 17B active per token, accepts image inputs, and supports variable inference effort. It scores 31 on the Artificial Analysis Intelligence Index, with reported gains in agentic coding, vision, knowledge work, and reasoning. Inkling 2 Small is initially available through Tinker for closed-weight inference, fine-tuning, and samplin

---

Core Release

Thinking Machines Lab has introduced Inkling 2 Small, a general-purpose Mixture-of-Experts model designed for vision input, agentic coding, and variable inference effort. It grows from the previous Inkling Small model's 276B total parameters and 12B active parameters to 594B total parameters and 17B active parameters.

The company reports a score of 31 on the Artificial Analysis Intelligence Index, with gains spanning agentic coding, vision, knowledge work, and reasoning. Inkling 2 Small is available through Tinker Playground for inference, fine-tuning, and sampling. The company plans to release the full weights and downloadable checkpoints under the Apache 2.0 license once external safety testing and red-teaming are complete.

How the MoE Architecture Works

The apparent gap between 594B and 17B is the defining feature of the model. The former represents the combined size of the routed experts and shared components; the latter is the amount of parameter computation typically activated for each token. A router selects a small set of experts for each token, allowing the model to retain the capacity of a much larger network while limiting the compute used per step.

Under balanced routing, the approximate activation ratio is:

[ 17B / 594B \approx 2.9% ]

This does not mean Inkling 2 Small can be deployed like a 17B dense model. All 594B parameters must still be stored somewhere and selectively loaded or accessed. Inference cost also depends on expert parallelism, routing balance, batching, KV Cache usage, the vision encoder, and communication across accelerators.

The central MoE bottleneck is expert load imbalance. If a small set of experts attracts disproportionate traffic, local compute and interconnect bandwidth can become saturated even when the average number of active parameters remains low. The reported support for variable effort is promising, but the announcement does not disclose effort tiers, routing thresholds, latency, throughput, or the shape of the quality-cost curve.

Pretraining and Reinforcement Learning

Thinking Machines attributes the capability gains to improvements across the training pipeline, including more efficient pretraining and a large unified reinforcement-learning run. A unified RL stage can reduce policy drift between separate post-training phases and allow coding, vision, and general reasoning objectives to be optimized in one framework.

For MoE models, however, unified training does not automatically mean cheaper or more stable training. RL rollout cost depends on response lengths, reward verification, and expert utilization. Poorly balanced routing can lower accelerator utilization even if the nominal active-parameter count remains unchanged. The announcement provides no training compute, reward design, context length, sampling budget, or RL hyperparameters, so the claimed improvement cannot yet be attributed or reproduced.

Interpreting the Benchmark

A score of 31 on the Artificial Analysis Intelligence Index places Inkling 2 Small in a more competitive tier, but an aggregate index cannot establish leadership in any individual capability. The composition of the score matters, including prompt templates, available tools, context limits, and inference budget.

The claim that the model substantially exceeds earlier Inkling models must also be interpreted through compute allocation. If the previous generation used less test-time computation while Inkling 2 Small benefits from variable effort, part of the improvement may come from additional sampling rather than architecture alone. A fair comparison should hold prompts, tools, context, concurrency, output length, and reasoning budget constant while measuring quality, latency, throughput, and total token cost.

Deployment Trade-offs

Inkling 2 Small is not a conventional single-consumer-GPU model. At BF16, the weights alone would occupy approximately 594 GB. FP8 would halve the theoretical storage requirement to about 594 GB as well in aggregate terms? More precisely, 594B parameters at one byte per parameter require roughly 594 GB, before KV Cache, vision components, runtime buffers, and communication overhead. INT4 could reduce storage further, but would introduce quantization error, expert-dependent precision sensitivity, and possible routing shifts.

The model therefore requires a multi-GPU or multi-node deployment even though each token activates only about 17B parameters. Operators must account for expert affinity, all-to-all traffic, batch scheduling, and the startup cost of loading a very large checkpoint. A local inference setup also needs to determine whether experts are sharded by accelerator, layer, or placement groups, and whether routing can keep high-demand experts from becoming hotspots.

Deployment Guidance for Developers

For now, Tinker is the most practical route for evaluation, fine-tuning, and sampling. Teams considering local deployment should:

1. Map effort to cost: Record how each variable-effort setting affects token limits, sampling behavior, latency, and quality.
2. Establish strong baselines: Compare against the previous Inkling Small and similarly sized open MoE models under fixed prompts, tools, and inference budgets.
3. Calculate end-to-end resources: Include weight memory, multi-accelerator links, expert communication, KV Cache, image preprocessing, and long-context usage rather than relying on the 17B active-parameter figure alone.
4. Re-evaluate fine-tuning after weights open: Once checkpoints are downloadable, test data mixtures by expert, routing stability, and post-quantization quality loss.
5. Run independent safety evaluations: Open licensing changes distribution, not the attack surface. Evaluate jailbreaks, image prompt injection, unsafe tool use, and multimodal contamination separately.

What to Watch

Inkling 2 Small combines vision, agentic coding, and variable inference effort in a very large sparse model. Its developer value will ultimately depend on three undisclosed measurements: the real latency and token cost of each effort tier, multi-node throughput with complete weights, and whether fine-tuning gains justify the additional training expense. If the planned Apache 2.0 release proceeds, the model could enter the open, auditable and adaptable model ecosystem; until then, Tinker remains the main channel for validating its practical value.

---

Source: Reddit r/LocalLLaMA (2026-11-06)

— END —

Tags: #Inkling 2 Small Follow #Thinking Machines Lab Follow #MoE Architecture Follow #Multimodal Models Follow #Agentic Coding Follow