Mirage Packs Vision Perception and Structured Reasoning Into 22 MB
By Mr.Xu Community Post
Published:
Summary:Cactus Compute has released Mirage, a 51.3M-parameter vision model packaged into a 22 MB weight file for on-device perception. It combines scene description, object detection, OCR, segmentation, spatial relations, and structured decisions in a single pass. Its HConvGLU image encoder feeds dedicated OCR and spatial heads, while a compact decoder built around Simple Attention and Hadamard MLPs produces a structured record that can be passed to a text-only LLM. Cactus reports end-to-end latencies
51.3M Parameters and a 22 MB Weight File for On-Device Vision
Cactus Compute has released Mirage, a compact vision model designed for phones, wearables, and robots rather than as a conventional general-purpose visual-language chatbot. A single 22 MB weight file supports scene description, object detection, OCR, segmentation, spatial relations, field extraction, and structured decisions. Cactus reports end-to-end latency of 74 ms on the NPU and 107 ms on the CPU of a Samsung Galaxy S26.
The model contains 51.3M parameters: 7.8M in the image encoder, 40.8M in the language decoder, and 2.6M across the OCR and spatial heads. Its central design choice is to produce structured facts directly rather than generate a long prose description and force a downstream LLM to recover objects, attributes, and relationships from that prose.
Architecture: Shared Features with Specialized Perception Heads
Mirage uses an HConvGLU image encoder that combines fixed Hadamard transforms with small learned convolutions for channel mixing and local feature extraction. This avoids the cost of large dense projections while retaining useful structural inductive biases.
Detection and segmentation share image features but use separate output heads. OCR and spatial relations also have dedicated heads, reducing the need for the language decoder to infer text and geometry implicitly. The decoder uses Simple Attention and Hadamard MLPs, building on the design family behind Cactus Compute's Needle and Whistle models. It generates names, properties, relationships, and a compact reasoning record, with short references connecting names to image regions.
The underlying hypothesis is that a model optimized for a fixed perception schema does not need to spend much of its limited capacity on connective prose and open-ended description. That capacity can instead be directed toward visual evidence.
The Main Use Case: A Perception Patch for Text-Only LLMs
Mirage's most useful role may be serving as an edge vision encoder whose structured output is passed to a text-only LLM. According to Cactus, on 256 VQAv2 questions drawn from 128 images, an unfine-tuned Qwen3-0.6B scored 9.3 with the question alone and 31.3 after receiving Mirage's structured record and full trace.
Gains were more modest for larger readers. Qwen3-1.7B improved from 27.5 to 31.6, while Qwen3-4B-Instruct-2507 moved from 33.4 to 37.3. The structured visual record also appeared more important than the reasoning trace: in one example, removing the trace increased the score from 22.2 to 33.1. Cactus interprets this as evidence that image-specific observations, rather than additional open-ended reasoning, are the main source of value.
These results require caution. The test contains only 256 questions, covers a limited set of readers and prompts, and mixes answer correctness with JSON-format compliance. Nemotron violated the required JSON format in 214 of 256 responses, while Qwen3-1.7B failed in 34. Those failures received zero credit, so the reported scores are not pure measures of visual-question answering quality.
Engineering Limits: Language Scale Does Not Repair Missing Perception
A skateboard example exposes the central limitation. Mirage missed the skateboard and reported cars, people, and shoes. Qwen guessed “a skateboard” from the question alone, but answered “a car” after reading Mirage's record and trace.
A stronger reader cannot compensate for absent evidence. If the perception module misses an object or produces a wrong detection, a downstream model may follow those errors more confidently because they arrive in a machine-readable form. Production systems should preserve confidence scores, absent categories, and conflicting observations rather than treating every field as equally reliable evidence.
The latency figures also need deployment context. They were measured on a Galaxy S26 under Cactus's own end-to-end setup. NPU operator support, memory bandwidth, image preprocessing, and model conversion can materially affect results on other chips. Moreover, a 22 MB weight file does not equal 22 MB of runtime memory: activations, image tensors, KV cache, and backend workspaces still have to be accounted for.
Deployment Recommendations: Start with Verifiable, Narrow Tasks
For phone, wearable, and robotics developers, Mirage is best suited to tasks such as extracting objects, attributes, OCR text, and spatial relations for a later LLM or controller; performing local visual state checks in privacy-sensitive or offline settings; prefiltering fixed scenarios such as shelf-stock checks; or generating structured observations with image-region references for evidence tracing.
Two safeguards are essential. First, the structured record should include field-level confidence, missing-object indicators, and conflicting observations so that the reader can reject weak evidence. Second, schema-constrained decoding or output validation should prevent formatting violations from being scored as task failures.
For workloads that depend on precise counting, tiny objects, severe occlusion, or complex geometry, Mirage should be treated as a fast first-stage model in a confidence-routing system rather than an automatic replacement for a larger cloud vision model.
Verdict: A Promising Edge Baseline, Not a General VLM
Mirage demonstrates that multi-task perception can be packaged into a shared encoder, specialized heads, and structured outputs that run at near-interactive latency on a phone. Its most compelling use is as a lightweight perception front end for existing language models, not as a standalone universal VLM.
The current evidence is still primarily project-authored, uses a limited evaluation, and combines answer quality with format compliance. Developers should reproduce the model on their target devices and real workloads, measuring p95 latency, peak memory, perception recall, and—most importantly—the downstream impact of incorrect observations before putting it into production.
— END —Source: Reddit r/LocalLLaMA (2026-11-06)
Tags: #Mirage #On-Device VLM #HConvGLU #Multimodal Reasoning #Cactus Compute
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments