Kokon 0.12 Synthesizes Editable Image Layers with Qwen-Image 2.1

Summary:Kokon 0.12 introduces a Layers Lab that builds editable image layers around Qwen-Image 2.1, a model that does not natively output layers. The pipeline combines interactive MobileSAM segmentation, CPU-based matting, masked Qwen-Image inpainting, and an optional relighting pass that synthesizes contact shadows and reflections. Objects can then be repositioned and exported as a layered PSD file. The release also adds a local server, password-protected LAN access, Docker packaging, an MCP server fo

---

Kokon 0.12 expands image editing from isolated local modifications to an editable-layer workflow. Rather than introducing a new generative model, it wraps Qwen-Image 2.1 in a segmentation-to-inpainting pipeline: MobileSAM selects objects, a matting model extracts their edges, Qwen-Image reconstructs the background, and a final pass can synthesize shadows and reflections before the result is exported as a PSD file.

How the layer pipeline works

The workflow consists of four stages:

1. Select: A click produces an initial mask through the 44 MB MobileSAM ONNX model on CPU. Clicking again expands the selected region.
2. Cut out: Fast mode uses a CPU-based matting model, while Quality mode asks Qwen-Image to redraw difficult boundaries such as hair and fur.
3. Fill: A masked Qwen-Image edit reconstructs the area exposed by the removed object. This is the most expensive stage in the pipeline.
4. Blend into scene: After an object is repositioned, the system generates contact shadows and reflections as a separate, adjustable layer.

This is an orchestration approach rather than a new end-to-end layer-generation architecture. Kokon divides the task into segmentation, alpha matting, generative inpainting, and relighting, each of which is better matched to a different model class. The tradeoff is error propagation: a faulty mask affects matting, residual foreground pixels interfere with inpainting, and an incorrect background reconstruction can be amplified by the blending stage.

Performance and hardware costs

The project reports background reconstruction at roughly 15 to 25 seconds per object on an RTX 5070 Ti and one to two minutes on an M2 Pro. The cost includes diffusion sampling as well as image encoding, mask processing, memory management, and differences in operator support across backends. Repositioning an object and running the optional blend stage requires another editing pass.

Kokon includes a native Swift/Metal engine for Apple Silicon. It runs Unsloth's Q4_K_M quantization of the Qwen-Image 2.1 Turbo model. The developer reports about one minute per 1 MP image on an M2 Pro, approximately four times faster than the generic backend. A contributor's M5 neural-accelerator path reduced processing time from 37 to 29 seconds on an M5 Pro. These figures lack unified precision, power, batch-throughput, and quality conditions and therefore require independent reproduction.

The recommended NVIDIA configuration is 12 GB of VRAM, with 8 GB available as a beta option. AMD and Intel support through Vulkan remains experimental. The application code is released under AGPL, while the model weights remain subject to Qwen's non-commercial license. Quantization lowers memory requirements, but image editing is sensitive to edge geometry, color consistency, and fine textures; the quality impact cannot be inferred from text-generation benchmarks.

What still fails—and why

The developer openly identifies several failure modes: removing a large person can cause the model to invent a different person; touching people in a crowd may be merged or split; hidden regions are not reconstructed; and per-object blending often fails more often than whole-scene blending.

These limitations expose a fundamental constraint of diffusion-based editing. Within an eight-step sampling budget, the model must infer object removal, occlusion, background semantics, and generation direction simultaneously. A click usually supplies only a two-dimensional location rather than explicit geometry or information about invisible content. As the edited region grows, conditioning becomes weaker and the model is more likely to replace the removed subject with another plausible subject.

The blend stage also combines two underdetermined tasks: inferring contact surfaces, shadow direction, and reflections from a single 2D image. Whole-scene editing retains more visual context and is therefore more coherent, whereas isolated per-object editing frequently loses the relationships required for physically plausible lighting.

Deployment guidance

Kokon 0.12 is best suited to local asset organization, concept-image decomposition, and lightweight PSD delivery. It should not yet be treated as a production-grade pixel-accurate cutout system.

• Use MobileSAM and CPU matting as a fast pre-processing path, and invoke generative edge repair only when needed.
• Keep the original image as a reference and manually inspect large inpainting regions, since background subjects may be unexpectedly replaced.
• Prefer at least 12 GB of VRAM for more demanding edits and higher-resolution workflows.
• When exposing the editor to agents through kokon-mcp, define narrow operations for selection, matting, and inpainting, and validate output paths and write permissions.
• Keep LAN access disabled by default; if enabled, retain authentication and rate limiting, and do not expose the service directly to the public internet.

The local server, Docker packaging, MCP interface, and Apple Silicon backend turn Kokon into a more accessible automation platform. However, non-commercial model licensing, backend-dependent performance, and the remaining gap between synthetic layers and deterministic compositing still define its practical limits.

---

Source: Reddit r/LocalLLaMA (2026-11-05)

— END —

Tags: #Qwen-Image 2.1 Follow #Layer Editing Follow #MobileSAM Follow #Local Inference Follow #MCP Follow