TLDR; Inkling 2 Mini 是一种新的循环 Transformer 模型。它是一个多模态混合专家(Mixture-of-Experts)模型,拥有约 260 亿个物理总参数量和每个 token 30 亿个活跃参数。它使用了一个巧妙的技巧来增加容量:部分层在一个重复三次的循环中被访问,这使得它表现得像拥有更多参数一样。它支持 100 万 token 的上下文大小,接受图像和文本输入。Thinking Machines Lab 以混合精度格式发布该模型,包含 NVFP4 和 MXFP8 专家权重。还包含一个 DSpark 草稿生成器,以较小的内存成本提供额外的加速。
Inkling 2 Mini 比 Inkling 2 Small 小得多,专为本地 AI 设置设计。我们认为这是一个令人耳目一新的优秀本地模型。在我们的直观测试中,它的表现至少与 Qwen 3.8 27B 相当。需要更多的验证,但我们对这个发布感到非常兴奋!
我们提供 transformers、Llama App (llama.cpp) 和 MLX 的支持。SGLang 也为数据中心硬件上的生产部署提供了零天推理支持。
Inkling 2 Mini 是一个具有 24 个解码器层的混合专家模型,但中间的 12 层在循环中运行三次。因此,该模型表现得好像它有 48 (6 + 3*12 + 6) 个逻辑层,而只有 24 个物理层。
与 Inkling 2 Small 类似,Inkling 2 Mini 为每个 token 执行 48 次层访问。隐藏层大小为 2,304,而不是 4,608。在每次层访问中,两种变体都将 token 路由到 8 个专家,但 Mini 从每层 128 个专家中选择,而 Small 从 384 个专家中选择。
解码器运行第 0–5 层一次,第 6–17 层三次,第 18–23 层一次。通过第 6–17 层的三次传递重用相同的权重。
Nanbeige 等架构之前已经探索过循环。但 Inkling 2 Mini 探索了一种有趣的方法。每次通过循环核心(第 6–17 层)都从相同的输入开始,即前六层的输出。信息通过跨传递 消息 在传递之间移动。每个循环层从其自身的学习查询(一个用于其查询、键、值和 MLP 输入的消息)读取来自前一次传递输出的消息。然后一个小门控控制器查看层的当前输入和消息。它决定忘记多少当前输入并添加多少消息。因此,每次传递都会改进前一次计算的内容。
除了旗舰版 Inkling 2 Mini 模型外,还发布了一个研究工件。它共享相同的架构,但这个变体被训练为接受可变数量的循环传递 🤯。将 1 或 2 次传递设置为 3 次,将使推理运行得更快,但生成的质量可能略低。我们测试了这个变体,它有效,但我们没有深入评估它。
Inkling 2 Mini 在 Artificial Analysis Intelligence Index v5 (AAv5) 中处于同类领先地位,并在各种类别的许多基准测试中脱颖而出。与 Gemma 4 31B、Muse Glimmer、Qwen3.8-27B 和 Nemotron 3.5 Super VL 相比,Inkling 2 Mini 在 48 个基准测试中的 32 个中领先。这些涵盖编码和自动研究、知识工作的代理搜索和工具使用、视觉解释和创建、推理和知识,以及安全性和对齐。请参阅 Inkling 2 Mini 模型卡 获取完整的基准测试列表和分数。
Inkling 2 Mini 是一个优秀的本地编码助手。它在 FrontierCode v2.0 中领先,在 Terminal-Bench 4.0 中与 Qwen3.8-27B 持平,在 AutomationBench-AA v1.0.6 中落后于 Qwen。你可以在下面查看我们在编码和知识工作方面的一些亮点。请注意,AA Briefcase 采用 ELO 评级,越高越好。
在这里,我们让 4 位模型为一个名为 Plant Therapy 的 SaaS 制作网站,其中植物与持证治疗师匹配。模型非常自主,它为网站的视觉效果制作了 SVG,甚至构建了匹配逻辑。
Inkling 2 Mini 在 SGLang 中自带 Day 0 支持。你可以安装最新版本的 SGLang 并执行以下操作以启动服务器。需要处理的一个先决条件是在 SM103 上运行它。
查看原文
TLDR; Inkling 2 Mini is a new looped transformer model. It’s a multimodal Mixture-of-Experts with ~26B physical total and 3B active parameters per token. It uses a neat trick to increase capacity: some layers are visited in a loop that repeats three times, which makes it work like it had more parameters. It supports a 1M-token context size, accepts images and text inputs. Thinking Machines Lab ships the model in a mixed-precision format, with NVFP4 and MXFP8 expert weights. A DSpark drafter is also included for additional speed-up at a small memory cost.
Inkling 2 Mini is much, much smaller than Inkling 2 Small, and is designed for local AI setups. We think this is a refreshingly new great local model. In our vibe tests, it's at least comparable with Qwen 3.8 27B. More validation is needed, but we are super excited about this release!
We ship transformers, Llama App (llama.cpp), and MLX support. SGLang also provides day-zero inference support for production deployments on datacenter hardware.
This post goes through what makes Inkling Mini different, some of our vibe tests, and how you can use it with your setup.
What is new with Inkling 2 Mini?
Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop. Therefore, the model acts as if it had 48 (6+ 3*12 + 6) logical layers with just 24 physical ones.
Similar to Inkling 2 Small, Inkling 2 Mini executes 48 layer visits per token. The hidden size is 2,304 instead of 4,608. At each layer visit, both variants route the token to 8 experts, but Mini selects them from among 128 experts per layer, while Small selects from 384.
The decoder runs layers 0–5 once, layers 6–17 three times, and layers 18–23 once. All three passes through layers 6–17 reuse the same weights.
Architectures such as Nanbeige have explored looping before. But Inkling 2 Mini explores an interesting approach. Every pass through the looped core (layers 6–17) starts from the same inputs which is the output of the first six layers. Information moves between passes through cross-pass messages. Each looped layer reads a message from the previous pass's outputs with its own learned query (one message for its query, key, value and MLP inputs). A small gating controller then looks at both the layer's current input and the message. It decides how much of the current input to forget and how much of the message to add. Each pass therefore refines what the previous one computed.

- Each six-layer block has five sliding-window layers and one full-attention layer. The local window holds up to 1,024 tokens.
- Attention uses 24 query heads and eight KV heads. It uses learned relative position logits, a learned sink, and an output gate. Full-attention layers use dynamic key and value convolutions.
- The router selects eight of 128 SwiGLU experts for each token. It normalizes the selected scores and scales the weights by 8.
- During passes 2 and 3, each core layer attends over two residual blocks from the previous pass. Four rank-64 gates mix separate messages into the query, key, value, and MoE inputs.
Text decoder width is 2,304, and the intermediate width of each expert is 1,152. Reusing core weights reduces the number of distinct layers the checkpoint stores. The model still executes all 48 visits, and each visit keeps its own KV and short-convolution state.
In addition to the flagship Inkling 2 Mini model, a research artifact has also been released. It shares the same architecture, but this variant was trained to accept a variable number of passes through the loop 🤯. Setting 1 or 2 passes instead of 3 will make inference run a lot faster, but generations may be of slightly lower quality. We tested this variant, and it works, but we haven’t evaluated it in depth.
Benchmark Results
Inkling 2 Mini is state-of-the-art in its class in the Artificial Analysis Intelligence Index v5 (AAv5), as well as across many benchmarks in various categories. Compared with Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL, Inkling 2 Mini leads in 32 benchmarks out of 48. These cover coding and autoresearch, agentic search and tool use for knowledge work, vision interpretation and creation, reasoning and knowledge, and safety and alignment. See the Inkling 2 Mini model card for the full list of benchmarks and scores.
Inkling 2 Mini is a great local coding companion. It leads in FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and trails Qwen on AutomationBench-AA v1.0.6. You can check out a few of our highlights below for coding and knowledge work. Note that AA Briefcase is in ELO rating, higher is better.

Inference Support
In this section, you can find how to use Inkling 2 Mini in transformers, llama.cpp, MLX, and SGLang.
Transformers
We provide support for both Inkling 2 Mini (fixed loop) and the TML-d24-varloop-research (variable loop) research variant.
Fixed loop with Inkling 2 Mini
You can use the AutoModelForMultimodalLM class.
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "thinkingmachines/Inkling-2-Mini"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id,
)
messages = [{"role": "user", "content": [{"type": "text", "text": "Why is the sky blue?"}]}]
inputs = processor.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
).to(model.device)
input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))
Variable loop
To set the number of loops in the research checkpoint, you can modify the loop_num_passes in text_config as follows. The example below uses two loops.
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "thinkingmachines/TML-d24-varloop-research"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id,
text_config={"loop_num_passes": 2},
)
messages = [{"role": "user", "content": [{"type": "text", "text": "Why is the sky blue?"}]}]
inputs = processor.apply_chat_template(
messages, reasoning_effort="medium", tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
).to(model.device)
input_len = inputs["input_ids"].shape[-1]
outputs = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))
Images and text
For an image prompt, replace messages in either example with the list below and infer as usual.
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": "/path/to/image.jpg"},
{"type": "text", "text": "What is in this image?"},
],
}
]
Reasoning Effort
Pass reasoning_effort="medium" to processor.apply_chat_template(...) to set the effort for a request. The chat template adds the numeric value to the prompt as a system message. The default is high (0.9).
inputs = processor.apply_chat_template(
messages, reasoning_effort="medium", tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
).to(model.device)
Available levels are none (0.0), minimal (0.1), low (0.2), medium (0.7), high (0.9; default), xhigh (0.99), and max (0.99). You can also pass a number from 0.0 to 0.99, such as reasoning_effort=0.35.
Tool calling
Let’s get our model to call some tools.
In this example we ask our model about the current temperature at a given location. With the prompt, we also provide the tool (get_current_temperature) to do it. The model asks for a tool call, we call the tool on its behalf and then add the output to the next instruction set. It uses the output of the tool and provides us with the final result.
If you want to know more about tool calling, we recommend going to our official docs.
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "thinkingmachines/Inkling-2-Mini"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
def get_current_temperature(location: str, unit: str):
"""
Get the current temperature at a location.
Args:
location: The location to get the temperature for, in the format "City, Country"
unit: The unit to return the temperature in. (choices: ["celsius", "fahrenheit"])
"""
return 22.0 # A real function would call a weather API here
def ask(messages):
inputs = processor.apply_chat_template(
messages, tools=[get_current_temperature], reasoning_effort="low", tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
output = model.generate(**inputs, max_new_tokens=1024)[0][inputs["input_ids"].shape[-1] :]
parsed = processor.parse_response(output, prefix=inputs["input_ids"][0])
return parsed
messages = [{"role": "user", "content": "Hey, what's the temperature in Paris right now?"}]
reply = ask(messages)
call = reply["tool_calls"][0]["function"]
print("Model calls:", call["name"], call["arguments"]) # the model fills in the arguments itself
temperature = get_current_temperature(**call["arguments"])
messages += [reply, {"role": "tool", "name": call["name"], "content": str(temperature)}]
print(ask(messages)["content"])
# Model calls: get_current_temperature {'location': 'Paris, France', 'unit': 'celsius'}
# Right now in Paris it’s about **22°C** (roughly **71.6°F**).
Llama App and Llama.cpp
Llama App lets you easily run and manage local language models on macOS (from the menu bar) or Windows (system tray). It’s built on llama.cpp, and by the same team. The app also provides a local OpenAI-compatible API for easy integration with coding agents, editors, and scripts.
For the Inkling 2 Mini release we provide GGUF 4-bit (IQ4_NL) and 8-bit (Q8_0) quants for both models.
If you’re on Linux or you want to serve the model from the CLI, you can run the following command for the fixed-loop model:
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NL
To run the variable-loop model, you can run
llama serve -hf ggml-org/TML-d24-varloop-research-GGUF:IQ4_NL
To load the 8-bit weights, change the suffix to :Q8_0.
Here we ask the 4-bit model to make a website to a SaaS called plant therapy where plants get matched to licensed therapists. Model is quite autonomous, it has made its own SVGs for the visuals on the website, and even built the matching logic.
SGLang
Inkling 2 Mini comes with Day 0 support inside SGLang. You can install the latest version of SGLang and execute the following to get the server up and running. One prerequisite to take care of is to run it on SM103.
export MODEL="thinkingmachines/Inkling-2-Mini"
python -m sglang.launch_server \
--model-path $MODEL \
--served-model-name inkling \
--attention-backend fa4 --bf16-gemm-backend torch --dtype bfloat16 \
--tp 1 --page-size 128 --grammar-backend xgrammar \
--host 0.0.0.0 --port 30000
After the server is up you should be able to request the APIs like so
curl -s localhost:30000/health_generate && echo OK
curl -s localhost:30000/v1/models | python -m json.tool
curl -s localhost:30000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "inkling",
"messages": [{"role": "user", "content": "What is 17 times 24?"}],
"max_tokens": 512,
"reasoning_effort": "low"
}' | python -m json.tool
One can also enable multimodality by setting --enable-multimodal.
Speculative decoding
SGLang can use a three-layer DSpark draft to propose tokens for the target model to verify. The draft reads the inputs to decoder visits 5, 23, and 41, plus the final normalized state at index 48. With the draft weights present, add --speculative-algorithm DSPARK --context-length 131072 to the server command.
We benchmarked the models with and without speculative decoding, and measured generation speed on a single NVIDIA B300 GPU where every request has a 1,024-token prompt and generates exactly 1,024 tokens and concurrency of 1, 4, 16 and 64.
Show SGLang benchmarking script
MODEL=thinkingmachines/Inkling-2-Mini
for TAG in base mtp; do
EXTRA=""; [ $TAG = mtp ] && EXTRA="--speculative-algorithm DSPARK"
# Start the server (with or without MTP)
python -m sglang.launch_server --model-path $MODEL \
--attention-backend fa4 --bf16-gemm-backend torch --dtype bfloat16 \
--tp 1 --page-size 128 --context-length 131072 --max-running-requests 64 \
--port 30000 $EXTRA > $TAG.log 2>&1 &
until curl -sf localhost:30000/health_generate > /dev/null; do sleep 10; done
# Benchmark at increasing concurrency
for C in 1 4 16 64; do
python -m sglang.bench_serving --backend sglang --port 30000 --model $MODEL \
--dataset-name random --apply-chat-template \
--random-input-len 1024 --random-output-len 1024 --random-range-ratio 1.0 \
--num-prompts $(( C*4 < 32 ? 32 : C*4 )) --max-concurrency $C \
--output-file results_$TAG.jsonl
done
kill %1; wait
done

MLX
We upstreamed MLX support to the mlx-lm project, including multimodal image and text inputs, and shared quants through the mlx-community organization.
MLX inference works with the original checkpoint too. In this case, the native nvfp4 + mxfp8 quantization scheme is used without reconversion: you get the original quality the authors intended.
Here’s an example CLI invocation:
mlx_lm generate --model thinkingmachines/Inkling-2-Mini --image image.jpg --prompt "Describe this image.<|end_message|><|message_user|><|content_image|><|unused_200054|>" -m 1000
Demos and Vibe Tests
We have given Inkling 2 Mini a shot in many fun tasks, check them out in this section.
3D shader flythroughs
The same prompt produced two sunset flythroughs over procedural mountains and reflective water, each rendered with a single WebGL fragment shader.
Sunset Ridge (Inkling 2 Mini)
Over the Still Sea (Qwen 3.8 27B)
Show shared flythrough prompt
Build the most impressive original real-time scene you can: a slow, cinematic camera flight over procedural mountains, through volumetric clouds and low over water with reflections, all lit by a sunset. Deliver it as a single self-contained index.html in the current directory that renders everything in one full-screen WebGL fragment shader, purely from code (no textures, images, fonts, libraries, external files or network requests of any kind). It must run smoothly in desktop Chrome at 1920x1080, aiming for 60 fps on a desktop GPU, with an automatic resolution-scale fallback that lowers the internal render resolution when the frame rate drops, and the flight must keep going indefinitely without glitches. To check your work, run node shot.cjs index.html shot.png 6 (the third argument is how many seconds to wait; a comma list such as 3,12,25 saves one screenshot per time, and an optional fourth argument sets the size, default 1280x720): it renders the page in headless Chrome, saves the screenshot(s), and prints the WebGL renderer, a rough frame rate and any console errors; then look at the images. The headless renderer on this machine may be software-only, so its frame rate can be far below what a real desktop GPU achieves and is not the target. Decide for yourself when the result is finished, then reply with a short description of what you built.
Starry Night, Alive

Vincent van Gogh, The Starry Night (1889)
Inkling 2 Mini recreation
We gave Inkling 2 Mini a reference image of Van Gogh’s painting and a screenshot helper in the Pi coding harness. Inkling wrote a self-contained HTML page with a single WebGL2 fragment shader. The shader draws the scene from code without loading the reference image or any other assets. It uses time to animate the swirling sky, the glow of the stars, the swaying cypress, and the flicker of the village windows.
Inkling rendered and reviewed its work over 24 rounds of screenshot checks. We recorded the final animation frame by frame at 1080p and 60 fps on an Apple M4 Max.
Show prompt
In this folder, starry-night.jpg is Vincent van Gogh's The Starry Night (1889, public domain). Recreate it as a living painting: a single self-contained index.html (all code inline, no external files, libraries or network requests) that fills the browser window and runs smoothly in desktop Chrome at 1920x1080. Everything must be generated procedurally in code, for example with WebGL shaders; the page must not load, embed or sample starry-night.jpg or any other image. Stay faithful to the painting's composition and palette (the great spiral and swirling currents of the sky, the haloed stars, the glowing crescent moon, the dark flame-like cypress on the left, the rolling hills, and the village with its church spire), rendered in Van Gogh's thick, directional brushstrokes, and bring it to life: the sky's swirls flow, the stars and moon pulse with light, the cypress sways, and the village windows glow and flicker. Aim for the most impressive result you can make. A helper is provided: run node shot.js --help to see how to screenshot index.html at several moments; look at the screenshots next to the reference and keep improving until you are satisfied, then reply with a short summary of what you made.
Napkin to App

Input napkin sketch

Generated Trip Split app
We gave Inkling 2 Mini a photo of a hand-drawn Trip Split wireframe and a one-paragraph prompt in the Pi coding harness. Inkling used the sketch’s labels, sample expenses, and handwritten notes to build the expense list, add-expense form, and balances view in a single HTML file with vanilla CSS and JavaScript. The app updates totals and balances when expenses change, suggests payments to settle the trip, and saves data in the browser’s localStorage so it survives a reload.
We ran three independent Inkling attempts and used Playwright to click through each build, checking screenshots where visual judgment was needed. The first attempt, shown here, passed all 13 checks. Adding the sketch’s $90 dinner raised the total from $405 to $495 and recalculated what each person owed.
Show prompt
The photo sketch.jpg in this folder is a product designer's hand-drawn wireframe of a small app. Build that app for real, as the most impressive, original, production-quality version you can make: a single self-contained index.html in this folder using vanilla HTML, CSS and JavaScript only (no frameworks, CDNs, external files or network requests; localStorage is fine), which must run smoothly in desktop Chrome at 1920x1080. Treat everything on the paper as the spec: every screen, label, button and number, the example data, and each red note describe behaviour that must actually work, while the visual design, typography and motion are yours to make beautiful. Look at the sketch carefully with your vision, and use your vision again to check your own work: node shot.cjs --help explains a provided helper that screenshots index.html in headless Chrome at any window size after clicks or other JavaScript steps (Playwright is also installed if you prefer your own scripts), and you can look at any screenshot with the read tool. Click through every feature, compare each screen against the sketch, and keep improving until you are fully satisfied; then reply with a short summary of what you built.
4-bit Model
We ask the 4-bit GGUF quantized Inkling 2 Mini (IQ4_NL) to make a website to a SaaS called plant therapy where plants get matched to licensed therapists.
Model is quite autonomous, it has made its own SVGs for the visuals on the website, and even built the matching logic, all from a short prompt: "Can you make a website for my saas called plant therapy where I match house plants with licensed therapists so plants don’t die, return single HTML”.
“Make yourself into a browser agent"
We have paired Inkling 2 Mini with Pi coding agent to build itself into a browser agent using Helium, Chromium and Hugging Face Sandboxes. We provide the traces and the final file (that can be used as a browser agent) in this bucket.
Wrapping Up
We are very glad to partner with Thinking Machines Lab to ship Inkling 2 Mini, a performant mid-sized model for local setups. Give this model a try and let us know what you think in comments!