原标题：nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB (text-generation)
NVIDIA 在 Hugging Face 发布 NVIDIA-Nemotron-3.5-Bolt-E4B-4GB，这是基于 Gemma-4-E4B-it 后训练的小型语言模型的 Q4_K_M 量化版，总参数 8B（有效 4B），上下文输入输出均最高 64K token。模型面向低内存、低 TTFT 场景，针对本地语音/设备助手、交互角色和嵌入式接口优化多轮工具调用、指令遵循与输入 grounding，支持通过 chat 模板开关思考模式并可搭配 DFlash drafter 做投机解码，采用 OpenMDW-1.1 许可，可用于商业用途。
README 给出参数、上下文长度、精度和部署方式等具体规格，读者可评估其在本地低内存场景的可用性。
本页的摘要、推荐理由与解读出自 AIsa Studio 的 AI 风向智能体，原文版权归原作者所有。
Hugging Face 热门模型第 29 名：nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB，164 赞，近期下载 44651 次。 标签：transformers, gguf, nvidia, pytorch, nemotron-3.5, text-generation, en, ar, bg, cs, da, de
README：
NVIDIA-Nemotron-3.5-Bolt-E4B-4GB is a small language model (SLM) further post-trained from Gemma-4-E4B-it by NVIDIA. Nemotron-3.5-Bolt-E4B carries forward Gemma’s strong conversational capabilities and further improves multi-turn tool use, instruction following, and input grounding for fast local voice/device assistants, interactive personas, and embedded/edge interfaces. Designed for low-memory, low-TTFT operation, the model allows agents to respond quickly while preserving headroom for applications or adjacent model pipelines.
For speculative decoding, use the companion DFlash drafter. For the full-precision (BF16) version, see NVIDIA-Nemotron-3.5-Bolt-E4B-BF16.
This model is ready for commercial or non‑commercial use.
| Total Parameters | 8B (4B effective) |
| Architecture | Gemma-4-E4B |
| Precision | Q4_K_M |
| Context Length | Input: up to 64K tokens; output: up to 64K tokens |
| Inputs / Outputs | Text / text |
| Deployment | llama.cpp (recipe pending) |
| Reasoning Mode | Chat-template controls (enable_thinking=True/False) and runtime thinking budget |
| Conversational Mode | Chat-template controls (conversational_mode=True/False) |
| Recommended Sampling | Temperature 1.0, top_p 0.95 in both thinking modes |
| Best For | Local voice and device assistants, interactive personas, and embedded interfaces |
| License | OpenMDW-1.1 |
Model Developer: NVIDIA Corporation
Model Dates: December 2025 - May 2026
Data Freshness:
NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.
Studioexport MODEL_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
export MODEL_FILE=Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf
export DRAFTER_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash
export DRAFTER_FILE=Nemotron-3.5-Bolt-E4B-DFlash-BF16.gguf
export MODEL_DIR="$PWD/bolt-models"
hf download "$MODEL_REPO" "$MODEL_FILE" \
--local-dir "$MODEL_DIR"
hf download "$DRAFTER_REPO" "$DRAFTER_FILE" \
--local-dir "$MODEL_DIR"
docker run --rm --gpus all \
-p 8000:8000 \
-v "$MODEL_DIR:/models:ro" \
ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
-m "/models/$MODEL_FILE" \
--alias nemotron-bolt \
--host 0.0.0.0 \
--port 8000 \
-c 65536 \
-ngl 999 \
--jinja
docker run --rm --gpus all \
-p 8000:8000 \
-v "$MODEL_DIR:/models:ro" \
ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
-m "/models/$MODEL_FILE" \
--spec-draft-model "/models/$DRAFTER_FILE" \
--spec-type draft-dflash \
--spec-draft-n-max 7 \
--spec-draft-ngl 999 \
--alias nemotron-bolt \
--host 0.0.0.0 \
--port 8000 \
-c 65536 \
-ngl 999 \
--jinja
curl --fail-with-body -sS \
http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "nemotron-bolt",
"messages": [
{"role": "user", "content": "What is 17 multiplied by 23?"}
],
"temperature": 0.0,
"max_tokens": 512,
"chat_template_kwargs": {
"enable_thinking": true
}
}'
For more details on how to deploy and use the model — see the Quick Start Guide below!
Governing Terms: Use of this model is governed by the [OpenMDW License Agreement, version 1.1](https://github.com/O
原文版权归 Hugging Face 热门模型 及原作者所有，本站转载仅供阅读。如需删除，请联系 support@aisa.one。