Hugging Face 热门模型第 29 名：nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB，164 赞，近期下载 44651 次。
标签：transformers, gguf, nvidia, pytorch, nemotron-3.5, text-generation, en, ar, bg, cs, da, de
README：
NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
Description
NVIDIA-Nemotron-3.5-Bolt-E4B-4GB is a small language model (SLM) further post-trained from Gemma-4-E4B-it by NVIDIA. Nemotron-3.5-Bolt-E4B carries forward Gemma’s strong conversational capabilities and further improves multi-turn tool use, instruction following, and input grounding for fast local voice/device assistants, interactive personas, and embedded/edge interfaces. Designed for low-memory, low-TTFT operation, the model allows agents to respond quickly while preserving headroom for applications or adjacent model pipelines.
For speculative decoding, use the companion DFlash drafter. For the full-precision (BF16) version, see NVIDIA-Nemotron-3.5-Bolt-E4B-BF16.
This model is ready for commercial or non‑commercial use.
Model Summary
|
|
| Total Parameters |
8B (4B effective) |
| Architecture |
Gemma-4-E4B |
| Precision |
Q4_K_M |
| Context Length |
Input: up to 64K tokens; output: up to 64K tokens |
| Inputs / Outputs |
Text / text |
| Deployment |
llama.cpp (recipe pending) |
| Reasoning Mode |
Chat-template controls (enable_thinking=True/False) and runtime thinking budget |
| Conversational Mode |
Chat-template controls (conversational_mode=True/False) |
| Recommended Sampling |
Temperature 1.0, top_p 0.95 in both thinking modes |
| Best For |
Local voice and device assistants, interactive personas, and embedded interfaces |
| License |
OpenMDW-1.1 |
Model Overview
Model Developer: NVIDIA Corporation
Model Dates: December 2025 - May 2026
Data Freshness:
- The post-training data has a cutoff date of May 2026.
What is Nemotron?
NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.
Quick Start
export MODEL_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB
export MODEL_FILE=Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf
export DRAFTER_REPO=nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash
export DRAFTER_FILE=Nemotron-3.5-Bolt-E4B-DFlash-BF16.gguf
export MODEL_DIR="$PWD/bolt-models"
hf download "$MODEL_REPO" "$MODEL_FILE" \
--local-dir "$MODEL_DIR"
hf download "$DRAFTER_REPO" "$DRAFTER_FILE" \
--local-dir "$MODEL_DIR"
Without drafter
docker run --rm --gpus all \
-p 8000:8000 \
-v "$MODEL_DIR:/models:ro" \
ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
-m "/models/$MODEL_FILE" \
--alias nemotron-bolt \
--host 0.0.0.0 \
--port 8000 \
-c 65536 \
-ngl 999 \
--jinja
With DFlash drafter
docker run --rm --gpus all \
-p 8000:8000 \
-v "$MODEL_DIR:/models:ro" \
ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
-m "/models/$MODEL_FILE" \
--spec-draft-model "/models/$DRAFTER_FILE" \
--spec-type draft-dflash \
--spec-draft-n-max 7 \
--spec-draft-ngl 999 \
--alias nemotron-bolt \
--host 0.0.0.0 \
--port 8000 \
-c 65536 \
-ngl 999 \
--jinja
curl --fail-with-body -sS \
http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "nemotron-bolt",
"messages": [
{"role": "user", "content": "What is 17 multiplied by 23?"}
],
"temperature": 0.0,
"max_tokens": 512,
"chat_template_kwargs": {
"enable_thinking": true
}
}'
For more details on how to deploy and use the model — see the Quick Start Guide below!
License/Terms of Use
Governing Terms: Use of this model is governed by the [OpenMDW License Agreement, version 1.1](https://github.com/O
The original belongs to Hugging Face 热门模型 and its authors and is reproduced here for reading. To have it removed, write to support@aisa.one.
Studio