A Nemotron 3.5 Bolt Hermes setup means running NVIDIA's new Nemotron 3.5 Bolt model, a 74B-parameter local AI model squeezed into a 2-bit build, as the brain of Hermes Agent on your own machine.
Hermes added it to its local runtime in v0.22.3, released on 21 October 2026, so you can pick it from Settings, then Providers, then Local Models, or from hermes model in a terminal.
NVIDIA says Bolt has 74B total parameters with 9B active, sees images and video, fits in 24GB of memory and is licensed under OpenMDW-1.1.
NVIDIA's own recipes run it on RTX Spark machines with 64GB or 128GB, or on RTX graphics cards with 32GB or more of memory.
This page is the operator walkthrough, so you'll get the verified specs, the hardware check, the four setup steps in Hermes, the manual llama.cpp route and the cloud route for its big brother, Nemotron 3.5 Super VL.
If you want my team to look at how AI agents and AI search can bring your business more leads, book a free SEO strategy session with my team.
Nemotron 3.5 Bolt Hermes At A Glance
I checked every fact here on NVIDIA's own model cards, NVIDIA's technical blog, the Hermes docs and the official Hermes release notes.
| Question | What the official sources say |
|---|---|
| What is it? | It is NVIDIA's vision-language model for running everyday agentic tasks locally. |
| Full name | NVIDIA calls it NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB. |
| Size | It has 74B total parameters and 9B active parameters. |
| Quantisation | It uses native 2-bit quantisation with quantisation-aware training, sold as IQ2. |
| Memory target | NVIDIA designed it to fit in 24GB of memory. |
| Context | It supports up to 256K tokens. |
| Inputs | It takes text, images and video, and it outputs text. |
| Licence | It is licensed under OpenMDW-1.1 for commercial or non-commercial use. |
| Release date | NVIDIA's cards and blog give 15 October 2026. |
| Hermes support | Hermes v0.22.3 added it to the local runtime on 21 October 2026. |
What Nemotron 3.5 Bolt Actually Is
Nemotron 3.5 Bolt is a smaller, faster version of NVIDIA's bigger Nemotron 3.5 Super VL model.
NVIDIA's model card says it was compressed from the 120B-A12B Super VL model using an NVIDIA method called Star Elastic.
The goal is written right on the card, which is to run everyday agentic tasks locally and run alongside your other applications.
NVIDIA's technical blog says the team designed it to fit into 24GB of memory and decode at a speed of at least 80 tokens per second.
The architecture is a Mamba-2 and Transformer hybrid mixture-of-experts model with a C-RADIO v4-H vision encoder, so it can look at screenshots, images and short videos.
The card says it's optimised for local agent harnesses and agentic tasks involving tool calling, web browsing and terminal use.
That's exactly what Hermes Agent does, which is why this pairing makes sense.
One small detail I found interesting is that NVIDIA's model card lists synthetic Hermes Agent reasoning traces among its training data.
🔥 Want my exact Nemotron 3.5 Bolt Hermes Setup setup, step by step? Inside the AI Profit Boardroom you get step-by-step video trainings, my installable Agent OS and four coaching calls a week with 3,400+ members building real automations. → Join the AI Profit Boardroom
The Hermes Side: What Nous Shipped
The Hermes v0.22.3 release notes, dated 21 October 2026, list "Nemotron 3.5 Bolt and one-slot DFlash drafts in the local runtime".
Those notes also say this part of the window is undocumented on purpose, and the full curated notes ship with v0.23.0.
So I went to the source, and the local runtime catalog file in the Hermes repo at the v0.22.4 tag has a Nemotron 3.5 Bolt entry.
| Catalog detail | What the Hermes catalog file says |
|---|---|
| Display name | The model shows as Nemotron 3.5 Bolt. |
| Description | Hermes describes it as NVIDIA's 74B mixture-of-experts in a 2-bit build that sees images. |
| Source | It downloads from NVIDIA's own GGUF repo on Hugging Face. |
| Model file | The IQ2 model file is about 22 GB. |
| Vision file | A vision projector of about 1.7 GB comes with it. |
| Draft model | A LiLiCorr draft model of about 1 GB is set up for DFlash speculative decoding. |
| Recommendation | The entry is set so Hermes doesn't auto-recommend it, which means you pick it yourself. |
The draft model is the speed trick, because a small model guesses the next few tokens and the big model checks them in one go.
Step 1: Check Your Hardware First
This is the step that decides whether Bolt is for you, so don't skip it.
NVIDIA's GGUF model card gives two llama.cpp recipes.
| Hardware | NVIDIA's recipe |
|---|---|
| RTX Spark 128GB or an RTX graphics card with 48GB+ | The 32GB recipe uses a 262,144-token context and up to 7 draft tokens. |
| RTX Spark 64GB or an RTX graphics card with 32GB+ | The 28GB recipe uses a 65,536-token context, up to 6 draft tokens and a q8_0 KV cache. |
The same card lists NVIDIA Ada Lovelace and Blackwell GPUs as supported, with Windows and Linux as the supported operating systems.
Here's the bit that confuses people, so I'll be straight about it.
The model is designed to fit in 24GB, but NVIDIA's own recipes are written for 32GB-plus cards and 64GB-plus Spark machines, because the context, the vision file and the draft model all need room too.
NVIDIA's card doesn't mention Macs, so I can't tell you how it runs on Apple Silicon.
The good news is that Hermes checks this for you, because the docs say every catalog model is priced against your machine before you download it.
Step 2: Update Hermes To v0.22.3 Or Newer
Bolt only appears in the local runtime from v0.22.3.
For a command-line install, the release notes say to run this.
hermes update
If you use the Microsoft Store app, the notes say it updates through the Store.
If you run the Docker image, the notes point you to nousresearch/hermes-agent:stable.
You can check which version you're on with this command.
hermes --version
🔥 Want my exact Hermes setup without the guesswork? Inside the AI Profit Boardroom, I've got a full Hermes section with step-by-step video tutorials, plus the installable Agent OS I run every day. You'll be learning alongside 3,400+ members with four coaching calls every week. → Get access here
Step 3: Install The Runtime And Download Bolt
In the desktop app, the Hermes docs give this flow.
- Open Settings, then Providers, then Local Models.
- Click Install runtime, and Hermes downloads the official llama.cpp build for your hardware.
- Find Nemotron 3.5 Bolt in the catalog and click Download.
- Click Use, and new chats run on the local model.
In a terminal, the docs say hermes model then Local models runs the same setup.
hermes model
The docs say that menu lists the catalog priced for your machine, installs the pinned engine, downloads the model with live progress, starts the server and makes the model your default.
Downloads pause and resume, and the docs say Ctrl+C pauses a terminal download while choosing the model again resumes it.
Plan for roughly 25 GB of downloads in total, because the model, the vision file and the draft model all come down together.
Step 4: Read The Memory Fit Before You Click Use
Every catalog row in Hermes shows a memory fit colour, according to the docs.
| Colour | What it means |
|---|---|
| Green | The model fits your GPU and runs entirely in GPU memory. |
| Amber | The model uses system RAM, so it works but runs slower. |
| Red | The model is too big for this machine. |
On Windows, the docs say local models run on CUDA for supported NVIDIA targets, on Vulkan on x64, or on the CPU.
On Linux, the docs say CUDA needs a CUDA 13 driver on x64 or arm64 NVIDIA cards.
If Bolt shows red, it won't run on that machine, and Hermes will still show you the reason.
If you'd like my team to map out where AI agents fit your business, grab a free strategy session with my team and we'll look at your site, your competitors and the keywords you're closest to winning.
🚀 Ready to put Nemotron 3.5 Bolt Hermes Setup to work in your business? Inside the AI Profit Boardroom you get step-by-step video trainings, my installable Agent OS and four coaching calls a week with 3,400+ members building real automations. → Join the AI Profit Boardroom
The Manual Route: Run NVIDIA's llama.cpp Recipe Yourself
Some operators like to run the server themselves, and Hermes supports that.
NVIDIA's model card gives this Linux recipe, which it calls the 28GB recipe, for RTX Spark 64GB machines and RTX cards with 32GB or more, after you've downloaded the model, vision file and drafter into the paths in its environment variables.
llama-server \
--model "$MODEL" \
--mmproj "$MMPROJ" \
--batch-size 2048 \
--ubatch-size 512 \
--ctx-size 65536 \
--parallel 1 \
-ngl 999 \
--spec-type draft-dflash \
--spec-draft-n-max 6 \
--spec-draft-model "$SPEC_DRAFT" \
--spec-draft-ngl 999 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--temp 1.0 \
--top-p 0.95 \
--flash-attn on \
--jinja
The card says to use the latest llama.cpp built from source for this.
The Hermes docs say that if a llama-server is already running on your machine, Hermes detects it and uses it instead of starting its own, and the default probe is port 8080.
If your server is on another port, the docs say you can list it under local_runtime.detect_ports or define it under providers in config.yaml.
providers:
llamacpp:
base_url: http://127.0.0.1:8081/v1
model: my-model
The Cloud Route: Nemotron 3.5 Super VL Through NVIDIA NIM
If your hardware can't run Bolt, its parent model is the next step up.
NVIDIA's model card says Nemotron 3.5 Super VL has 120B total parameters with 12B active, supports up to 1M tokens of context and was released on 15 October 2026.
The Hermes v0.22.1 and v0.22.2 notes added Nemotron Super 3.5 default parameters, added Nemotron to the NVIDIA NIM default model list and switched to the released name, Nemotron 3.5 Super VL.
The Hermes docs say to create a free API key at build.nvidia.com and add it to ~/.hermes/.env like this.
NVIDIA_API_KEY=nvapi-...
Then you start a chat with this command from the docs.
hermes chat --provider nvidia --model nvidia/nemotron-3.5-super-vl-120b-a12b
For a local NIM on a DGX Spark, a DGX Station or a local GPU, the docs say to set the endpoint instead.
NVIDIA_API_KEY=local-no-auth
NVIDIA_BASE_URL=http://localhost:8000/v1
To make it your default, the docs give this config.yaml block.
model:
provider: "nvidia"
default: "nvidia/nemotron-3.5-super-vl-120b-a12b"
What NVIDIA Says Bolt Can Do
These are NVIDIA's own published numbers, not mine, so treat them as the maker's claims.
| Benchmark | Nemotron 3.5 Bolt (IQ2) | Qwen3.6-35B-A3B | Gemma 4 26B-A4B |
|---|---|---|---|
| Terminal-Bench 2.1 | 50.98 | 44.38 | 37.22 |
| BrowseComp | 66.75 | 48.74 | 26.30 |
| GPQA Diamond | 86.43 | 83.40 | 79.61 |
| PinchBench | 87.82 | 88.07 | 74.70 |
NVIDIA's blog also reports 83.99 tokens per second on its coding speed test, measured with llama.cpp on an RTX Spark laptop with 128GB.
In the same table, Qwen3.6 35B-A3B and Gemma 4 26B-A4B came out faster on that speed test, so Bolt's pitch is more intelligence per GB rather than raw speed.
Limits And Honest Caveats
The Bolt support in Hermes is brand new, and Nous says this part of the release is undocumented until v0.23.0.
The Hermes Local Models docs still say Hermes never offers builds below 4-bit, while Bolt is NVIDIA's own 2-bit build, so the docs haven't caught up with the catalog yet.
The 24GB name is the model's memory target, not the size of card NVIDIA's recipes ask for.
NVIDIA's blog lists the checkpoints under the name "Nemotron 3.5 Super VL 74B-A9B", while the model cards call them Bolt, so search for the Bolt name on Hugging Face to avoid confusion.
I haven't run Bolt myself, so everything here comes from NVIDIA's pages, the Hermes docs, the Hermes catalog file and the official release notes.
What I have done is run Hermes on a free local model, which I showed in my Gemma 4 video below using Ollama and a custom endpoint.
🔥 Want to build a local agent crew that runs while you sleep? The AI Profit Boardroom has the Agent OS, the Hermes and local model training, and four weekly coaching calls where you can show me your setup and get a straight answer. → Join the AI Profit Boardroom
Who Should Set Up Nemotron 3.5 Bolt In Hermes
Operators with an RTX card that has 32GB or more, or an RTX or DGX Spark machine, should try it first.
Anyone who wants a private agent that can read screenshots and run terminal tasks without sending data to the cloud should look at it closely.
Anyone on a smaller card should start with a smaller catalog model, or use Super VL through NVIDIA NIM instead.
If you're still choosing your main brain, my guide to the best Hermes Agent model compares the options, and my Solar Mini 4 Hermes guide covers another free model for Hermes.
Related Reading
- My Hermes Agent v0.22 walkthrough explains the release that brought one-click local models to Hermes.
- My best Hermes Agent model guide helps you choose the brain for your agent.
- My Hermes Agent Microsoft Store guide shows the easiest way to get Hermes on a Windows PC with an NVIDIA card.
Also On Our Network
- 🌐 Six ways a business can use Nemotron 3.5 Bolt inside Hermes
- 🌐 Every Nemotron 3.5 Bolt spec checked and scored
- 🌐 The Nemotron 3.5 Bolt quick start for an Agent OS
- 🌐 Whether any Nemotron model is on GoldieBench yet
FAQ: Nemotron 3.5 Bolt Hermes Setup
What is Nemotron 3.5 Bolt?
Nemotron 3.5 Bolt 74B-A9B 24GB is an NVIDIA vision-language model built to run everyday agentic tasks locally.
NVIDIA's card says it has 74B total and 9B active parameters, uses native 2-bit quantisation to fit in 24GB and supports up to 256K tokens.
How do I use Nemotron 3.5 Bolt in Hermes?
Update Hermes to v0.22.3 or newer, then open Settings, Providers, Local Models, install the runtime, pick Nemotron 3.5 Bolt, download it and click Use.
In a terminal, hermes model then Local models runs the same setup.
What hardware does Nemotron 3.5 Bolt need?
NVIDIA's GGUF card lists recipes for RTX Spark with 64GB or 128GB and for RTX graphics cards with 32GB or 48GB and up.
It lists NVIDIA Ada Lovelace and Blackwell GPUs, with Windows and Linux.
When was Nemotron 3.5 Bolt released?
NVIDIA's model cards and technical blog give 15 October 2026.
Hermes added it to its local runtime in v0.22.3 on 21 October 2026.
How do I use Nemotron 3.5 Super VL in Hermes?
The Hermes docs use the NVIDIA NIM provider with an NVIDIA_API_KEY from build.nvidia.com in ~/.hermes/.env.
Then you run hermes chat --provider nvidia --model nvidia/nemotron-3.5-super-vl-120b-a12b.
If you'd like my team to look at how AI agents and content can win your business Google and AI search traffic, book your free SEO strategy session here.
📺 Video notes + links to the tools 👉
🎥 Learn how I make these videos 👉
🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉
About Julian
I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.
I help business owners scale with AI agents, automation and SEO.
- I've built a 7-figure SEO and link-building agency, Goldie Agency, with 70+ people in Bangkok.
- I've grown a YouTube channel to 400,000+ subscribers.
- I wrote the book "Link Building Mastery", which covers nine link building strategies.
- I run GoldieBench, where I score AI models on real one-shot builds.
→ Get my best AI training inside the AI Profit Boardroom
Check your card, update Hermes, download it from Local Models and read the memory fit, and your Nemotron 3.5 Bolt Hermes setup will be running on your own hardware.
