Hermes Agent prompt caching got a single lifecycle controller in v0.23.3 on 20 November 2026, and this guide shows exactly how it warms, compacts and expires your cache, and which settings to touch.
Prompt caching lets the model provider reuse the start of your prompt (system prompt, skills, earlier turns) at a discounted read rate instead of reprocessing it every time.
The v0.23.3 release notes list four changes: warming through long tool runs, compaction of idle chats before the one-hour cache expires, Nous Portal on the full one-hour lifecycle, and prompt_caching.cache_ttl defaulting to auto.
This builds on v0.23.2 from 16 November, which introduced one prompt-cache clock per session.
The config block
prompt_caching:
cache_ttl: "auto" # default; or "5m" / "1h"; "off" disables
route_ttl_ceilings: {} # e.g. {nous: "5m"}
Per the configuration docs, "5m" and "1h" are sent as-is, any off value disables caching, and any other value falls back to "5m".
Caching turns on automatically when the provider supports it; there is nothing to enable.
What "auto" actually does
- A session you type into (CLI, TUI, Desktop, Telegram, Discord, Slack and other chat platforms) starts on the 5-minute tier.
- It moves to the 1-hour tier from the first message you send after a pause of more than five minutes, and stays there for the session.
/newstarts on 5 minutes again.- Machine-paced sessions stay on 5 minutes: subagents, cron,
hermes -qone-shots, webhooks, Kanban workers and the API server. - Delegated subagents are always clamped to 5 minutes.
The reason: on Anthropic-style caching a 1-hour write costs 2x the base input rate and a 5-minute write 1.25x, so the 1-hour tier only pays off when turns are more than five minutes apart (see Anthropic's prompt caching docs).
The Hermes docs report that on one install with parked and resumed sessions, auto cut the interactive cache-write bill by roughly 40 percent.
Step 1: keep the default
Leave cache_ttl: "auto" unless you have a reason to pin.
Pin "1h" only if every session has long gaps; pin "off" for OAuth plans that bill cache writes as extra usage or proxies that add their own cache_control.
Step 2: read the cache clock
Run /context in a session.
It prints lines like Prompt cache: warm until 14:32 (58m 10s left; 1h, proven).
Rank in AI Search or It's FREE →
"Proven" means the response named the tier; "declared" is what was requested; "assumed" or "unknown" means Hermes will not warm or compact at the cache edge.
🔥 Want Hermes tuned for long agent runs? Inside the AI Profit Boardroom you get step-by-step video trainings, the installable Agent OS, four coaching calls a week and 3,400+ members running the same agents. → Join the AI Profit Boardroom
How warming works
At five sixths of the cache lifetime (about 4m10s on 5 minutes, 25 minutes on 30, 50 minutes on 1 hour) Hermes re-sends the last request with a one-token output cap.
While the chat is idle it does this at most three times per message you send.
While a tool call is running it keeps warming until the tool returns, bounded only by cost: the keep-alives never cost more than the single full re-send they prevent.
Keep-alives need known cache prices and a cached prompt of at least 1,024 tokens, and are never sent for budget-based extended thinking.
How compaction at the cache edge works
When the idle keep-alives run out, Hermes compacts the chat if the cache window is at least 30 minutes, the context is at least half the compaction threshold and the chat grew since its last compaction.
Removed tool results are saved byte for byte under the cache/pruned folder inside your Hermes home with a pointer left in place.
Expire and offload
Otherwise the entry expires and old tool results are offloaded to disk, so the chat is already small when you come back.
Provider differences
| Route | Cache behaviour per the docs |
|---|---|
| Claude (Anthropic API, OpenRouter, Nous Portal) | Cache markers; 5m or 1h as requested, capped by route ceiling |
| Nous Portal | Ships a 1h ceiling |
| Qwen Cloud | Capped at 5 minutes |
| OpenAI-compatible routes | Automatic caching, treated as 30 minutes |
| Responses API (openai-codex, xAI) | Never warmed |
| Local endpoints (llama.cpp, vLLM, Ollama) | Never expire on the clock |
Route ceilings
Use route_ttl_ceilings to cap a provider that keeps less than it is asked for, for example {nous: "5m"}; a ceiling also sets the warming schedule.
Where the cache breakpoints go
Anthropic allows at most four cache_control breakpoints per request, and Hermes uses a strategy the developer guide calls system_and_3.
Join the AI Profit Boardroom →
- Breakpoint 1: the system prompt, stable across all turns.
- Breakpoints 2 to 4: the third-to-last, second-to-last and last non-system messages, as a rolling window.
The marker is {"type": "ephemeral"}, with "ttl": "1h" added for the one-hour tier.
For string content Hermes converts the message into a text block carrying the marker; for list content it adds the marker to the last element; tool messages carry it directly on native Anthropic.
What breaks the cache
Cache hits need an exact prefix match, so anything that changes earlier content invalidates everything after it.
- Editing the system prompt mid-session: breakpoint 1 changes and the whole prefix is rewritten.
- Inserting or removing earlier messages: the prefix no longer matches from that point on.
- Switching model: the guide is explicit that model identity is part of the cache key, so a
/modelswitch, a fallback to another model or a credential-pool rotation onto another account gives zero cache hits on the next request. - Compression: the compressed region loses its cache, but the system prompt cache survives and the rolling window rebuilds within one or two turns.
If cost spikes after a fallback fires, this is why.
Human-paced vs machine-paced sessions
Under auto, the TTL resolves once per session from the session's source.
Machine-paced sources stay on 5 minutes: subagent, cron, oneshot, webhook, kanban, api, api_server, tool, batch and curator.
Side agents started with /background, /btw or /refine also take 5 minutes, and a fork routed to a different model is clamped to 5 minutes like a delegated child.
Human-paced sessions, where you take breaks between turns, are the ones that benefit from the one-hour tier.
How compaction and caching interact
Hermes runs two compression layers, per the same guide.
The agent's own context compressor fires at 50 percent of the context window by default, inside the tool loop, using real token counts from the API.
A gateway safety net fires at 85 percent before the agent sees a message, to catch sessions that grew overnight in Telegram or Discord.
The v0.23.3 change adds cache-edge compaction on top: when idle keep-alives are spent, Hermes compacts the chat before the one-hour cache expires, so the next turn starts small instead of re-sending a cold, full history.
Replay size and warming cost
A keep-alive replays the last request with a tiny output cap: one token for most models, 16 for OpenAI GPT-5 and o-series models.
Warming decisions happen once at five sixths of the TTL, at least 10 seconds before expiry.
Pricing uses the catalog's one-hour write rate when the catalog lists one, otherwise a fixed 2x multiple, per the guide.
Anthropic's own prompt caching docs explain the read and write pricing behind those multiples.
Cost-saving habits
- Pick one model per session and avoid mid-chat switches.
- Keep skills and system prompt edits for new sessions.
- Use
/newfor a new topic rather than dragging a long history along. - Leave
cache_ttlonautounless you have measured a reason to pin. - Check
/contextafter long tool runs to confirm the cache stayed warm.
OpenAI-style routes and long retention
The developer guide splits routes into two specs.
A request carrying cache_control markers is treated as Anthropic-spec on any wire or gateway, with 5m or 1h declared as asked and clamped by the route ceiling.
Every other cached request is OpenAI-spec: declared at 30 minutes, with each read refreshing the window to the later of the existing end or read time plus 30 minutes.
A request sent with prompt_cache_retention: 24h keeps its warming deadline at 30 minutes but its estimated end at 24 hours, so it is not pruned cold on the clock; the guide notes chatgpt.com Codex strips that field, so 30 minutes stands there.
Learned routes
Hermes also learns how long each route really keeps a cache, across sessions.
It records the gap between requests and whether the cache was still there, and only adopts a learned rule after enough agreeing samples from distinct sessions.
Learned facts expire over time, so a provider change does not lock in stale behaviour.
This sits under the Hermes home directory and needs no configuration.
Verify savings in the API usage fields
Anthropic responses report cache_creation_input_tokens and cache_read_input_tokens in the usage block, per the prompt caching docs.
On a warm session most input should show up as cache reads; a turn dominated by cache creation means the prefix was rewritten.
Rank in AI Search or It's FREE →
Line those fields up against /context output to confirm the controller is doing what it reports.
Nous Portal and catalog markers
v0.23.3 puts Nous Portal on the full one-hour lifecycle, per the release notes.
The developer guide adds that a provider profile can let its /models catalog decide which models get cache markers; for Nous, a listed cache-write rate means markers are sent.
If a Nous route keeps less than it is asked for on your account, cap it with route_ttl_ceilings so warming follows the real lifetime instead of the requested one.
A working session pattern
- Start a session with
/newand pick the model once. - Load the skills you need at the start, not halfway through.
- Let long tool runs finish; the controller keeps the cache warm meanwhile.
- Step away freely; idle keep-alives and cache-edge compaction handle the gap.
- Start a fresh session for the next unrelated job.
That pattern lines up with how the controller is designed and gives the highest cache read share.
Troubleshooting
Keep-alives: off (missed twice): the cache was gone twice in a row; a/modelswitch restarts warming.unknown: a model switch, fallback or compaction moved the cache.- High cache-write cost on subagents: expected, they stay on 5 minutes.
For model picks that cache well, see our best Hermes Agent model guide, and learn the full setup in the AI Profit Boardroom.
Join the AI Profit Boardroom →
FAQ: Hermes Agent Prompt Caching
Is prompt caching on by default in Hermes?
Yes. Hermes turns it on automatically when the provider supports it, and cache_ttl defaults to auto.
What does cache_ttl auto do?
Typed sessions start on the 5-minute tier and move to 1 hour after your first pause of more than five minutes; machine-paced sessions stay on 5 minutes.
How do I turn prompt caching off?
Set prompt_caching.cache_ttl to "off" in config.yaml.
Why does Hermes send tiny requests while a tool runs?
Those are keep-alives that keep the cache warm so the next turn reads the cache instead of re-sending everything.
How do I see the cache state?
Run /context; it shows when the cache stays warm and whether the tier is proven, declared or assumed.
📺 Video notes + links to the tools 👉
🎥 Learn how I make these videos 👉
🆓 Get a FREE AI Course + Community + 1,000 AI Agents 👉
About Julian
I'm Julian Goldie, an SEO entrepreneur, author and founder of the AI Profit Boardroom, which has 3,400+ members.
I help business owners scale with AI agents, automation and SEO.
- I've built a 7-figure SEO and link building agency, Goldie Agency, with a team of 70+ SEO experts.
- I've grown a YouTube channel to 400,000+ subscribers.
- I wrote "Link Building Mastery".
- My Udemy courses have taught over 29,000 students.
→ Get my best AI training inside the AI Profit Boardroom
Update to v0.23.3, leave cache_ttl on auto and check /context after your next long tool run.
For guided setups, join the AI Profit Boardroom, and keep this page as your Hermes Agent prompt caching reference.











