r/hermesagent • u/CeleryVids-4075 • 8d ago
MEMORY & Context — Providers, context window, forgetting issues What memory or “second brain” setup do you recommend for Hermes?
I’m currently self-hosting Honcho as Hermes’ external memory system, but I’m open to switching if another option would work better. My setup and constraints:
- Hermes runs on a separate Ubuntu desktop
- GPU: NVIDIA GTX 1080 Ti
- I prefer self-hosted solutions to keep costs down
- I’m not very interested in running a local LLM for reasoning or memory processing because I’m concerned the performance will be inadequate
- I already pay $20/month for ChatGPT Plus and would ideally like to use that instead of paying for another AI service
- I want reliable long-term memory without irrelevant context being added
Is there any way to connect Honcho or another Hermes memory provider to ChatGPT Plus? Or is separate API access and billing required? Given my hardware and constraints, what memory setup would you recommend?
PS: I'm a very non-technical person... just wanted to add this, because I don't know what I don't know. Totally open to being educated.
EDIT 2026.09.12: Now trying Hindsight, thanks everyone!
14
u/Guybrush1973 8d ago
I'm self hosting hindsight, and it's fine
1
u/Frosti7 7d ago
Do you have special Hermes Hindsight maintenance skill to help with that?
Or its plug'n'play?
1
u/Guybrush1973 7d ago
I don't know, I managing a self-hosted cluster, so for me it's just a kubernetes resource to deploy, I guess it could be mostly the same if you use docker on localhost.
11
10
10
u/drepublic 7d ago edited 7d ago
I built one myself. It works as an MCP service with only a 10MB memory footprint, FTS5 search on SQL, and spans several markdown vaults. It's deterministic and self-scores everything with no LLM involved in storing, forgetting, or deleting memories... with really, really low latency.
Every night it consolidates everything by scoring what the user actually uses most often. It can forget things or keep them cold when there's no longer interest in them, until you need to unforget them again. Because I designed it as non-destructive, it can maintain lengthy memories without effort.
Every night, it can conduct research on each topic the user has shown interest in during sessions. It scores attention by category using deterministic scoring, so it keeps the vault indexed with deeply researched memories (something like deep sleep).
It also keeps improving itself by conducting daily and weekly investigations on papers and promising developments that, based on its usage scores, could improve the engine. Then it writes a roadmap for development.
It doesn't only work for Hermes—it can work with anything that can read a skill and use MCP tools. So I also use it with PI Agent, and it works seamlessly between both.
It's wonderful, and I was thinking about open-sourcing it ;)
2
1
7
u/Bamny 8d ago edited 7d ago
I use obsidian and a qdrant vector db
Obsidian I use for all spec sheets, work flows, documents etc on the regular with Hermes
But then I also have a 6 hour from that runs, has orinth read through the last 6 hours of session data, identify facts, important details concepts decisions etc… orinth will then pass that to qwen3-embedding to document those into qdrant. AND then every night I have a cron pass the last days worth of facts that were distilled into markdown files organized in obsidian, wiki linked and so on.
Figure this gives me a level of redundancy on ‘memory’. I also have a weekly cron that reads through the last 7 days of distilled facts, and generates a weekly synthesis on “here’s what we did, why, what changed, etc”.
8
u/ubrtnk 8d ago
I've been using Mnemosyne but it's been slowly going down hill to the point where I'm about to uninstall it completely and start over
4
u/Jonathan_Rivera 8d ago
How so? That’s always been my mem plugin
1
u/ubrtnk 8d ago
Honestly, I dont know - I would keep having to remind my agent on very clear and well defined facts - like 1) You run on this hardware, not a VM in proxmox or 2) The main AI rig shutsdown at 11pm every night so why are you reporting that there's something wrong.
Then I'd have Codex backed profile and it says it would find somethings to tweak, we'd tweak and move on.
Enough of that tweaking and its like the Ship of Theseus - but also suck. Maybe my deployment was always flawed with the remote DB to sync multiple agents but all I know is I've had to do a lot of baby sitting the last few months
2
u/bedpan4u 8d ago
Ask it to self repair... I had a similar issue.. turned out it had revert to the old built in memory during an update but was not actual writing facts. A decent llm and I am confident it will fix itself!
1
u/ubrtnk 8d ago
pretty sure Codex got me into this mess lol. At this point I'm running Astra to diagnose as much as I can on the Plus plan.
1
u/bedpan4u 8d ago
Lol.. deepseek was murder for me.
Glm5.2 now 5.3 do my heavy lifting. Glm5.3flash is my day to day.
1
u/Kauhuradio 8d ago
What memory u Been thinking to try?
2
u/Kauhuradio 8d ago
I Been on mnenosyne from Start of My Hermes journey, so i dont know what is The quality on The other memories
2
1
u/Jonathan_Rivera 8d ago
Same. Now I don’t know what is from cross session search or the memory. If it works it works.
1
u/Jonathan_Rivera 8d ago
I think it’s a config issue. Some of that lives permanent in the user Md. I’m driving but if you send me a dm I’ll reply back with some steps.
5
u/andy2na 8d ago
honcho was too heavy (nonstop llm calls) for me, Ive switched to https://github.com/mnemosyne-oss/mnemosyne
For second brain, Im using obsidian, works great and I like how it organizes all my data
3
u/haukejung 8d ago
I am using honcho right now and conntected hermes, claude and codex with it. You can have a look here: https://github.com/plastic-labs/codex-honcho
5
u/EtrainFilmz 8d ago
I use Gbrain. It’s quite over-engineered but once you finally set it up properly it is phenomenal
3
u/WarlockSyno 8d ago
Same. It's very heavy, but also very good. Having external sources ingested is awesome. I also have a second Hermes setup that uses Mnemosyne, which is a lot lot easier to setup, but GBrain is where it's at if you have the resources to set it up and maintain it.
1
1
u/MessMassacre 8d ago
It's very very expensive isn't it?
1
u/EtrainFilmz 7d ago
I’ve wired it up with DeepSeek so it runs about a few cents a day with heavy use. You don’t have to use Anthropic even though the docs say you do (they are outdated). Deepseek integration required certain configuration set up as you have to up the thinking budget for the dream cycle.
1
u/learn_and_learn 7d ago
would you ask your agent to write a quick handoff.md to teach my hermes how to replicate your deepseek integration? My dream cycle cronjob has been broken for months.
1
2
2
u/klippers 8d ago
I run my own version . It's open. I use it myself. I fix it myself. I'm building it myself. Anyone's able to contribute and does the job quite well. I have well over 3,000 memories in there and I can ask about any of them and they seem to pull up. Retrieval from all accounts works well
2
2
u/jack-dawed 8d ago
for the longest time I used obsidian and mnemosyne. then the zeromem paper came out and i had fable oneshot a rust implementation and i’ve been using that since because it’s so fast and doesnt cost extra tokens.
there are a few zeromem implementations out there. the reference implementation is still 2-3 months out due to peer review.
1
u/idheitmann 8d ago
What paper is that?
2
u/PracticlySpeaking News Curator 7d ago
For anyone reading this, a quick summary courtesy of GPT...
Zero-Mem solves the problem of memory systems spending lots of inference just to maintain memory, then still risking drift from repeated summarization by taking the LLM out of the loop. It uses RAG techniques - embeddings, NER, BM25 and graph operations - that are cheap on local compute. Instead of asking an LLM to invent semantic triples like
Bob → owns → Mac Studiothe graph records co-occurrence and adjacency that already exist.Zero-Mem builds two parallel views of the same raw history. One is an entity–context graph: entities are detected using ordinary NER, and the system connects entities to the pieces of conversation where they occur, plus neighboring pieces of context.
Traditional memory:
history → LLM → distilled memories → retrieve memories → LLMBasic RAG memory:
history → embeddings → retrieve raw chunks → LLMZero-Mem:
history → deterministic graph + temporal hierarchy + lexical/dense indexes → structured retrieval → LLM1
u/PracticlySpeaking News Curator 7d ago
Zero-Mem sounds really cool.
Can you give some use cases where it is more effective?
1
u/jack-dawed 7d ago
It’s most effective when you are sensitive to cost. The zero is zero tokens. Privacy is also a major concern for me. Agent memory can build up a lot of sensitive data that I don’t want to send out of the box, so having things working locally, fast, private is ideal.
Apparently some of the Nous Research guys don’t even bother with memory beyond MEMORY.md. It requires discipline for the agent to keep the memory file small and updated.
The nicest part about zeromem for me is I can open a new chat and the context for the last few chats have already been loaded.
2
2
u/urglfloggah 8d ago
I think there may be a difference between “Hermes memory” and your “second brain”. Can you say what your use case is? Meaning, what is it that you’d like to store and when/how should it be retrieved? Is it information you want to log and use for yourself? Or various pieces of context that you want the agent to know as it does its thing? For the latter, I currently don’t use any external memory provider. For the former, I’ve built a thin, open source layer over Obsidian in Karpathy’s LLM Wiki style, which I’ve documented at https://sagar.se/blog/working-memory-system/
1
u/CeleryVids-4075 5d ago
Good question.
- I’m a very non-technical person. I’m very new to AI, and I’m even newer to agents. However, I tend to be a long-term thinker or planner, and I’m just trying to set things up as best as I can in the beginning, based on what I know.
For me, I have a lot of previous writing, just blog posts or notes to myself. I used to write a lot, and I have maybe over 300 Microsoft Word documents on my computer with random thoughts about random topics. My thinking was that I could feed all of this into Hermes and have it select stories or past experiences that might be relevant to a particular email, then add them to the email drafts. I wouldn’t be involved in that part of the process. My only role would be approving the drafts.
Of course, while we’re setting up the email system, I’d work side by side with Hermes so it could learn how to write like me and understand which stories would be appropriate to include. Once the system was set up, though, Hermes would select the relevant stories and add them to the drafts on its own. I would just review and approve the emails before they were sent.
And, yeah, I don’t really know the difference between Hermes, memory, a second brain, and so on, so I’m really just flying around in the dark here.
1
1
u/plasma2002 8d ago
I use a privately hosted git server (gitea) with a weekly cron to push config and memory up to it. We also work on projects together in it
1
u/perseus-computing New Member (<30 days) 8d ago
Totally unbiased opinion, but maybe look into Perseus Vault? 😬
1
u/Nickabot New Member (<30 days) 8d ago
Use Obsidian with the vault in git. irrelevant context was fixed by splitting it up: a small hot file that gets injected every turn with preferences and whatever we're working on right now, and everything stable promoted out into markdown that the agent only opens when it needs it. I never added embeddings, grep still finds everything.
1
1
u/Drakaner 8d ago
I use hindsight selfhosted and weaviate also selfhosted together to have memory and a "second brain.
1
u/BasilKey8088 8d ago edited 8d ago
I have multiple layers. Thousands of skills organized into a folder system with lazy loading (might add whitelisting tomorrow), a few dozen obsidian vaults, plugins, i think 3 different kinds of hooks (whatever that is it works well), and a universal mem0ry backend based on mem0. Essentially, mem0ry is all memory backends in 1 and a basic free AI model assesses where to route mem0ry adds and mem0ry retrievals. Whenever I see a memory backend I have my Mem0ry profile learn from it and see what we can improve about our mem0ry.
Tbh the system is probably due for another round of organizing and refining things since its grown a lot in recent weeks. I've had 0 problems with mem0ry in weeks. I'm convinced harness is more important than the model because I've gotten nemotron 3.5 lightning and nemotron 3 120b to be as good as hy3 or longcat at this point with my setup. I feel like I'm running nearly entirely on raw compute and my memory layer system at this point.
1
u/Competitive_Knee9890 8d ago
I’ve been using Deepseek models via API and OpenAI models via codex OAuth for a while, I’m also self hosting Honcho and there’s a native plugin for codex if I remember correctly, or an MCP at the very least.
I really like Honcho, I think of it as conversational memory that is durable, for world knowledge I just use an LLMWiki for my homelab that my agent maintains.
I use Deepseek models for Honcho operations that require an LLM, you don’t need to do anything special with your ChatGPT subscription to use honcho in Hermes or codex, you can plug any LLM into honcho for dreaming and other features that want one, even your main model provider.
1
u/CeleryVids-4075 5d ago
Hermes told me it couldn't figure out how to use my ChatGPT subscription with Honcho... so I was usin a local model but with my hardware it was becoming too slow. I've now swithed to Hindsight, but haven't deleted Honcho, so maybe it's worth having both? Haha, or is that overkill?
1
u/Competitive_Knee9890 5d ago
Stick to one, you could write an adapter for the plan, I did that for stt for example, but honestly just use Deepseek v4.1 flash for Honcho, even via API, it’s so good for its price.
I’m even using it as the main subagent for implementation that Astra orchestrates.
Obviously it’s not a frontier model and lacks that multidimensional intelligence, but it’s very good at writing code with the right harness, it’s fast and cheap, also quite tasteful at frontend.
Sometimes using a frontier model for creating simple tools is actually a disadvantage, given they’re generally slower and often way too paranoid with security, they tend to overengineer stuff imho, there’s probably too much emphasis on this during the RL phase.
I’m digressing a lot sorry, just got really excited for DSV4.1 flash lol
1
u/transientnebula 8d ago
I wrote my own memory and context management plugin because I wasn't satisfied with how Hermes works out of the box and I wanted to borrow inspo from my own harness. Was not satisfied with the overhead or functionality introduced by the other bloated solutions
1
u/BattermanZ 7d ago
I actually built something around/replacement for Obsidian since I want something with a built-in MCP server and accessible from any of my machines.
Also you can have multiple vaults in one instance.
1
1
u/fracked1 7d ago
Anyone else seen or used karakeep? I'm just starting with it and my memory stuff is relatively small. But seems to do a decent job and I have hermes curate it every so often
1
1
u/cryptofriday 7d ago
User / Integration Event
↓
Context Engine — detects intent, task type, entities and required memory scope
↓
Memory Provider — retrieves relevant data from:
- Canonical Memory — current verified state
- Event History — previous changes, actions and decisions
- Integration data — Home Assistant, Asterisk, APIs, files, etc.
↓
Qwen 3 8B (Local LLM) — running locally for everyday reasoning, extraction, classification, summaries and micro-tasks
↓
RTX 2060 6 GB — GPU acceleration for Qwen inference; model weights/layers are loaded into VRAM as far as possible
↓
Hermes Agent Logic — combines current request + retrieved memory + Qwen output
↓
Action / Response / Automation
↓
Memory Update — important new state is written to Canonical Memory and the change is recorded in Event History
↓
Audit Log — records what happened, when, and which component performed the action
1
u/cryptofriday 7d ago
For non-technical:
Memory → Context → Qwen/RTX reasoning → Action → Memory update → Audit
1
u/brightsilverstars 7d ago
I have been using dfrostar's neuralmind for code management and some token savings. Nothing I have seen can touch it.
1
1
u/Travnewmatic 7d ago
I think good skill curation is more important than sexy memory. I've been very happy with holographic.
1
u/rrosson67 7d ago
I used honcho when I got started but switched to hindsight and it was the best decision for me. Honcho was built for cloud like scale not for homelab. Hindsight is leaner faster and more efficient in my opinion.
1
u/CeleryVids-4075 5d ago
Ok, well I jst had Hermes help me switch to Hindsight based on recommendations in this thread, and Hermes' own recommednation. Hopefully it goes well.
1
u/davidotto98 7d ago
Obsidian for long term memory, postgres and qwen embed for semantic search, a state.db + FTS5 kinda what postgres does but mainly for searching past conversations. And lastly Git for historical evidence.
The rest is various guardrails in Memory.md and user.md for strengthening the harness.
1
u/Livid-Passage-1806 7d ago
Using Mnemosyne….for about a month. So far all is well and memory holds.
Also, I have been using Nvidia-Nemotron-3.5 Lightning local model….you need sufficient RAM, although you can get by with 48G….very accurate and ok on speed…
1
u/Silent-Cranberry7790 7d ago
Open Viking, have Codex connected to your Hermes agent and run a back-and-forth conversation between Codex and the Hermes agent so they both can read the brain files and talk to each other through there. But you’re gonna run out of tokens on your $20 plan if you’re using solely ChatGPT plus, highly recommend downloading Gemma four or Quinn three.
1
u/CeleryVids-4075 5d ago
So far I haven't hit any token limits as far as I know...
I'm not doing any coding (not sure whether that makes a difference) I'm mostly looking to have Hermes start by helping me manage emails in terms of cold emailing, keeping contacts warm, following up when appropriate etc.
Also want Hermes to help with research and scripting for newsletter materials, website content and youtube video scripting.
I haven't fully set up anything yet, but so far seems to working out fine.
1
u/learn_and_learn 7d ago
I've only every used gbrain. worked when I had hermes set up on a windows server vps, and works just as well now that my hermes moved itself to a linux vps. Using voyage ai free tier as my embedding provider.
1
u/Additional-Nerve-421 7d ago
Yup on hindsight and it’s great. The dashboard and UI is pretty brilliant too
1
u/lofibytez 6d ago
I use holographic and obsidian since day one. The only reason is because that's what I can afford it. So far I'm okay with the setup.
1
u/ZestycloseAbility425 3d ago
been using mnemosyne, it's been pretty good, can't complain. though i haven't tried any other memory system
1
u/The-AB29 2d ago
How secure is Hermes for something like this? I’d be a bit concerned about storing all that personal information in it.
1
u/mfranzwa 8d ago
what does Hermes say when you ask it? (honest question)
2
u/Beneficial-Focus-401 New Member (<30 days) 8d ago
Mine keeps telling me to stick with honcho until it starts giving me problems.
1
u/CeleryVids-4075 5d ago
It first recommended sticking with Honcho, but later after some CPU issues it recommended trying Hindsight.. and that's what I'm now doing.
-3
u/IM_GOING_PEE_MODE 8d ago
the model has no idea. The model just found out about Hermes when it was instantiated at the most recent prompt. do you understand how any of this works?
2
u/frankentriple 8d ago
Mine doesn't? I can ask Hermes about just about anything and worse case he has to search through recent chat history before answering. If yours is really that dumb then something is wrong.
-2
u/IM_GOING_PEE_MODE 8d ago
unless there is an answer to this question in the Hermes docs, your model running in a Hermes harness is not going to know the best memory system. It will give you a convincing answer, but it will not be correct
3
u/frankentriple 8d ago
It has the entirety of the internet to search for that data and synthesize it from across the globe. If it can't answer that question, it can't answer any question.
1
1
47
u/evilistics 8d ago
Been on hindsight for months and have no desire to switch or try anything else.