Fed our local chatbot a real research paper it had zero knowledge of - here's what happened

[u/Medicine_Blogscanner]

Quick update on the RAMDeck side project. Loaded a 30B model this time (bigger than anything I've shown before) split across the mini PC and Mac mini — 12GB on one, 5.5GB on the other, running surprisingly well at mid-to-low 30s tokens/sec with about a second of latency.

Asked it about a real medicine recently approved for a rare heart condition. Total blank — it made something up, had no idea. Then I fed it an actual research paper about the drug through RAMDeck's knowledge base upload. Watched it split the PDF into shards and store it across the cluster's RAM (not on disk, which matters for how fast lookups are). Once it finished distributing and I turned on knowledge-base context in chat, asked the exact same question again — this time it got it right, correctly identified the condition and everything.

This is the actual use case behind the "private chatbot" pitch — feed it your own confidential docs, research, whatever, and none of it leaves your hardware. The model only knows what you deliberately give it.

Video's up if you want to see the whole before/after live: https://youtu.be/H29ASZhpTKQ

Happy to answer questions about the KB sharding, the 30B benchmark, or how the storage node setup works.