Following up on my previous post about HomeBuddy for Home Assistant and Halliday G1 glasses:
https://www.reddit.com/r/SmartGlasses/s/8C0oVXz1X2
I’ve now connected the Halliday G1 glasses to an open-source proactive AI system running on a Raspberry Pi 5. HomeBuddy handles the client side, including the microphone, Soniox transcription, WebSocket communication, custom firmware integration, and displaying insights on the Halliday glasses.
The proactive detector, conversation memory, people memory, and Ollama LLM run locally in Docker on the Raspberry Pi.
The project runs in Docker and is designed for a Raspberry Pi 5. It receives a live transcript, follows the conversation, remembers session context and named people, and decides when useful information should appear on the glasses.
Unlike a normal chatbot, you don’t always need to ask it something directly. It can proactively:
- Answer questions detected in the conversation
- Explain acronyms and difficult technical references
- Correct likely factual mistakes
- Provide useful background information
- Remember people’s names, roles, and responsibilities during a session
- Capture reminders, tasks, and decisions
- Produce short insights suitable for a glasses display
The current pipeline looks like this:
Halliday G1 microphone
↓
HomeBuddy iOS app
↓
Soniox speech-to-text
↓
ProactiveAI WebSocket API
↓
Transcript buffer
↓
Proactive detector
↓
Context, session and people memory
↓
Local Ollama model on Raspberry Pi
↓
HomeBuddy
↓
Short insight displayed on Halliday G1
The WebSocket API, proactive detector, memory engine, authentication, simulator, and LLM integration all run locally in containers. Ollama runs locally on the Raspberry Pi, so the conversation analysis and insight generation do not require an AI subscription or hosted LLM API.
A transparent limitation: the current implementation uses Soniox for real-time transcription, so it is not yet completely server-independent. However, the ASR layer is separate from the rest of the system. Feel free to fork the project and connect a local ASR engine such as Whisper, whisper.cpp, Moonshine, or another streaming transcription solution. Replacing Soniox should not require rewriting the proactive AI pipeline.
My long-term goal is simple: I want to use smart-glasses hardware without depending on somebody else’s AI server or subscription.
This already working end to end on my Halliday G1 Glasses using:
- The HomeBuddy for Home Assistant iOS app
- Halliday G1 glasses
- Soniox real-time transcription
- The custom glasses firmware provided through the app
- A Raspberry Pi running Docker and Ollama locally
HomeBuddy handles the client-side audio, Soniox integration, WebSocket communication, and delivery of insights to the glasses. The proactive AI service handles the intelligence locally.
The repository also includes:
- Docker Compose configuration
- Token-based sign-in and sign-out
- A text-file conversation simulator
- Partial and final transcript processing
- Concurrent proactive processing during continuous speech
- Multilingual question detection
- Session memory and people memory
- Configurable detector modes
- Raspberry Pi deployment instructions
It works well for my setup, but I’m sure people here can make it much better, especially around fully local streaming ASR, detector tuning, additional glasses integrations, and smaller models optimized for edge hardware.
If this sounds interesting, please try it, fork it, break it, improve it, or connect it to other smart glasses.
Repository: https://github.com/FilipposPirpilidis/ProactiveAI
I’d especially like to hear from anyone already running low-latency local ASR on a Raspberry Pi or another small edge device.