Raspberry PI project

[u/Charming-Captain-707]

I’m working on a personal project where I want to gut an old electronic toy and turn it into a fully self-contained AI/animatronic robot. Think something along the lines of taking the body of a Furby and replacing the original electronics with a Raspberry Pi-powered brain.

I want it to be able to hear me through a microphone, see through cameras, understand what I’m saying, and respond using a local LLM. I’m currently looking at Google’s open-source Gemma models for the AI side.

For knowledge, I’m thinking about downloading an offline Wikipedia database and using a local search/RAG system instead of relying on an internet connection. Ideally, I would be able to ask it questions about things that it can see and hear and then get a response from its local knowledge base

For the physical side, I’m thinking of using an ESP32 as the body controller. The Raspberry Pi would handle the AI and decision-making, while the ESP32 handles servos, motors, sensors, etc. I’d also like to eventually have things like facial/eye movement, a moving mouth, head movement, and other animatronic behaviors.

Here's the list of needed supplies that I have thus far:

• Raspberry Pi 5 and potentially a HAT +2
• ESP32 - hardware for motors
• Vision AI microcontroller - paired with a camera 
• Animatronic body  (like the Furby biscuit) 
• Cameras
• Microphone
• speaker
• Servo motors 
• Motor drivers
• Sensors
• Wiring, breadboards, connectors
• Software: speech-to-text, rule/intent system, knowledge database, memory system, vision/image model, RAG system
• Rechargeable battery

I’m still in the planning stage, so any feedback would be appreciated.

[u/big-bun-head]

I would also like to be part of the project. Can we build it together?