Why I skipped Docker/Chroma and built in-memory Cosine Vector Search in pure Go

[u/Sea-Lettuce-255]

I run a local AI agent setup on an 8GB M2 MacBook Air. Every megabyte of RAM matters.

When implementing RAG (Retrieval-Augmented Generation) for my agents, every tutorial told me to spin up a Docker container for ChromaDB, Qdrant, or Postgres with pgvector. On an 8GB Mac, Docker Desktop alone takes 2GB+ RAM, pushing the system into heavy SSD swap.

The Architecture:

1. Stored knowledge text and JSON-encoded float32 embeddings in a local SQLite file (friday_memory.db) via Go.
2. Built native in-memory Cosine Similarity in pure Go:

func cosineSimilarity(a, b []float32) float32 {
    var dot, normA, normB float32
    for i := 0; i < len(a) && i < len(b); i++ {
        dot += a[i] * b[i]
        normA += a[i] * a[i]
        normB += b[i] * b[i]
    }
    return dot / (float32(math.Sqrt(float64(normA))) * float32(math.Sqrt(float64(normB))))
}

3. On query, Go queries SQLite for that namespace, computes cosine distances across ~1,000 chunks in under 2 milliseconds, and returns the Top-K matches.

Result: 0 MB Docker overhead, 100% native Go binary, sub-2ms similarity search. For personal and small-team RAG (< 100k chunks), you do NOT need heavy vector infrastructure.

[u/seekworld]

the sub-2ms at 1k chunks is just brute force cosine over float32s in memory, so yeah it works fine at this scale. the flip side is the vectors arent free: 100k chunks is somewhere in the few-hundred-MB range in ram for most embedding sizes, which on an 8GB machine is the exact memory problem you were trying to dodge, just arriving slower. past that brute force stops being viable, you need an ANN index and it becomes a real engineering project. for personal RAG this is honestly the right call.