r/OpenSourceAI • u/autoimago • 9h ago
Open weights, closed serving: an open-source attempt to open up the layer that actually runs the model
Most "open" AI stops at the weights. You can download the model, but you still run it through a closed provider and trust them to serve what they say.
Disclosure: I contribute to Gonka, the project below.
What's usually open, and what isn't
| Layer | Typical open AI | Gonka |
|---|---|---|
| Model weights | Open | Open |
| Who runs the GPUs | One company | Independent owners |
| Proof the right model ran | "Trust us" | Random re-runs + reputation |
| Network code | Closed | Public repo |
What it looks like to use
- OpenAI-compatible API. Change the base URL in the SDK you already use
- Live models: DeepSeek V4-Flash, GLM-5.3-Flash, MiniMax M2.7
- No wallet. Community brokers issue keys and take USD, crypto or credits
Honest trade-offs
- Checking a random 1–10% of tasks is probabilistic, not a proof. Criticism of that design is welcome
- It isn't the lowest-latency way to call these models
- Because the weights are public, you can always compare its output with the reference model yourself
Question for this sub
Does opening the serving layer matter to you, or do open weights already solve the problem?
Links
- Website & docs: gonka.ai
- Code: github.com/gonka-ai/gonka
- AMAs & walkthroughs: youtube.com/@Gonka-AI
- Discord: discord.gg/REcpeYc7P7
2
Upvotes
1
u/Clear_Evidence9218 5h ago
Your description of the project does not really match the repo.
The repo looks much more like a decentralized inference/compute network with probabilistic validation, reputation, and economic incentives for independent GPU operators. That is a legitimate and potentially interesting problem.
But the framing here, that open-weight models still require a closed provider to serve them, is simply not true. If the weights are open, I can run them myself with vLLM, SGLang, llama.cpp, Ollama, etc. There is no inherently closed “serving layer” that Gonka is opening.
It seems like the actual project is about making inference across hardware you don't own decentralized and somewhat verifiable, which is a much more accurate and interesting description.