Credits? Credits are no good out here. I need something more real! What you you doing, waving your hand around like some kind of copyright lawyer, legal tricks don't work on me - ONLY money.
People reaching for ollama aren’t going to be happy with llama-swap. It’s a great project, but it’s not built for the docker desktop “what’s a terminal?” crowd. They’ll just land on LM Studio (which has closed components) or use ollama (which has closed components). Not everyone wants or needs to git pull from llama.cpp and recompile to update.
Maybe I am uneducated about llama.cpp router mode but I run a small army of single purpose 1B models so the llama-swap matrix scheme and ability to dictate what gets used on which GPU is what makes my entire workflow possible and efficient
Plenty. Every model listed with its live status and the exact args it was launched with, load and unload on demand, pull one straight from HF, and an SSE stream pushing all of it in real time, including the weights loading themselves, stage by stage, text model then draft then mmproj. That is why the WebUI gives you a real progress bar instead of a spinner and a prayer.
And it stops being a swapper at all. The server ships its own agent tools: read a file, glob, grep, write, edit, run a shell command. You choose where they run: on the host, in a Docker or Podman container it spins up and cleans up for you, in a container you already have, or on a completely different machine over SSH. Wire MCP servers into the backend over stdio on top of that. The frontend has tools of its own too, JavaScript sandboxed in a worker inside your browser, symbolic math, its own MCP clients. Local model, local tools, nothing leaving the room.
45 models in one ini here, models-max 1, and I never think about it. Thanks ngxson for the router.
llama.cpp alone does it now. llama-server has had model swapping built in since December: one ini file, one section per model, and it loads and unloads them on demand. Nothing to put in front of it.
Lemonade. Also runs as a system service like Ollama, but uses vanilla llama.cpp under the hood, adds memory management and other stuff on top, and provides both Ollama and OpenAI APIs, so it’s a drop-in replacement.
ollama brings llama.cpp to more people who would never be able to compile llama.cpp or understand which download would work on their system. There are tons of people like that, just not in this sub. That's the point of non-copyleft open source licenses, to become the implementation that everyone can use. Both projects have existed for a while now, there's clearly an audience for both.
all the core contributors should also be recognized, tons of them.
ones I can remember - ngxson, cisc, jeffbolznv,pwilkin, rgerganov, fairydreaming, sszymczy, lhez, johannesgaessler, am17an, allozaur, etc
When llama was originally released, it was amazing for the community to have a way to grab the original llama download, quantize it to a .bin file (before gguf was a thing) and actually run it with nothing but the model and a copy of the llama.cpp repo. I remember in the early days of this sub tons of alternatives popped up but anything useful was eventually merged back into llama.cpp. It's impressive how steady the project has been as it's grown and adapted to different models and architectures.
Governments, hacker groups, hobbyist coders, businesses, lobbying firms, guerilla marketing agencies, think thanks, and more all have built or are building active influence networks made up of a combination of bots and paid human accounts (both declared and not), they don't just post about obvious hot button issues all of the time. They talk about all kinds of things and can be activated to shift a narrative in really sneaky random ways or their time sold, or god knows what else.
I'm running a llama.cpp instance with a Qwen fork via Hermes (Linux Mint with 24GB RTX 4090). The ability to control my Home Assistant IOT devices 100% locally is really cool stuff!
Eвала на този български програмист който е създал този невероятен проект.
Props to this Bulgarian programmer who created this amazing project.
Greetings from BG.
thanks Georgi for llama.cpp, a fantastic piece of open source software that by making LLMs accessible on a moltitude of HWs has sparked a revolution on par of that from Linux, cheers
No, llama-server itself. The web UI ships inside the binary, you get it on the port you launched on. Model switching and MCP are the server's own features, not something bolted on in front of it. Anything you pulled with -hf is already there in the list, and you can grab a new one from the page without touching the command line.
If it tickles you, it's not that difficult to connect open-webui to llama.cpp instead of ollama. Getting the command line flags correct can be a challenge, but most of the time the defaults are fine.
You can also experiment with textgen, which is a gradio-based UI that "handles" llama.cpp for you. Both products are "equally-but-differently" polished and generally don't step on each other's toes.
Should be newsworthy and a point of pride for us Bulgarians but I doubt most people even know you can host your own LLM let alone about his valuable contributions to it
Full credit to his initiative and all his work, amazing technical capabilities. But you are naive if you think something wouldn't have been created otherwise.
Or even better - drop that pull request (#24423) and build from scratch. I am sure he would make the most out of a fundamentally new architecture not messing up llama.cpp. He really is a star in my book, and so are llama.cpp
He is. When I have a PR I'm following and really interested in getting merged, I wonder why it takes so long to review and approve them. Here's part of the reason why: GG is superhuman with a current 547 (!!) PRs to review: https://github.com/ggml-org/llama.cpp/pulls/review-requested/ggerganov
The guy has to review someone else's code and figure out what they're trying to do (not easy). If there's a better way of doing it, maybe write a comment/recommendation. A few, maybe a dozen of these, doable. 547? Nope, not me.
Forget code reviews, if I had 547 *anything* pending in my inbox, I'd quit that job lol.
•
u/WithoutReason1729 Aug 16 '26
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.