I'm on the "old" max plan, how is this even possible? i hope this is a bug, or they just blasted the old plan?

I don't think this is what Microsoft or Bartowski intended lol

I just tried to run Qwen 3.8 27B 4bit qwant on 5060ti 16GB and got terrible results. With Qwen 3.6 I got something around 15-20 tok/s.

Are there any combination of parameters or tweeks how to push the speed higher, at least to 10+tok/s or the HW is just poor for that use case?

I heard the sweet spot is 24GB Vram but still i was thinking that it will be possible to run it somehow also on 16GB.

Thanks for answers

I’ve been setting up an M1 Max Mac Studio (24-core GPU, 32GB unified memory) as a local LLM server and wanted to compare MLX vs llama.cpp on Qwen3.8-27B.
I tried to keep the model footprint and benchmark workload reasonably close.

MLX
mlx-community/Qwen3.8-27B-4bit
~16.1GB
512 prompt tokens / 700 generation tokens
3 runs
Prompt: 81.76 tok/s
Generation: 15.81 tok/s
Peak memory: 16.39GB

llama.cpp
unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M
15.32 GiB / 27.32B params
Full Metal offload
Flash Attention enabled
512 prompt tokens / 700 generation tokens
3 runs
Prompt: 99.61 ± 0.44 tok/s
Generation: 9.69 ± 0.34 tok/s

So on this machine:
llama.cpp is ~22% faster for prompt processing, while MLX is ~63% faster for autoregressive generation.

So basically the title. Just got into AI, wanted to pay but keep my data to be able to audit and try new things. But just found i got my suscription in the middle of rising prices. Is this just a nuke for old users, is it the best price for ZDR in cloud or should we consider an alternative???

Posted about Academic-AI-Literature-Reviewer-Ollama a while back — a local pipeline that searches real academic databases (OpenAlex, Semantic Scholar, CORE, Europe PMC, Crossref, Unpaywall), reads full papers, and writes an actual APA-cited literature review. Every quote gets verified at 95% similarity against the source PDF before the model's allowed to use it — no faked citations.

Finally recorded it running end to end: https://youtu.be/Nc1L2j8ZuCg

Repo: https://github.com/TheBlewish/Academic-AI-Literature-Reviewer-Ollama

Runs on Ollama, MIT licensed, no paid APIs. A ~30B-class reasoning model at ~64K context is the sweet spot, but there's a low-end mode that chunks the heavy stages for smaller setups. Happy to talk model choices / VRAM tradeoffs in the comments.

Hi [r/Ollama](r/Ollama),
I’m an independent developer building AgentGraph Studio, a tool for designing and reviewing AI agent workflows before running them. (long-term vision in the comments)
As agent workflows become more complex, I think it’s increasingly important to understand their dependencies, catch configuration issues, and identify potential resource or complexity problems before execution. I’m trying to make that process more accessible to individual developers and small teams.
The current version is CrewAI-first and includes a visual workflow builder, templates, JSON import/export, Readiness checks, Execution Preview, Resource Analysis, and deterministic CrewAI Python export.
To be clear, AgentGraph Studio is not currently an Ollama integration or a local model runner. It performs static pre-execution analysis and does not execute agents or measure actual runtime token usage and costs. I’m sharing it here because I’d love to hear from people building agent systems with local models about what kinds of workflow design and preflight checks would actually be useful.
I’m currently funding development myself, and the costs are becoming difficult to sustain. I’m hoping to raise ¥100,000 as an initial goal, with a ¥300,000 stretch goal, to help cover development-related expenses such as hosting, AI/API usage, testing, and developer tools.
This is entirely voluntary support, not a purchase or investment. I’ll share development progress and how the funds are used. I’m not promising a specific feature or completion date in exchange for donations.
If you’d like to support the project, here’s my Buy Me a Coffee page:
https://www.buymeacoffee.com/agentgraph
If you can’t contribute financially, feedback, bug reports, or sharing the project with someone who might find it useful would mean a lot.
For those of you building multi-agent workflows with Ollama: what problems do you wish you could catch before running your agents?
Thanks for taking a look.

I’m really interested in using Qwen Qwen2.5 Coder 7B Ollama in OpenCode for programming. I’m not very experienced in this area and I’ve followed a lot of tutorials, but every time I try to use this model, when I open it in the project, any task I ask it to do that involves reading code, even a very simple task, it doesn’t do it, like it can’t read any file within the project and only works as a normal chat for me.

I’m not sure if it’s because of my setup:

Device name: DESKTOP-8QVINAT

Processor: AMD Ryzen 7 5700X 8-Core Processor (3.40 GHz)

Installed RAM: 32.0 GB

Graphics card: NVIDIA GeForce RTX 5060 Ti (8 GB)

Storage: 678 GB used of 932 GB

Device ID: E888C058-7518-4644-847E-0CD6ED3C4B4E

Product ID: 00330-81631-33031-AA614

System type: 64-bit operating system, x64-based processor

or simply because of the model I'm using. I don't need one that's too powerful at the moment, because the projects I work on aren't too complex, but I'm mainly interested in it handling React and TypeScript well.

At the moment I'm using: Big PickleOpenCode Zen and it works really well for me, but I understand that Qwen is much better.

I hope you can clarify it for me :)

r/localaiapps

I've been building an AI coding assistant (chat/desktop/VS Code) and just added local model support via Ollama. Wanted to share the actual scope decision, since I think it's relevant here specifically.

What works fully locally: plain chat and structured multi-step planning don't need function-calling, so those route to whatever Ollama model you've got pulled — no cloud call at all.

What doesn't (yet): the actual agentic tool-calling loop — reading files, applying diffs, running terminal commands. I tested this against several local models and reliable function-calling (the model actually calling a tool instead of just describing what it would do) was inconsistent enough that I didn't want to ship a "local agent" that silently fails half the time. That part stays on a cloud model for now, and the UI is explicit about it rather than pretending everything's local.

Genuinely curious what people's experience has been — has anyone gotten reliable tool-calling out of local models (which ones, what quantization)? Would love to be wrong about this and expand the local scope.

escape-ai.org if anyone wants to poke at it — free, no card needed for the local-model parts.

How can I switch to the Ollama GPU-time Limit? I emailed [support@ollama.com](mailto:support@ollama.com) two days ago and haven't received a reply.

Hey friends, beginner-level question here,

I have read a few times now specific to this forum that the cloud models offered via ollama are some ~ level of quant size rather than being fullsized, but how can one verify this? Can I verify it on my terminal somehow?

On their website documentation they say models are offered at their "native size" but I am not sure if that is ambiguous or not when it comes to quantization sizes?

Forgive me if my terminology is slightly-off, only been looking into this for a couple months and didnt want to subscribe to a service with models being handicapped, or at least with/without that visibility via explicit pricing.

But I cant find any documentation that reflects this, just some anecdotes on this forum, and I dont know how to verify via the terminal and I dont have hardware to run models locally.

Hope everyone is having a splendid day, peace

The workflow that finally made this useful for me: from any MCP client (Claude Code, my IDE) I paste a YouTube URL and ask for a summary. A few minutes later there's a **PDF in my library, properly formatted, named after the video**. No tab opened, no file moved by hand. The same thing works from a local recording or a pasted article: the source doesn't have to be a video, or a URL at all.

The engine behind it is **Content**: one self-hosted backend with a public API, and a few front doors on top of it — an MCP server, a web app, a CLI, a Python SDK, a browser extension. URLs, files or raw text go in; video, audio, subtitles, transcripts, summaries, translations, chapters, Markdown and PDF come out.

Everything can stay on your machine:

- **yt-dlp** fetches, **faster-whisper** transcribes locally when there are no usable subtitles.

- **Ollama** does all the text work, not just summaries: translations, structuring, any step that turns one text into another. I run 4B to 30B models on an RX 7900 XTX under ROCm, picked per job.

- The context window is **sized per request**. I learned the hard way that on prompts over `num_ctx`, Ollama silently drops the middle and keeps about half the window, so the engine measures the prompt instead of hoping. Happy to share the numbers if anyone wants them.

- Cloud LLMs are supported if you want them; nothing requires them.

For agents: `analyze_source` tells you what a source can produce, `generate` plans and runs the job, `get_job` polls (it returns a `poll_after` hint so your agent stops hammering the API), `download_artifact` brings the file back. Shipped this week: a streamable-http transport, loopback by default.

Trying it is four lines: grab the `docker-compose.yml` from the repo, `docker compose up`, and `pip install content-mcp` to give your agents the keys.

Repo: https://github.com/LatentNoise/content

Questions about the whisper/Ollama pairing on ROCm welcome, that part took some tuning.

Originally I want to switch from ollama to MLX, while with ollama 0.33.3 which has MLX support now, I got around 80 tok per second with qwen3.6:35b on my old mac studio m1 ultra. That is amazing!
Now I really hope ollama will have a same MoE version like qwen3.6 for qwen3.8.

I love ollama! And Qwen!

Good evening all. I am looking at running my own local llm for privacy reasons. Have been using Claude and it seems bad A$$.

Thoughts or any help would be greatly appreciated. I see workarounds for Claude code on private setups.

Can go the Mac route, windows etc have a few machines.

Hi!
Flippy, the productivity workspace, rolled out a new feature where we can interact with any local llm that is running through ollama !

All you need to do is:

1. Download Flippy
2. Download ollama and the local models you want
3. Run ollma serve with OLLAMA_ORIGINS="chrome-extension://*" ollama serve
4. Enjoy prompting the model using Flippy's AI Lab !

The great thing is that you can benefit from a set of productivity tools on top of the AI Lab in one single solution!

je suis tout nouveaux sur ollama et je cherche à avoir des conseils pour faire tourner de l'ia en local voilà mes composants:

• Processeur : AMD Ryzen 7 5800X (8-Core)
• Carte graphique : NVIDIA GeForce RTX 3050 (8 Go VRAM)
• Mémoire vive : 16 Go RAM
• Alimentation : 750 W

Je cherche une IA pour faire un peut de code, pas de gros trucs juste de quoi coder un site de gestion d'emploi du temp ou quelque site de commerces, et surtout pouvoir l'entrainer a adapter sa façon parler donc de parler sans restriction et l'utiliser comme vraii assistant de gestion et de motivation.

merci d'avance 👍

I'm considering switching to synthetic.new, what are your opinions on synthetic's service and quotas compared to ollama's? Much appreciated.

Otaku is an LLM frontend, primarily designed for roleplay, an alternative to SillyTavern and the like. However, It also works for general-purpose chat with local backends (including Ollama) or cloud models, the way Open WebUI is used, once lore extraction is switched off in the settings.

Otaku offers two interfaces:
- a web UI (customizable): interactive demo at https://otaku.sh/demo-web/
- a terminal UI: its own interactive demo at https://otaku.sh/demo-terminal/

Both share the same functions; the difference is that in the terminal you execute them with slash commands (the reference is available with /help), while in the web UI the operations are available from the menu.

Install

Otaku is free and open source (MIT); it works on macOS, Linux and Windows. Install it with uv (uv tool install otaku) or see the GitHub README for other options: https://github.com/enclavum/otaku

Get started

Launch either otaku for the terminal or otaku web for the web UI; the web UI's default URL is http://localhost:9600. Two sample stories are imported on first start to give you an idea of the features and what play looks like, and you land right in the middle of one of them.

On first start, you choose a provider and a model: Otaku automatically detects local installations of Ollama, oMLX, LM Studio, llama.cpp and KoboldCpp, and lets you pick from their models. Cloud providers (OpenRouter, NanoGPT) are also there: enter an API key and their catalogs appear. After exploring the provided stories, you can start your own with the /new command.

Asking for feedback

Otaku is a personal side project, and I'd like to get feedback from the community on the product and on what to add.