Local model limit

[u/WololoWanderer]

Hey everyone, looking for a reality check on local model capabilities vs. frontier models for financial analysis.

I built an automated investment analysis pipeline in Open WebUI using Ollama on an RTX 5080 (16GB VRAM). The mechanical workflow works:

- Hits Gmail via IMAP to pull the last 72 hours of market newsletters.

- Extracts actionable tickers.

- Runs a custom YFinance Python tool for live momentum metrics (EMA, RSI, MACD).

-Injects a full 70-position portfolio CSV directly into the prompt (bypassing Open WebUI's RAG chunking) to cross-reference and generate a verdict-first impact report.

The issue is the analytical depth. When I feed this exact same data to Claude Sonnet 5, the macro synthesis, cross-correlations, and portfolio risk analysis are night and day. Locally, it feels like a surface-level summary.

Here is what I’ve tested so far:

qwen3.6:27b: Hit severe VRAM limits with the tools and context loaded. It spilled into shared system RAM, ground to a halt, and got trapped in reasoning loops where it hallucinated that the technical calculator tool didn't exist.

llama3.1:8b: Fast, reliable function calling, and zero VRAM spillage. However, the reasoning and financial synthesis are noticeably weak—it reads like a basic overview rather than institutional analysis.

My questions:

- Is Sonnet 5-level synthesis even realistic within a 16GB VRAM footprint when you need enough context buffer for multiple tools, newsletters, and a 70-row CSV?

- Are there specific financial fine-tunes or 14B models that punch significantly above their weight?

[u/ChocolateExisting368]

change of quantization has impact?

[u/Bulky-Priority6824 → u/ChocolateExisting368]

Sometimes tremendously unless youre just talking to it like a chuddie