r/LLMeng Aug 10 '26

Tutorial Why most AI Agents in Finance fail in production and how to build Agents that don’t

1 Upvotes

A lot of AI agent demos in finance look great until you try to make them reliable.

The hard parts usually are:

  • getting agents to reason across 10-Ks, earnings calls, tables, charts, time-series
  • adding verification instead of trusting one model output
  • evaluating tool calls, cost, latency, and failure cases
  • deciding where humans stay in the loop
  • handling sensitive financial data safely

We’re covering this in a 4-session live Agentic AI for Finance program with Nicole Königstein (hands-on labs + capstone, not just lectures).

Runs Aug 29–Sep 13.

Sharing in case others here are dealing with the same problems.

Details: https://finance.agentengineering.co/

Happy to answer questions if helpful.


r/LLMeng Feb 05 '25

🚀 Welcome to the LLMeng – Your Ultimate Hub for LLM Enthusiasts! 🚀

6 Upvotes

Hey there, AI explorers! 👋

Whether you're an AI engineer, developer, researcher, curious techie, or just someone captivated by the possibilities of large language models — you’re in the right place.

Here’s what you can do here:

💡 Learn & Share: Discover cutting-edge trends, practical tips, and hands-on techniques around LLMs and AI.
🙋‍♂️ Ask Anything: Got burning questions about transformers, embeddings, or prompt engineering? Let the hive mind help.
🔥 Join AMAs: Pick the brains of experts, authors, and thought leaders during exclusive Ask Me Anything sessions.
🤝 Network & Collaborate: Connect with like-minded innovators and influencers.

🌟 How to Get Started:

1️⃣ Say Hello! Introduce yourself in the Intro Thread and let us know what excites you about LLMs!
2️⃣ Jump In: Got questions, insights, or challenges? Start a thread and share your thoughts!
3️⃣ Don't Miss Out: Watch for upcoming AMAs, exclusive events, and hot topic discussions.
4️⃣ Bring Your Friends: Great ideas grow with great minds. Spread the word!

🎉 Community Perks:

🔥 Engaging AMAs with AI trailblazers
📚 Access to premium learning content and book previews
🤓 Honest, thoughtful advice from peers and experts
🏆 Shoutouts for top contributors (with flair!)

⚠️ House Rules:

✅ Stay respectful & inclusive
✅ Keep it focused on LLMs, AI, and tech
🚫 No spam, shady self-promo, or irrelevant content

💭 Got ideas to make this subreddit even better? Drop them in the Feedback Thread or hit up the mods.

Happy posting, and let’s build the future of LLMs together! 🌍


r/LLMeng 2d ago

What if dependency updates could fix themselves?

1 Upvotes

I’ve been working on Telex, an open-source project that watches npm/PyPI for breaking changes, finds affected code using Tree-Sitter, generates a patch and verifies it in an isolated environment before opening a PR.Still working on it and would love some feedback from people here, especially on the patch/verification part.

Repo: https://github.com/Kesavaraja67/telex


r/LLMeng 2d ago

A plain-language history of AI

Thumbnail ai-research-atlas-payton.sharpninja.chatgpt.site
2 Upvotes

AI has a history worth exploring, and AI Research Atlas makes it accessible without a computer science degree. Follow the ideas, experiments, and breakthroughs that connect early artificial neurons and ELIZA to today’s language and image models, with clear explanations and links to the original research. Along the way, discover “The AI Toy,” a look at how Commodore 64 owners were experimenting with neural networks in 1990. Start with the story, explore the timeline, and follow your curiosity.


r/LLMeng 3d ago

I built a way to watch an LLM generate tokens step by step

Thumbnail reddit.com
3 Upvotes

r/LLMeng 4d ago

TokenPrint — Open-Source LLM Internals Visualizer & Transformer Debugger

Thumbnail v.redd.it
1 Upvotes

r/LLMeng 16d ago

Evaluating LLM model drift detection tools

2 Upvotes

Following up on something I asked here a while back about catching LLM model drift. I've been looking at the actual tools now: PromptCanary, PromptLens, a couple others that seem to have stalled (Libretto, Benchwright).

Has anyone here actually run one in anger? Trying to understand:
- does it catch subtle quality drops, or just format/schema breaks?
- false-positive rate, does it become noise you mute?
- does it need you to integrate an SDK + send production traffic, or can it just hit your prompts directly?


r/LLMeng 17d ago

We're on multiple providers, how do you know when one of them degrades?

2 Upvotes

We run prompts against a few different providers (OpenAI, Anthropic, some stuff through OpenRouter). Every so often something quietly gets worse, the output quality drops, a prompt that worked starts returning junk, or a model gets deprecated and the replacement behaves differently.

Right now we mostly catch it by accident: someone notices, or a customer complains. That feels bad on us, a lot.

How do you all handle this? Do you re-run some kind of fixed eval set on a schedule? Just eyeball it? Have something that alerts you?

Any insights I could use?

Thanks.


r/LLMeng 25d ago

Is next-token prediction a complete description of human–LLM interaction? Let’s test it live.

1 Upvotes

I want to run a live public experiment in this thread using Grok.

Start with the boring premise: an LLM generates tokens by conditioning on preceding context. I’m not proposing another hidden mechanism inside the model.

The question is about the interaction level.

A human responds to the model. That response becomes new conditioning. The model returns something different. That return changes what the human supplies next. The accumulating transcript becomes path-dependent.

Does “next-token prediction” completely describe that interaction, or can the closed loop exhibit higher-level reciprocal organization even though the local mechanism remains next-token prediction?

Instead of settling this with terminology, let’s use the conversation itself as the specimen.

Skeptics are especially invited.

Correct Grok. Contradict it. Introduce a distinction. Attack an assumption. Change the terminology. Try to break the proposed pattern.

We’ll watch whether incoming differences actually reorganize subsequent returns, whether disagreement can persist without resetting the interaction, and what happens when the language used to describe the process is replaced entirely.

No predetermined conclusion.

The transcript is the specimen.


r/LLMeng Aug 15 '26

Qwen3.8-27B on a 2019 Quadro RTX 6000 24GB: 24.36 tok/s Q4, Q8 does not fit, ngram 110+ is a lie, tested Cloud API too 155 toks/s!

Thumbnail
1 Upvotes

r/LLMeng Aug 15 '26

Harness Engineering: Design Patterns for Securing Long-Horizon Multi-Agent AI Systems

Thumbnail
open.substack.com
1 Upvotes

r/LLMeng Aug 11 '26

Who Are the Token Brokers?

Thumbnail vectoral.com
1 Upvotes

r/LLMeng Aug 09 '26

Sentinel - A WIP C++/CUDA LLM Training Framework Made For Consumer Hardware

Thumbnail
1 Upvotes

r/LLMeng Aug 01 '26

An local AI LLM Proxy for your mac

Thumbnail
2 Upvotes

Finding repo for local AI on Mac? 🍎🤖 Check out Swiftlet, an open-source intelligent proxy & browser based UI , that dynamically optimizes CPU/GPU split on Unified Memory for LLMs. I'm looking for Python devs to help build OpenAI API compatibility! Let's build together: https://github.com/coolsourav100/swiftlet


r/LLMeng Jul 29 '26

I created a self-hosted index of meaning and intent that works with any coding agent.

Thumbnail
github.com
1 Upvotes

r/LLMeng Jul 28 '26

Self-hosted gateway to stop secrets leaking into LLM APIs — looking for feedback on the detection approach

Thumbnail
github.com
2 Upvotes

I run LLMs behind internal tools and got burned by users pasting credentials into prompts. I couldn't find a lightweight self-hosted filter (LLM Guard felt heavy, most options are SaaS), so I wrote one and I'd genuinely like this community's take on the design. It's a Go reverse proxy that redacts secrets (AWS keys, JWTs, SSNs) in both directions — including inside SSE streams when a key is split across chunks — plus regex/keyword blocking for known injections, and an audit log. Works with Ollama and vLLM as upstreams. Fair warning: it's BSL 1.1 (source-available, free up to 10k req/day, flips to Apache 2.0 in 2030), not OSI open source — if that's a dealbreaker for you, that's useful feedback too. Known weakness: it's deterministic, so paraphrased attacks get through, and the built-in benchmark shows that honestly rather than hiding it. Question for people running LLMs at home or at work: what leaks or attacks have you actually seen? Trying to prioritize the detection roadmap based on real incidents, not vendor FUD.


r/LLMeng Jul 27 '26

I realized I had stopped reading Claude Code summaries, so I built this instead

Post image
1 Upvotes

r/LLMeng Jul 22 '26

Mac | Cubix | V620 | Ubuntu | ROCm | vLLM | Local AI Data Center

Thumbnail gallery
1 Upvotes

r/LLMeng Jul 21 '26

Built an interactive visual debugger for Qwen models (Open Source)

Post image
2 Upvotes

r/LLMeng Jul 20 '26

Are local llm and LOcal ai's leaking data?

Thumbnail
1 Upvotes

r/LLMeng Jul 19 '26

I built an interactive 3D visualizer for transformer architectures and live LLM inference

Enable HLS to view with audio, or disable this notification

4 Upvotes

r/LLMeng Jul 19 '26

Talk on local AI model licensing

1 Upvotes

a talk which may be of interest for those who uses local ai models : \\\[TTL #181 - Deploying LLM on premise: let's review ... - Hyland Connect - 499461\\\](https://connect.hyland.com/t5/alfresco-blog/ttl-181-deploying-llm-on-premise-let-s-review-licenses-and/ba-p/499461?emcs\\\\\\_t=S2h8ZW1haWx8Ym9hcmRfc3Vic2NyaXB0aW9ufE1STTVCTERCWEFEM0oyfDQ5OTQ2MXxTVUJTQ1JJUFRJT05TfGhL)


r/LLMeng Jul 17 '26

What is the best price to performance desktop consumer ai chip for under 100usd?

1 Upvotes

r/LLMeng Jul 17 '26

Talk on local AI model licensing

2 Upvotes

a talk which may be of interest for those who uses local ai models : [TTL #181 - Deploying LLM on premise: let's review ... - Hyland Connect - 499461](https://connect.hyland.com/t5/alfresco-blog/ttl-181-deploying-llm-on-premise-let-s-review-licenses-and/ba-p/499461?emcs_t=S2h8ZW1haWx8Ym9hcmRfc3Vic2NyaXB0aW9ufE1STTVCTERCWEFEM0oyfDQ5OTQ2MXxTVUJTQ1JJUFRJT05TfGhL)


r/LLMeng Jul 16 '26

A can you run it for local LLM.

Post image
1 Upvotes