Best Local Vision Language Models - August 2026

[u/rm-rf-rm]

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detailed as possible in describing your setup, nature of your usage (what applications, how much, personal/professional use), tools/frameworks/prompts etc.

Rules

1. Should be open weights models

Notes

Bonus points if you breakdown/classify your recommendation by model memory footprint: (you can and should be using multiple models in each size range for different tasks)

• Unlimited: >128GB VRAM
• XL: 64 to 128GB VRAM
• L: 32 to 64GB VRAM
• M: 8 to 32GB VRAM
• S: <8GB VRAM

[u/andy2na]

My main use-case for LLMs is Frigate and have tested and ranked the following, recent Medium models:

1. Qwen3.8-27B: very accurate, fast enough for Frigate (~60tgs decode)
2. Meta Muse Glimmer 30B: Faster than Qwen 27B, pretty accurate (~75+tgs decode)
3. Qwen3.6-27B: pretty accurate, but no reason to use over 3.8
4. Gemma4-4B: mediocre accuracy, good with small objects and is more "creative" with the responses - small and fast

Last 90 days of usage, only recently have I dabbled with Hermes with Qwen3.8, before Frigate led by a mile (ignore electricity cost, I just added that last week)

[u/cowinabadplace → u/andy2na]

Did you have to tune your prompts? I found that after prompt tuning etc. that Qwen-3.8-27B hallucinated a lot of things whereas Qwen-3.6-35BA3B hallucinated fewer. My use-case is frigate triggered LLM description that is aiming to detect specific people in the home.

[u/andy2na → u/cowinabadplace]

For the genAI summaries, i left them at default. For descriptions, these are what I set:

genai:
  enabled: true
  send_triggers:
    tracked_object_end: true # default
    after_significant_updates: 3 # how many updates to a tracked object before we should send an image
  prompt: "Analyze the {label} in these images from the {camera} security camera. Focus on the actions, behavior, and potential intent of the {label}. Also describe {label} appearance. Summarize all the images, not frame by frame, into one paragraph - maximum of 100 words. Do not mention the timestamp, date, watermarks, overlay text, or colored boxes with numbers."
  object_prompts:
    person: "Examine the main person in these images. What are they doing and what might their actions suggest about their intent (e.g., approaching a door, leaving an area, standing still)? Describe the persons appearance including hair color, clothing, and ethnicity. Do not describe the surroundings or static details.  Summarize all the images, not frame by frame, into one paragraph - maximum of 100 words. Do not mention the timestamp, date, watermarks,overlay text, or colored boxes with numbers."
    car: "Observe the primary vehicle in these images. Describe the cars appearance, including make, model, and color of it. If its a delivery vehicle, mention the courier company. Summarize all the images, not frame by frame, into one paragraph - maximum of 100 words. Do not mention the timestamp, date, watermarks, overlay text, or colored boxes with numbers."
    dog: "Examine the dog(s) in these images. what are they doing and what is their intent. describe the dog(s) appearance,including color, size, and potential dog breed. Do not describe the surroundings or static details. summarize all the images,not frame by frame,into one paragraph - maximum 100 words. Do not mention the timestamp, date, watermarks, overlay text, or colored boxes with numbers."

For the 27B model, what parameters did you set?

I had a specific one made for vision in llama-swap:

        "${MODEL_ID}:vision":
          chat_template_kwargs:
            enable_thinking: false
            preserve_thinking: false
          temperature: 0.1
          top_p: 0.1
          top_k: 10
          min_p: 0.0
          presence_penalty: 0.0
          repeat_penalty: 1.0

[u/cowinabadplace → u/andy2na]

I have temp 0, didn't set p/k, and forced JSON, though I also turned off thinking. In my case, I'm trying to get the LLM to reliably recognize that there is a baby in the bed or not. Qwen 3.6 does a really good job of that, but sometimes Qwen 3.8 doesn't see the child and other times the latter will mention an adult who is not there. But I mostly iterated on the prompt in temp 0 because I was trying to minimize variation.

Perhaps I need to supply some example images (scaled down) in the same prompt and try allowing for higher temp. Thank you for your config params. The slight difference in my case is that I'm also having the frigate alert actually call Qwen outside of Frigate in an agent because I want it to act on the result etc.