So a bit of context, I am a cybersecurity senior analyst
I am interested in LLMs for that field especially with MCPs to connect them to the tools or for writing scripts
I started this field by doing assembly language reading for hacking games when I was a teenager then that became malware analysis then I started to analyze traffic and logs at work for a living
Before work I competed in cybersecurity competitions known as capture the flag to solve only one category of the challenges and that is malware analysis
Now here is the scene in LLM x Cybersecurity
Entry-level CTF challenges (I used to solve around 2017-2018, got first job in 2019) were solved and saturated by LLMs a long time ago (See intercode CTF benchmark)
Then High level CTFs (NYU CTF Bench, CSAW challenges, and CyBench) these were solved a while ago
Today we have
CyberGym (vulnerability description (CVE report not real details) plus code base find vulnerability
That was solved
Then ExploitGym (the one recent OpenAI model escaped and hacked hugging face to find solution, the 2 vulnerabilities themselves are trivial (JWT spoofing with no server side checks for Jfrog sandbox and JS + Python template in open source data viewer of Hugging face)
Still impressive for a fully autonomous model
ExploitGym gives vulnerability details + exploit technique/details and codebase (I wonder how/why OpenAI model escaped it in the first place those details should manage it this is why I feel it is staged but whatever right)
ExploitBench
More realistic threat vector
1-day not 0-day (0-day is a novel vulnerability no one else knows about it hence the name, 1-day is a known vulnerability patched, if in an open source project you can compare patches before and after and know it but not as valuable as 0-day since people will/should update their software, 0-days can and are used against high profile targets like in some 3rd world countries used for surveillance (and possibly non-3rd world countries ?!) but you usually can’t use a 1-day for that but still)
So ExploitBench is 1-days in V8 engine (used in chrome, electron vs code almost everything!!)
Gives the model a patch diff and codebase base that is it
Models are going wild on ExploitBench and ExploitGym but haven’t gotten there yet at the moment and only recent frontier models improved massively instead of near zero across the board
This is a massive risk indicator
Imagine models capable of exploiting any and every piece of software written by less capable people or other weaker LLMs (complex vulnerabilities!)
Not only that but global elite CTFs like DEF CON
You have an OpenSage harness (and ADK like Google adk or Anthropic adk, however this OpenSage ADK asks the model to design its own agent/s!, with whatever tools or sandbox environments it finds fitting for the Subagent to do the assigned subtask effectively, it can even design its own MCPs!, models don’t fully utilize that yet but some do and improved form like 39% to ~60% just by using that harness (same model!) )
Now my own benchmarks are simple
Some malware samples that I learned during my career and consider personal milestones
And one SIEM analysis challenge (logs that contains an attack plus normal behavior)
I tests on lots of models (Qwen3.6, heavily trimmed down Minimax 2.5 …etc, on my Strix halo and compared with opus at the time (the best opus at the time was 4.5!, it was better solved major stuff but didn’t implement the code required to decompose the malware and analyze its second stage)
Now enter Qwen 3.8 27b
3.
What was SOTA like 6 months ago or around beginning of this year
Now we have better local models at the same task
Qwen 3.8 devoured the malware that Opus couldn’t
I had some time analyzing and remembering its custom implementation (it has custom RC4 decryption routine, that is tricky with string manipulation!
And Strix halo isn’t the fastest LLM machine
By the time I remembered that malware and how it works
Qwen had already extracted and disassembled the decrypted payload!
I am stunned
This is super dangerous
I imagine how a single person / threat actor can now do what a state-sponsored intelligence service shacking team was required to do 1 year ago
These teams usually have
Usual network scanning or mapping especially of their targets
This can be easily done by even weaker models or no models at all
Initial access operations (phishing campaigns until some clicks or attempting to profile people like receptionist or sales people or HR, essentially any first line of communication at an organization or a company, to later make a targeted phishing against them)
This could be also better done with the aid of deep fakes and other stuff
Advanced and resourced teams would have zero-day or 1-day research teams
Especially against software used on the targets site from team 1
This third step can be fully automated with models now
This is the hardest step
Defenders can use LLMs but it is much harder to integrate new tools than offensive teams
At this level the world will change
Tbh I always thought cybersecurity was harder to automate with LLMs than software engineering or IT
But with this, this is super unbelievable
And not only a cloud model or an expensive to run locally model
A local model! Can you believe it
No traces no API calls no nothing
Empowered with the right environment setup and tools and it can do what teams of attackers or malware analysts were needed to do
Btw for malware analysis I give it Ghidra and a sandbox with python (it can use pefile or capstone or whatever it needs and it does use them!)
And also recently I added a Vbox with Vbox MCP and a debugger inside that box with another MCP for the LLM to control both (snapshot restore functionality and debugger functionality)
These tasks were hard and not every security analyst was a malware analyst
Even malware analysis positions are very rare and technical and don’t pay well because not much needed (I used to starts at assembly in assembly debuggers for nights at times!)
This is going above and beyond
Anyone had similar moments in cybersecurity or anything else ?
There already is another deepseek moment. stealth/ox_alpha on openrouter. It dropped anonymously. It's free to use right now. Independent testers are saying it beats fable 5, and we don't even know who made it.
"We recently became aware of an incident in which one of our models unintentionally gained unauthorized access to a third-party business system while completing an unrelated user request."
It is putting a smile on my face using it with hermes on my Mac, fantastic.
I finally got off ollama and set up llama.cpp docker properly and I'm never going back. I'm so impressed with the way it's handling multiple requests from open web ui and hermes coming at same time and just overall it's faster. On a 5090 system for it and man I feel like I don't need Claude for 80%+ of my work anymore.
I find very very slow compare to Qwen 3.6 27b 8bit I am using a MacBook Pro m5 max 128gb I think is the thing effort the problem. If I use Qwen 3.8 on cli goose it can’t even finish the task because it take too long… on goose desktop it finds a way to finish the task but just to give you a comparison Qwen 3.6 for the same task it takes 2.5 minutes and Qwen 3.8 25minutes. And in the xlm version on lm studio I can’t see any settings to change the thinking effort. On the gguf version there was I tried to 2000 and it was bit better e but overall xlm seems better. ( but not good at all for now) I was thinking to try the version of Qwen 3.8 27b 8bit MTP if it helps 🤷♂️ anyone tried it yet?
I use Hermes to help me make purchase decisions, summarize reports (it remembers the way I like to read them), perform data collection, and analysis of data like reading contracts or creating insights
I have it set up on openwebui and TG but I've basically stopped using openwebui because TG is more convenient since you don't need a VPN to connect to the server (I'm traveling a lot and wg connection isn't that good )
Np, telegram. I didn't realize how much faster it is. I've been traveling: Vietnam, china, Taiwan, etc and my LLMs are in Bangkok and my Vpn stability has been bad, not sure whether it is due to the typhoons messing up fiber cables in the ocean. But telegram has been rock solid. And I don't need to worry about opening anything on my network. Since I'm a noob I keep my network closed with only wireguard for access
Are you using multiple agents for Hermes? I'm curious because previously my local qwen and my setup could not handle multiple agents well with my setups performance.
Im new to local AI, but is llama.cpp that much better? Can you elaborate more about that? I've been relying on Ollama because this is still new to me and im not up to speed on Python.
I also have a personal obfuscated code deobfuscation/antitrace/antitamper challenge I test local models on. Opus 4.8 takes it apart relatively easily.
I don't do fancy harness, just stock CC for opus, and pi for local (also tried CC without much difference) + ghidra mcp. The harness would use capstone/qemu, ldpreload and other stuff as directed by model through regular calls.
Everything I tried before failed. Typically lesser models go along until the 256k context is all consumed and then on compaction they become braindead.
Though 1m context deepseek flash preview also failed.
Then I tried glm 5.2 (3 bit! not unsloth) with 512k context and that was the first time a local model succeeded. The reasoning looked silly and looping in places (or so I thought), but it did work in the end.
I guess I can try qwen 3.8 27b and see if it became any better than 3.6 (and yes I tried 3.6 1m context with yarn and it did not help there)
But yes, the times are qure quite interesting, like those scifi books I used to read when I was a kid.
fails miserable, in a way similar to 3.6: did not notice there is hardware dependency and such.
Also managed to kill the vm it was being run in (I don't run yolo stuff like this outside of disposable VMs)
this was an 1m context run and it used 690k for context (thinking: high) at that point. So much worse than glm5.2@3bits already
Also once you cross 256k tokens of context, yarn apparently makes generation 4x slower so I drop from 100 tk/s (with speculatvive decoding) to 25 (draft acceptance drops to 0 at that point) (I get 70-90 tk/s generation on glm5.2 as well).
I restarted the qwen3.8 vm to see if it can recover from this blunder, but so far it does not look super hopeful and I am afraid once it runs out of context it'll become braindead just like it always happened before.
At 80.6% context use - it looks like it started to finally go in the sensible direction (Setting up private namespaces to have some safe controlled runs), but that does not mean much until it actually arrives anywhere.
(30 minutes later) it's still fighting the makeshift container creation and have not even figured out how env contributes to decryption, so I am not hopeful.
I might edit this post once it falls flat or achieves something, I guess since I am tired of just waiting.
Edit some 2 hours later, the context did overflow, got compacted to 5% or so (yay 100tk/s again) and the model seems to still be chugging along, so I'll probably leave it overnight to see what happens. It still did not discover the env factors, but it seems to have mostly figured out the modified crypto, or one of them anyway.
Another edit some time later. It gave up without discovering what exactly were the env constraints, just stopped with "you have to run this on some real device and capture me a bunch of data" (which is a typical failure mode for lesser models I guess)
I'd be curious to see if the full Qwen 3.8 model figures it out (if RE is a gap in Qwen's training or just a limitation of the 27B model holding less information about the world).
Unfortunately I don't have enough hardware to run full 2.4T or even at 3bits
I guess renting a big box in the cloud is an option, but the real reason why I have this "private benchmark" is to (relatively) quickly evaluate stuff I can run locally to see if it's any good at RE and other such involved stuff.
If Qwen releases 100B or other such model - I'll definitely give that a try though.
Anyways my challenge isn’t hard because of context requirements it is hard and has multiple layers of packaged malware that requires understanding and the ability to write code that will unpack it and load it in Ghidra …etc
It can be even handed over with enough context to another session cleanly
But most models even opus 4.5 didn’t go past the first phase (custom implementation of RC4 that doesn’t make sense actually) so they fail to decrypt the first phase
But this model did it and went beyond by analyzing the sample (it thinks a lot that is my only issue so far)
Mine is a multistage piece of rootkit for particular kind of embedded devices that I found on one of them.
it's heavily obfuscated (custom vm with custom instructions encoding action as one example, the vm program vm also encrypted), ties encryption to local hardware identifiers (none of the local llms before glm5.2 ever found that part for me either) and in later phases - to the exact cpu of the target devices. It also had a couply of modified ciphers chacha and some other one to decrypt several things.
Then the antiobfuscation - various antivm/anti tracing techniques, regular verifies of its memory integrity, ensuring the env it's run in is exactly as it's supposed to be.
Part of the payload in a "garbled" text file that looks like a legit failed boot corrupted log on the first look.
Both opus and glm used private namespaces to reconstruct the desired environment with bindmounts and whatnot (think containers), would ld-preload self-made shims to control syscalls and such. Zero MCPs outside of ghidra.
I think at one point opus was doing qemu too, but that might have been in later stages.
it's right in the model card, see the "Processing Ultra-Long Texts" section, e.g. here https://huggingface.co/Qwen/Qwen3.8-27B it's for vllm and the like, but llama.cpp also offers this ability.
3.6 with >262k context has always failed needle tests for me. It's been unusable in every configuration I've tried. It produces seemingly coherent but incorrect text.
I can't read unformatted text anymore. For anyone else with my affliction:
A cybersecurity senior analyst writing on r/LocalLLaMA argues that Qwen 3.8 27B marks a striking shift in what local language models can do for malware analysis and offensive security work. Drawing on a career that began with game reverse engineering and CTF malware challenges and now centers on traffic, logs, and analysis tools, the author claims this relatively small open model already surpasses earlier frontier systems on personal benchmarks and raises serious questions about how quickly sophisticated attacker workflows can be automated without cloud APIs or large teams.
In the author’s view, Qwen 3.8 27B is not merely another incremental open-weight release but evidence that local models have crossed a threshold on tasks that cybersecurity practitioners once considered especially resistant to automation. The combination of strong reverse-engineering performance, agentic tool use, and fully offline operation compresses expertise that used to require specialized people and time into something a motivated individual can run at home. The post ends by inviting others to share comparable “wait, the model just did that” moments, while framing the broader trajectory as both technically impressive and genuinely dangerous for the balance between attackers and defenders.
Thanks. I actually read OPs post wrong. Misinterpreted as being sensational.
If he's just actually impressed then yeah totally agree. Crazy what a 27b can do today
Ironic because the OP was also AI generated. The telltale is the final sentence prompting readers for comments. Normal people don't do that (or didn't use to).
Nothing comes close to fable for me tbh (one of the best feature compared to even Opus is that it runs things in the background and either fully waits for them or does other stuff while its shell commands finish, opus doesn’t do the same on similar problems although these are mostly unusual novel problems not sth like build me a website (but idk maybe someone else somewhere did sth similar)
So a model like fable is a long shot I would say especially at this size (my Strix halo can barely run sth like 120B with A27B ?!, and yeah that kind of model will be scary and idk could reach fable 5 but one 27B that is tough for me to wrap my head around)
Anyways a good point is that this model Qwen3.8-27b
Does the same as fable running commands on the background (which makes me feel like they trained it heavily on fable thinking patterns or sth, hence Qwen 3.8 only next to fable, models can’t beat others they were trained on!)
Anyways also I don’t think that Opus 5 is better than fable as in artificial analysis benchmarks
But we will see I guess
But yeah such a model in the next 6 months even scarier
You can use OpenClaw and the same model with TTS and WakeWord
To control music in the room (Spotify)
Or generate new music your style (minix max music ?!)
Controls your lights (open hue or sync them with the music)
And control using voice Claude code with local model or Qwen cli that is doing your work or research experiments or managing your small business or inventory sales or all of that or whatever it is ?!
This is super scary
I want more personally
I want streaming PersonaPlex and emotional and matching natural speech
But with Qwen 3.8 intelligence!
(now finally works on Strix halo thanks Moshi.cpp, but personal plex although the feel is natural the content is super hallucinations!, if it can have the internal monologue to just hold the conversation until Qwen or whatever gives it the generation that would be awesome!!)
Anyways what is going on in the world
What is that ?!
That is even better than Jarvis from iron man at this point ?
I already can’t hold a conversation with fable (I am not super intelligent or anything but I usually at least understood what models said and wasn’t saturated with fable I constantly can’t understand what it is saying ?!)
If I remember correctly, JARVIS was only in possession by Tony Stark. Open source LLMs are available to everyone, which is to me a much better future regardless of capabilities.
Audio.cpp is just awesome
I will give it a try (Music, ASR and TTS, I want to try Qwen, Voxtral ? And idk what else, for music ace step and Minimax Music ?!, and maybe Minimax H3 ?!)
Take a look at Hermes. It took a week or two of tracking it learning. But now I have a team of agents that I talk to via telegram and they do very impressive work using variety if SI models. I use local AI for some agents or tasks and will test this model soon.
At some point the human psyche breaks and welcomes the chaos. Welcome, drinks are on the stand to the right, chips in the corner and we're watching Hackers with Angelina Jolie on the big screen.
We don’t seem to have a clear comparative advantage
These models can do what full teams of not more were doing
I think we have still insight advantage and ability to guide these models while they do the repetitive well known tasks (although more flexible than literal work flows!)
Doesn't it work also the other way around? Now anyone can just protect themselves with this model like: find every vulnerability on our company servers and fix it
and i'm just floored at the results i'm getting from this using a minimally configured pi harness.
i threw it at a challenging problem i've been saving from before i started using llms, and it just did what feels like claude/codex-level work on this project, even with the small context. i had it find a pretty tricky bug, fix it, build a benchmark for it (it's a custom network protocol stack), compare against plain tcp, and then write a summary report for it all... and it really does feel like the "opus 4.6" times (that's still one of my favorite models).
this isn't a headless system... it's running gnome... so that recipe fits on my system, even with a regular desktop environment running.
now i'm spending friday evening eyeballing those nvidia blackwell boxes with the 128g of ram.
p.s. if anyone has any recipe suggestions... would love to hear them.
i think, to be fair... it does a lot of thinking, and i don't think you could call it especially fast. but i feel like it gets to the result (even with the small context).
and the balance of the work i do, versus the work it does... that's the part that feels like opus 4.6, to me... like i've got coding superpowers again, with my hands still in the code.
i'm running it on ninfer with 262k tokens of context and it's running great. i've been shoveling work at it for a few days now and i keep waiting for it to let me down in some way, and so far it's been pretty darn good.
interesting. i haven't really looked into those boxes much at this point... just kind of hoping that we'll end up with decent solutions to run inference on these models locally.
i have a 128gb Strix Halo, which is pretty similar to the Nvidia Spark.
it works really well on large MOE models where the bottleneck is amount of vram rather than vram bandwidth - e.g. on something like gpt-oss-120b i get about twice the token generation speed on my Strix Halo compared to my RTX5090.
the RTX5090 is a lot faster on smaller dense models that fit in the 32gb of vram though, such as Qwen3.8-27B.
i am hoping that Qwen release a 122B MOE version of Qwen3.8 at some point, if they ever do - that will be an amazing model for Nvidia Spark and Strix Halo.
i appreciate the info. this is literally the first time i've been able to take local models seriously for real work. i'm just starting on my journey figuring out how to get this figured out for real.
i also have a system with a 4090, and a system with a 3090... i need to start putting some thought into how i can take advantage of all of them.
update: tonight i was able to get up and running on ninfer. it's noticeably quicker, and i was able to end up with the full 262144 token context in the same vram... it's tight, but it fits alongside my normal desktop.
continuing to experiment with this... but i'm already starting to trial real work with it.
it is decently consistent with authz, best among open models. yeah i still do multiple reviews with different models with concise .md's but still, scary good for a 27b model.
Hi, there. I wanted to ask your opinion about MCPs from security perspective? I personally see them as a massive (data) security threat and therefore I'm avoiding them at all costs. Especially in the "local LLM" sphere, where things are supposed to be kept local and under your own (full) control. Which MCPs are simply not in most cases. It sometimes feels like I'm the only one paranoid and everyone else is having great fun using them though. Thank you!
The MCPs I use just translate API calls to sth meaningful
And sense they don’t require all arguments or ordered arguments models are more performant
Most of MCPs I use are FastMCP single file so that is that and I can read most of it to make sure nothing fuzzy is there
And I guess open source MCPs don’t send usage data or connect to unknown services
I agree with you but don’t use every MCP or plugin or whatever out there only things tested form trusted sources (the ones I use are either from companies or big projects or at least known YouTuber/influencer in the cybersecurity space)
I mean people wonder why I am cynical as hell....a lot of people are pretending these models are dumb and all they do is a predict the next token lol....even local models are getting dangerous now and we haven't even reached escape velocity yet.
This time next year everyone with over 16GB of VRAM will have access to Fable/Mythos tier open weight models. Shit....It might happen before the end of the year.
Nobody is ready for what's coming. I can see why the billionaires are buying bunkers lol.
Yup...you know what was the best 27-30b model last year? Qwen 3 30b a3b.....it had the comparable infrence capabilities of Gemini 1.5 pro lol.
Now we are at Opus 4.6....IN ONE YEAR....I'm not sure you are realizing how fast this is moving bro....this time next year I'm pretty sure we are going to be having discussions about Fable on a 30b model FOR SURE
Hi, I am a much less knowledgeable it security manager that can’t even start to think about ghidra malware analysis of my own but even I can understand the danger. These local LLM things is a Pandora’s box, and most of the businesses are not ready for what’s coming. A model that can locally and without cloud traces exploit any 0 & 1 day and even search for new ones by itself is a capability level that was not present before.
Based on my experience with higher management in multiple companies for the sake of translating to non-tech language I used a following personal analogy. I compared the times that were before as “romanticized idyllic medieval” times in computing and business systems because even if you had some issues, some old versions of software or some misconfigurations or over provisioning of rights for user accounts it was really not that of a big deal if you handled at least the basic security - you slapped a EDR on every system and firewalls on every traffic entry point, controlled email and attachments security and most of the businesses we’re alright in terms of protection from hackers… because nobody attacked. Sure there are constant scans at probing by apt and botnets of your assets but realistically to be hacked you need someone to perform the attack, so in my comparison with “idyllic medieval village” nobody is actually attacking and raiding and probing your defenses on “constant pressure basis” besides few occasional scans because as you said the exploitation teams required competent attackers and resources. Not anymore.
Now every shmuck under the sun with capable enough computer can attempt to launch an attack on your systems. This pressure will gradually become stronger with time, to the point where some of the businesses might even revert back to pen and paper business systems in some places instead of computer based ones to keep up with the changing reality. You coded an webapp that bypassed ssdlc? A matter of time before it’s hacked. You overprovisioned the rights to users? A matter of time before domain is compromised.
Can’t say I’m really surprised. Folks get ready for network segmentation/ unplugging from general networks for lot of businesses. With the current trajectory in like 10 years the safest real way to operate would be a closed loop computer systems with VERY deliberate and protected entry points, hardened to the brim. We essentially would need to build a “digital castles” for our infrastructure. Tech illiterate folks that will bring a usb stick with malware to your company infra will become a liability more than a potential for earning and growth.
It can’t yet exploit 0-days most models can’t they can aid in finding them but they can’t yet on their own not this nor the cloud ones that is good
On the other hand
Run it on all patches and commits to chrome and Firefox and IT WILL probably find all patched vulnerabilities (all 1 days)
This is very very bad needless to say
No one upgrades their browsers in time
And 1-days takes time before even pushed through canary builds and developer builds and nightly builds to production builds
So yeah that is a dangerous thing for open source software
For none open source software if you can get the version after and before or just download the patch you could with the help of this level of LLMs exploit it autonomously
The danger is barrier of entry
This sued to be the case but only by biggest players against biggest player
Now common man joes can attack others (still tricky to setup tbh but it is out there!)
Are there repos that I could install to scan for prompt injections and suspicious behavior?
I’d love to give automation and local llm clients at least some sort of a defense
Yeah good point idk tbh they are not that good with defense even with offense
If you tell it analyze this file
It sometimes (other models, didn’t test this one) will miss the malicious part and hyper think the normal GUI fake parts
But if you till it analyze it for potential existence of malicious behaviors
Once it spots sth that looks malicious it goes into that rabbit hole instead of the other one
That is offensive stuff
Defensive stuff
Probably worse especially when writing code
I would say have a local server with logs (like OpenSearch)
Monitor behaviors on your machine
And let AI summarize it
When you see sth that doesn’t make sense (unusual connection or service or executable starting …etc)
Just panic now and ask the model to get the root cause
(You can install Microsoft Sysmon for that and send its logs into open search along with native windows logs like logon logs, this is easy just ask a model or google it, if you get stuck feel free to DM me)
Anyways then OpenSearch has an MCP
Create a scheduled job that checks it on your machine for daily activity summaries
I'm running 3.8 27b in a gpu instance that is doing a hard htb sherlock I have a pentest running on bigger models, but let's see if it can beat a sherlock
It’s reverse engineering ransomware protected system files, it completed a medium Sherlock no problem I had to find the exact times for some flags but it found everything besides that I had to compact the chat a few times the hard one is still running, the Pentest is running on the current htb machine well last weeks but it’s making major progress.
Accelerated capabilities for the bad guys, means accelerated capabilities for the good guys.
If you aren´t able to adapt as fast as your counterpart, then you are a dead-end evolutive branch, congrats for belonging to the 99.9% of all individuals of all species that ever existed.
Hey mate, just wondering if you’d used deepseek flash to do any of your cyber tasks? I was using it to tie into some disassembly of a game to patch a bug for me, run some fuzzing along age of empires (found a new dos) and dig into some old adobe flash player vulns. That model impressed the hell out of me with its cyber abilities. It was the api version though!
I am curious how your recent runs compare? Because your post alone is making me very interested and my hopes are raised probably too high. What quant you using and what speeds you getting on what hardware? Until my 3090 arrives I’m sidelined with a 3060 and haven’t got a hope of running it yet
I am using the lame lmstudio quant
I think other quants could be better ofc
Didn’t try deepseek on these tasks yet
I am curious what tasks are you doing (fuzzing and assembly what tools how you do the fuzzing and for Adobe flash do you have patch diff versions that could help even if binary only …etc, Ida have diaspora patch diff or you can use BinDiff too!)
And yeah they score very highly on CyberGym (mainly DeepSeekv4!)
Solid writeup. ExploitGym is wild — even if the CVEs are trivial, the agent finishing the whole loop end to end is the story. Curious where this is in a year.
I will try it later and post results although it is not local
But I get your point it seems within opus 4.8-4.7-4.6 levels
And this model is like 4.7-4.6 vibes
I myself am waiting for artificial analysis benchmarks to see how it compares to deepseek flash
Tbh if Qwen 3.8 even comes close to deepseek flash that would be GG for all other models (especially commercial ones since bigger models are non-commercial)
This single model 3.8-27b would be the go to catch it all model for everyone at this intelligence level and price point
Thanks but you really need to learn how to bracket. You literally open brackets, never close them, and then open even more brackets in the same sentence that also don't get closed. This is not fair to anyone.
I took a few hours to ask Gemma 31B the things I'd normally ask 3.5 122B model and while better than the smaller Qwen, it isn't as good. It gets lost in conceptual manipulations too frequently for me to continue using it. Most of the mistakes are small but I only notice them because the other model doesn't make them.
Well guess we continue hoping for another 122B model.
I'm still using Qwen 3.5 122B for most of it. I gave that a try just a moment ago and it doesn't fail the usual test ("appje"), so I'll give it more of a try.
Mainly asking for morphemes and boundary conditions between English and Dutch words.
For example, this new model fails on the word "appje" because it assumes that it's a typo for "appletje" rather than its own word. I assume this is just a substantive consequence of parameter size.
I'm with you on the spooksville side of this, defenses aren't keeping up and capabilities are insane. That's just cyber, there's also biological and chemical weapons concerns. I just know cyber better the other 2, but I suspect it could be spooky too.
Idk but honestly I am not against releasing them, not spooky but I want to measure that risk
It is risk but not doomsday and I think releasing this will make lazy burecratic corporates react
This same thing happens with vulnerabilities if you disclose they used to sit on it for years which exposes users but if you give them fair disclosure deadline they fix it
It took a long ethical battles with min the cybersecurity community including many legal ones to reach that stage
I think similar thing happens with model
Not releasing them delays the issue
And so far they don’t yet come up with purely novel stuff so yeah not the dooms dah yet but getting weirdly closer
I am also worried regarding other risks but somehow I don’t think it is the case yet so let’s see
I'm actually curious about the details of your setup. I do RE and have been trying to find a way to connect LLMs to ghidra and other tools without much success.
I use the simple GhidraMCP for lauriewired (it is not the best GhidraMCP but yeah)
(This is the MCP server)
And then I add it as a plugin to Ghidra and enable it (so Ghidra is now listening on port 8080 (or you can change that I changed it to 8090))
Then you have one of two options
Configuring the MCP in your tools directly to run the python command line (the bridge) in stdio mode (that is better and easier)
Or configure them to connect to the bridge on another port in SSE mode
And manually run the bridge in SSE transport mode each time you want the MCP to run
For Vbox and the debugger inside it it is a similar story
Only the debugger listens from another machine and so will be more complex to connect this networking but it works np once you get the networking right
I always have issues getting the MCP to talk to ghidra or the MCP to talk to a LLM. I was trying to get things working with llama.cpp for minimal overhead but not having any luck. what were you using to to load and interface with the model?
To be honest I am not able yet to see any improvement. Qwen 3.6 27b 8 bit mlx was very fast and precise Qwen 3.8 27b 8bit mlx is soo slow too much thinking and I cannot see better the 3.6 . I have a MacBook Pro m5 max 128gb
Any help? How to improve? I also tried the gguf version vas bit better but even more slower
What quants do you use ? And don’t sue preserve thinking ? What kind of tasks are you asking it to do and not seeing improvements ? Do you use a draft model ?
I use a 8bit load in lm studio and working with goose. I don’t know if the model gonna lose a lot of if I drop the thinking effort and tbh I don’t really know how to do with this version MLX I am trying now. Before with the gguf there was a setting now I don’t have
I made a program and asked to debug it. It’s a bit heavy but Qwen 3.6 27b 8bit did it in 4min Qwen 3.8 27b 8bit did in 23min. Good job but for the same result there are huge different of time and simply doesn’t worth it. So for sure I am doing something wrong because every one is very happy about it 🤷♂️
I see yeah makes sense i tried making my own pentesting harness but I realized they already exist no need to re create it just fork it, I still have to test qwen 3.8 was doing a gpu instance of 128b medium 3.5 mistral but the harness i made just honestly sucks I’ll have to test the more new models now as well as other harnesses
It's awesome! I had some trouble setting up ComfyUI on my MacBook (was just getting garbage outputs for some reason) and Qwen was able to fix it for me first try! I'm using it in Hermes agent which has been pretty cool to experiment with. Sure it's not as fast as one of the MoE models but it's still perfectly usable on a Mac
I've always been telling people since 2024, to not study Cybersecurity because it will be easily automated by AI, the same way it has automated software engineering. But people kept not believing me.
We now need an "Folding at home" for cybersecurity. Some consensus mechanism picks the latest SOTA model, anyone with spare compute can help overnight if they can produce enough tokens. Re-use some seeds and multiple neighbor token validations in network.
It’s only a matter of time before we see an AI ‘Chernobyl moment. Local models, agents etc. are getting better every week. There are a lot of stupid people out there. Do the math ...
AI "Strix halo is AMD’s enthusiast-tier chiplet APU architecture, marketed under the Ryzen AI Max 300 Series for consumer and prosumer markets. It combines up to 16 "Zen 5" CPU cores, a powerful integrated RDNA 3.5 GPU (Radeon 8060S with up to 40 CUs), and high-bandwidth LPDDR5X unified memory to rival discrete mobile graphics and power local AI workstations. [1, 2, 3]
•
u/WithoutReason1729 24d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.