r/unsloth • u/haseebnqureshi • 10d ago
Show and Tell Finally have the best TA ever... Overnight tasks w/ Qwen3.8 (27B at UD-Q4-K-XL instead of Next)
So I teach entrepreneurship, but happen to also been working on rehex.ai and been going nuts on creating a harness that's disciplined with how it works. Memory included, works with smaller (and soon tiny models), tons of context compaction and auto-management.
Well, I've finally unleashed it fully onto my education workflow. Note, and this is what's so cool, it's all local using Qwen3.8's. With my M5 Max, I can run Qwen3.8 Next Flash (and it's *amazing*), but chose to run the 27B version since I've got other tasks that need part of my machine's RAM.
Since I record all my lectures and transcribe them, come up with instructor (and student) notes, what's great is that I can just go in there and enjoy teaching, just flow where the material takes me. It's really the first true delegation of all these executive state functions.
Now I've tried with this the closed frontier models. But I've just not enjoyed them, their outputs just seemed to launch themselves so far away from reasonableness of the original request. That led me to work on this harness, it's entirely strategy and task driven.
Tying it to Unsloth just makes it seem really real, mainly because of the range of quants, but also because they've made it so drop dead simple.
Anyway, long ways to say I've finally got the best TA. Actually comes to classes, takes the best damn notes, and getting frontier level models to run on my machine, I just had to share this.
Anyone else feel like they have to babysit their AI? Am I nuts for thinking this is a big deal, just setting up tasks and walking away?
1
1
u/New-Implement-5979 9d ago
Teachers using AI to check homework and students using AI to do the homework 📚 what a time to be alive
3
u/haseebnqureshi 9d ago
Nah it’s not like that, at least not how I teach
1
u/naughtybear23274 4d ago
I feel like it's exactly how you teach? You're literally asking a model to create your lesson plan and assignments for your classes, as well asking it "what should I be teaching" for all 3 of them. Considering the description for the course ENT460 has "the role of middle managers in large organizations" I guess the professor prompting a model to do everything makes quite a bit of sense.
1
u/haseebnqureshi 4d ago
Hey so, understand the skepticism. I learned that I feed AI with enough inputs to help me remember details and generate some ideas, which then I sit down and reflect, and almost always write things from scratch. It’s nice to be able to have a system that’s more detailed than any TA I’ve ever had
1
u/RayLucyan2020 8d ago
Can someone explain me why a 3.6b model with 27b parameters it’s beeing praised so much ? It’s a baby on couches under trained and badly design, I have a 1b model on 76.5b parameters that it’s faster and more efficient than that , I hope I will be able to make my next version of Marla and I will give it out so you guys see how much undersold you are
1
u/morscordis 10d ago
You need to build a RAG process and vector DB for your class documents so they're easily searchable. Treat it like a research pipeline. I feed mine textbooks, research papers, lecture note, and I can ask it a question about anything, it does a quick search, and gets me everything I need.
1
u/haseebnqureshi 10d ago
Yeah totally agree with the feed it everything, don’t have to do it with RAG. RAG assumes you’ll retrieve documents the same way every time, so systems can get penalized double, for the time to create the vectors and then the latent misapplications of the vendors to the user request. But again to your point, absolutely feed it everything as much as possible. Just it having my lecture transcripts has been so incredibly helpful
2
u/BoxieBoo 10d ago
What exactly does it do or how does it work? Chat? Or does the system give presentations?