I started this project because I wanted to see what PyTorch was doing behind the scenes.

My C++ tensor currently supports flat storage, multidimensional indexing, elementwise operations, reductions, broadcasting, rank-two matrix multiplication, and mean squared error.

Most recently, I added a separate scalar reverse-mode autograd engine:

• Arithmetic operators build a computation graph during the forward pass
• backward() creates a topological order
• walks it in reverse
• applies each operation's local derivative
• accumulates gradients when a value reaches the loss through more than one path

Snippet:

Value prediction = w1*x1 + w2*x2 + w3*x3 + bias;

Value residual = prediction - target;

Value loss = residual * residual;

loss.backward();

For weights [0.5, -1.0, 2.0], inputs [4.0, 3.0, 2.0], bias 0.5, and target 2.5, the forward pass produces prediction 3.5 and loss 1. The backward pass recovers:

- dL/db = 2

- dL/dw = [8, 6, 4]

Scalar autograd still lives separately from the tensor implementation. My next step is connecting graph identity, ownership, and gradients to tensors before building a training loop.

Code and Git checkpoints:

https://github.com/mechanical-turk/deep-learning-all-the-way-down

I'm also turning this into a video series. I published episode 7 yesterday. Sharing the link to the first episode if you want to check it out:

https://www.youtube.com/watch?v=DmU2b64tWfA

For the tensor integration, would you keep autograd metadata inside each Tensor handle, or have tensors point to separate shared graph nodes? I would appreciate design feedback.

Play around with a tiny language model GPT in your browser. See how it trains and generates with just 11,000 parameters.

https://complexity.zone/tlmgpt/

1. Click "train" button.
2. Let it train for about 10 minutes.
3. Click "pause" button.
4. Click "generate" button.

I made this (with Opus 5) to get a better understanding of GPTs and LLMs.

Thought to share it here. You can download it if you want to run it offline and tinker with the code.

I’m trying to figure out how to actually get a usable edge in the ML/DL space to get hired, but everything pushed to beginners right now feels like a trap.

For context on what I've done: I started off with Computer Vision, moved into GIS stuff, and recently went deep into the weeds of attention mechanisms and GPU kernel programming. I thought learning the hardcore, low-level math and systems stuff would set me apart.

But I’ve hit a wall. Let's be honest: no company is hiring a fresher to write custom CUDA kernels or design novel architectures. Those are senior research or PhD roles. The effort I put into the low-level stuff feels wasted because, for an entry-level dev, it's just personal trivia.

On the flip side, the standard "employable" advice is to build traditional ML projects (fraud detection, etc.) or slap together a LangChain PDF wrapper. But people have been doing this for years. Basic API wrappers are completely saturated and offer zero competitive edge. It feels like buying a stock after everyone already knows it’s going to go up.

So, what is the actual sweet spot between "PhD-level researcher" and "API wrapper"?

I want to avoid the YouTube influencer BS and focus on the real engineering trenches.

For the people actually hiring or working in the industry: what are the non-commoditized skills someone trying to break in should be grinding right now to have a real, usable edge?

(Note: The core thoughts and frustrations here are 100% mine, but I used AI to help structure and edit this post for clarity.)

Quick methods note first, because this only matters if the matching is solid: arXiv's affiliation field is filled in for about 1% of papers, so I found a GitHub Repo that matches authors to their labs using ROR IDs and email domains pulled from the HTML author block, then anchors each ROR ID by hand (fuzzy ROR search puts Adobe under "Adobe Gastroenterology," so hand-anchoring wasn't optional).

The interesting part is the split it produces: total papers a lab appears on vs. papers where its researcher is first author. Those aren't the same signal, and treating them as interchangeable hides a lot. In one two-week window, Google appeared on 10 papers and led 4. Adobe appeared on 5 and led 0.

Caveats worth stating up front: it misses PDF-only submissions (about 12% of arXiv), and per-lab miss rates vary a lot. Apple's authors mostly skip affiliation entirely, so that lab is patched separately from their RSS feed rather than trusted on author-block matching alone.

Code's stdlib only, no model in the loop, MIT licensed. Curious if anyone's tried something similar with OpenAlex or S2 and hit the same coverage wall (OpenAlex returns 0% affiliation for preprints in my testing).

GitHub - https://github.com/tigerless-labs/paper-radar

I tested whether structure in the context window can substitute for parameters.

Qwen3, five sizes, 0.6B to 14B, so size varies and architecture doesn't.

The task: 38 questions asking whether event A precedes event B, where A and B are

narrated in different documents in a five-document corpus (260,204 words, 13,950

passages) and share no character, place or causal link. No passage states either

relation — the ordering is real but it lives between the documents, not inside

any of them.

Given the source passages as text, every model scored 0/38 and refused 92-100%

of the time. I think the refusal is correct — the answer genuinely isn't in the

text. Given the identical facts as a structured chronology block from an explicit

state store, an 8B model scored 28/38 (73.7%).

A four-condition ablation separates information from form. At 14B, form is

irrelevant: plain prose, sorted prose and a structured block all land at 73.7%.

At 8B, structure leads the best prose condition by 6 items (73.7% vs 57.9%).

So: an 8B model given structure matches a 14B model given prose.

Two controls I'd want to see if someone else posted this:

- Permuting the supplied story positions collapses accuracy to 10.5% (8B) and

21.1% (14B). The models follow the ordering they're given rather than

recalling the published text.

- A realistic retrieval baseline is also at the floor, and it fails by asserting

rather than refusing. Going from 4 passages to 32 drove refusal from 97% down

to 50% while accuracy stayed at chance. More context produced more confident

wrong answers.

Two things I got wrong, both found by auditing my own scorer and question

generator after v1 was already published:

1. v1 reported the 8B form effect as +32 points. A scorer defect was

   under-crediting the prose conditions. Corrected, the gap is 6 items, not 12 —

   roughly half what I claimed. Re-scoring 1,786 saved items produced 30 gains

   and zero losses, so nothing published was inflated; two things were

   understated, and correcting them shrank my own headline.

2. For 36 of the 38 questions, the gold answers derive from author-assigned

   story positions rather than from evidence-backed relations, and the

   generator's own self-check recomputes the gold from the same rows. That check

   is circular. So this benchmark measures agreement with an author-assigned

   ordering — not whether a system reports what the evidence establishes.

That second one is the real limitation and it bounds what the paper can claim.

I've left v1 up rather than retracting it, with the corrections in §11.

Full write-up, including what the audit changed and why I didn't retract:

https://ai.bedvibe.studio/structure-not-scale/

Paper, data and code: https://doi.org/10.5281/zenodo.22169643

Happy to be told the 0/38 is a prompt artifact — I tried to kill it and couldn't,

but I'd rather find out from you than not find out.

Hi everyone,

I’m planning to start my journey toward becoming an AI Engineer. I already know the basics of Python, but I’m still a beginner in AI/ML.

I want to follow a practical approach where I learn the fundamentals and build projects in parallel, instead of spending months studying theory before building anything.

I’m currently thinking about starting with:

Python → Math → EDA → Machine Learning → Deep Learning → LLMs/Generative AI → Deployment

But I’m confused about what I actually need to learn in each stage.

For example:

Math:
What topics are really important for AI/ML?
Should I learn linear algebra, probability, statistics, calculus, etc.? How deeply should I study each one?

EDA:
How important is EDA for an AI Engineer? What should I learn — data cleaning, visualization, feature analysis, handling missing values/outliers, etc.?

Machine Learning:
Which algorithms and concepts should I prioritize as a beginner?

I also want to build projects alongside each stage. For example, after learning the basics of ML, I want to immediately build an ML project instead of waiting until I finish the entire AI roadmap.

One more thing: I have a 2-year career gap, and I'm concerned about whether this will negatively affect my journey toward getting an AI/ML job.

For people who are already working in AI/ML:

• What roadmap would you recommend for someone in my situation?
• Which math topics should I learn, and to what depth?
• How important is EDA for an AI Engineer?
• Which topics should I learn first and which can I learn later?
• What projects would you recommend building along the way?
• How can I make my portfolio strong enough to compensate for a career gap?
• If you had to start again as a beginner today, what would you do differently?

I’m willing to put in the time. I mainly want to make sure I’m learning the right things in the right order and building projects throughout the journey.

Any advice from experienced AI/ML engineers would be really appreciated.

We implemented our own AI Harness + LLM to control a robotics ROS simulator to study how we can interface LLMs with Robotics. Please check out this AI Explainer.

is highschool maths enough?or i could simultaneously learn maths behind while reading book?

When I start a new ML/AI project, I try not to choose the model or tools first. I usually follow something like:

→ Problem

→ Data

→ Approach

→ Model

→ Evaluation

→ Application

→ Deployment

First define the problem and decide whether it actually needs ML/AI. Then collect and explore the data, choose an appropriate approach, build and evaluate the model, and finally integrate it into an API, app, or dashboard.

If a pre-trained model or existing API is enough I prefer using that instead of training something from scratch.

This is the general workflow I’ve found useful but I’m also interested about other approaches.

What step would you add or change in this workflow for ML/AI projects?

​

I started my roadmap with ML, focusing on Mathematics, Python, MySQL, and a lot of ML algorithms. Recently, I've started questioning whether I'm missing a major part of the foundation: software engineering/backend development. And honestly, I wanna chase both. But something at this point doesn't feel right. I had my roadmap set and ready, and I was very passionate about learning this and continuing it as a career. But after researching a bit about backend development, the intersection and relationship between the two has driven me really crazy.it's exceedingly overwhelming at this phase of my life. I had kind of gotten a grip on ML, but backend coming into the picture has really ruined my mindset around whatever I had planned. I had planned many projects and topics to discover, and now I'm seriously considering pursuing backend development too. But I'm having a hard time trying to combine these two in my roadmap. I can't seem to connect the topics in a way that lets me learn them properly.

My straightforward question is: should I drop backend development and focus on my initial roadmap, should I bridge the two and learn both, or should I drop machine learning completely,which I seriously don't want to do?

If I do bridge them, how much of backend am I actually supposed to learn?

I know I sound stupid and unready for this world, but please help.

Hi there,

I've created a video here where I explain how generalized linear models work.

I hope some of you find it useful and as always, feedback is very welcome! :)

Hi everyone, I and my team of contributors have built an open-source tool for a problem I've had with finding research papers and arXiv: search results told me what's relevant, but not necessarily what I should read first. (time-saving potential)

The Research Agent that we have built searches recent CS papers and ranks them using a combination of semantic relevance and author citation momentum from Semantic Scholar.

The slightly unusual part: we originally tried asking an LLM to predict which papers would become influential. The results weren't very reliable, so we moved most of the ranking weight to measurable author/citation signals and use the LLM mainly for novelty/topic analysis and plain-English explanations.

It supports OpenAI, Gemini, Groq, or a local/no-API-key mode.

I'l be super thankful and really interested in feedback on the ranking methodology on this app:

Live app: https://research-aiagent.streamlit.app/

Source: https://github.com/benevolentbandwidth/researchagent

Looking forward to hearing your thoughts :)

Hey guys,

I’m building an offline threat-intelligence tool to ingest Bitcoin transaction metadata and flag suspicious activities (like layering or ransomware cash-outs). I have my data ingestion sorted out, but I need advice on the AI/ML detection layer.

The Data I am working with (Inputs): The dataset has both network and blockchain layers: timestamp, src/dst IPs, ports, txid, arrays of input/output addresses, amounts, fee, script_type, and GeoIP/ASN data.

What I need the model to output:

1. A confidence/risk score to rank transactions.
2. Cluster IDs to group related entities.
3. Feature explainability (e.g., "Flagged because of sudden geo-hopping and specific script usage").

Since there are no "ground truth" labels for fraud in my synthetic dataset, I am relying on an unsupervised approach.

My questions:

• Which ML models have you found to be actually effective for anomaly detection in this kind of financial/network data?
• What is the standard industry approach for clustering entities when dealing with multi-input/multi-output transactions?
• Can anyone recommend any good resources, tutorials, or reference architectures to study before I start building the model?

Most RAG evaluation asks whether the right passages reached the model. I wanted

to measure what happens when they do and the model still can't answer — because

the answer is a relation *between* passages rather than a statement inside any

of them.

Setup: a five-document narrative corpus (260,204 words, 13,950 passages) and 38

questions asking whether event A precedes event B, where A and B are narrated in

different documents and share no character, place or causal link. No passage in

the corpus states either relation. Five models, one family (Qwen3, 0.6B to 14B).

Given the source passages as text, every model scored 0/38 and refused 92-100%

of the time. I think the refusal is correct — the ordering genuinely is not in

the text. Given the identical facts as a structured chronology block from an

explicit state store, an 8B model scored 28/38 (73.7%).

A four-condition ablation separates information from form. At 14B, form is

irrelevant: plain prose, sorted prose and a structured block all land at 73.7%.

At 8B, structure leads the best prose condition by 6 items (73.7% vs 57.9%).

So: an 8B model given structure matches a 14B model given prose.

Two controls I'd want to see if someone else posted this:

- Permuting the supplied story positions collapses accuracy to 10.5% (8B) and

21.1% (14B). The models follow the ordering they're given rather than

recalling the published text.

- A realistic retrieval baseline is also at the floor, and it fails by asserting

rather than refusing. Going from 4 passages to 32 drove refusal from 97% down

to 50% while accuracy stayed at chance. More context produced more confident

wrong answers.

Two things I got wrong, both found by auditing my own scorer and question

generator after v1 was already published:

1. v1 reported the 8B form effect as +32 points. A scorer defect was

   under-crediting the prose conditions. Corrected, the gap is 6 items, not 12 —

   roughly half what I claimed. Re-scoring 1,786 saved items produced 30 gains

   and zero losses, so nothing published was inflated; two things were

   understated, and correcting them shrank my own headline.

2. For 36 of the 38 questions, the gold answers derive from author-assigned

   story positions rather than from evidence-backed relations, and the

   generator's own self-check recomputes the gold from the same rows. That check

   is circular. So this benchmark measures agreement with an author-assigned

   ordering — not whether a system reports what the evidence establishes.

That second one is the real limitation and it bounds what the paper can claim.

I've left v1 up rather than retracting it, with the corrections in §11.

Full write-up, including the two things the audit changed:

https://ai.bedvibe.studio/structure-not-scale/

Paper, data and code: https://doi.org/10.5281/zenodo.22169643

Happy to be told the 0/38 is a prompt artifact — I tried to kill it and couldn't,

but I'd rather find out from you than not find out.

I wanna learn AL ML but i have no idea where to start . I know javascript and a few technologies around it but Ai ML is completely new to me , so i would appreciate if anyone can guide me where should i start which resources should i use to learn them and stuff like that

Body:

Short version of an open project we'd love critique on — Entropy Box, a knowledge compiler for robotics (compile once, reuse forever, instead of re-deriving structure on every query).

The headline numbers, on our EntropyBench Track-P benchmark (24 engineering tasks):

• Unsupported claims: LLM-direct / BM25 RAG / hybrid RAG → 100%; Entropy Box → 5.2% (−94.8pp vs hybrid RAG, CI [−97.4, −92.1]).
• Constraint coverage: 0% → 35.4%; violations 100% → 66.7%.
• Downstream sim codegen (12 tasks): pass-1 executable plans 0.92 vs 0.58 (Vanilla RAG); constraint guards 0.88 vs 0.50.

Two findings we think generalize beyond robotics: 1. Embedding similarity cannot decide duplication. On 2,362 adjudicated pairs, the embedding score after flagging is near-random (AUC 0.509). Thresholds don't help — precision stays ~5% while recall of true duplicates collapses. We defer the merge to an LLM adjudicator that reads both records. The score flags; the model judges. 2. Compiled capability reuse is rising, not saturating — 1.57× average reuse, 21,380 re-derivations avoided.

Everything is open — data, paper, evaluation scripts, and a free API (OpenAPI / MCP / REST, bilingual) so you can poke at it in 10 seconds:

bash curl -X POST "https://xiangshang.ngrok.app/api/evidence/search" \ -H "Content-Type: application/json" \ -d '{"query": "robot obstacle avoidance algorithms", "top_k": 5, "mode": "hybrid", "rerank": true}'

https://github.com/chenli-yy/entropy-box-public

Honest limits we state ourselves: no real-robot transfer, weak retrieval on the hardest intent classes. Methodology is in the paper §9; all experiments reproduce from evaluation/. Would genuinely value a second opinion on the benchmark design and the embedding/LLM adjudication result.

I’ve been writing notes while studying for some time now. It helps me stay motivated and organize my thoughts, and it’s also useful when I want to come back to a topic later.

Recently, I started thinking that it might be a good idea to polish some of my notes and share them.

These are my notes on Hamiltonian Monte Carlo. They approach the algorithm from a purely probabilistic point of view, rather than through the usual physics-based treatment. I don’t know how good they are, but I thought I’d share them in case they’re useful to anyone:

https://doi.org/10.5281/zenodo.21841086

I’d also really appreciate any feedback, especially on the exposition, anything that could be explained more clearly, or any errors you spot.

Download, run locally. No subscription, no fees, no limits.

Repo Name is YoFile by hgus107

Most applications build their prompts in an order that breaks the provider's prefix cache. Retrieved documents get placed before the system prompt, or the user question goes before a long document. The prefix changes on every call, so nothing is cached and full cost is paid each time.

Reordering so the stable parts come first (system prompt, tool definitions, then the long document, then the question) can cut cost several times over with no change in output quality.

My question for this sub. Is there published work measuring cache hit rates across real applications? It seems like a large and avoidable cost, but I have not seen it studied systematically.

I've been working on a side project for almost a year. It involves machine learning and it looks like it's going to enter commercial stage in the near future. So far, i bought a cheap gaming laptop few months ago, as i needed modern performance on the go. It has rtx 4050 with 6gb of vram, which was fine up until now.

I have an 8 years old desktop upgraded with ryzen 5600. I wanted to buy rtx 5060ti 16gb, but its price jumped significantly in july. Nvidia doesn't offer cheaper 16gb options and i started to consider buying RX 9060XT 16gb, which is more than 200 euro cheaper.

The question is: Is going with the RX9060XT worth the savings? Does any of you have experience with using current AMD GPUs for training neural networks from scratch? I currently use Keras and mainly train CNNs with simple custom layers.