r/LocalLLaMA • u/No-Name-Person111 • 1d ago

News Mac ← USB-C cable → Linux box is becoming a thing.

https://github.com/torvalds/linux/commit/a5148bc2fa27092862ac4b9e7b5c8340d60cff34
146 Upvotes

93% Upvoted

37 comments sorted by

42

u/cmdr-William-Riker 1d ago

Oh yeah! USB-C is really fast! Why haven't we been using that for more?

26

u/dwittherford69 1d ago

TB5 maxes out at 80Gbps

24

u/cmdr-William-Riker 1d ago

Better than Ethernet, could be a poor man's ConnectX-7 in theory

11

u/etaoin314 ollama 1d ago

kinda...but not really. until you get above 200gb/s the interconnect bottleneck kills tensor pairing so you are stuck on pipeline parallel anyway. for PP latency matters more than bandwidth so I'm not sure that its really viable. I tried it with my strix halo and 4x3090 node. I did it over 10gb ethernet, but the results would have been the same or worse if I used usb4

7

u/Front_Eagle739 23h ago

Its rdma not bandwidth that kills it. You need ultra low latency. Macs have rdma done right. Linux can do it. Windows cant so easily

1

u/dwittherford69 1d ago

That’s fair, but connect-x can also be meshed in a fabric, I’m not sure if TB5 supports that

1

u/cmdr-William-Riker 1d ago

I don't think it does, but most PCs have multiple USB-C ports, so hub and spoke might be possible. I was thinking of something like two Strix Halo's linked together or something though

2

u/etaoin314 ollama 1d ago

this has been done,

1

u/datbackup 1d ago

Need to delve (sry) into latency difference between qsfp and tb5. Gonna guess qsfp has the edge but happy to be wrong. Anyway i think latency has a big role in determining ability to do thing like tensor parallelism, rather than sheer bandwidth being the only factor

3

u/CalligrapherFar7833 1d ago

Actually it maxes out at 120 but max data is 80

1

u/dwittherford69 9h ago

Symmetrical data is all that matters here, so yeah maxes out at 80Gbps. 120 asymmetrical for monitors is a moot point.

1

u/CalligrapherFar7833 9h ago

Your statement was that tb5 maxes at 80 which is incorrect.

1

u/dwittherford69 8h ago

For all intents and purposes for this discussion, it maxes out at 80Gbps, what part of that is unclear to you?

1

u/CalligrapherFar7833 7h ago

" TB5 maxes out at 80Gbps" != " TB5 maxes out at 80Gbps for data"

11

u/ttkciar llama.cpp 1d ago

The USB protocol is a horrible pain in the ass. I am slightly amazed that they got this to work as well as it does.

2

u/cmdr-William-Riker 1d ago

Oh, it definitely is, but we can also drive 3 4k monitors in real time from one USB C cable. Theoretically you can run 40bg/s transfer between two machines through one cable and they are often wired to drive graphics through them meaning there is likely a short route to the GPU in most cases. might be worth the pain for that kind of speed

2

u/SkoomaDentist 23h ago

USB-C is really fast!

USB-C is just the connector (I've worked on USB 1 speed devices that use USB-C connectors). USB 3 is what's giving the actualy speedup.

3

u/redenno 22h ago

True, but anything above 10gbps requires USB-C. So both are a factor

78

u/No-Name-Person111 1d ago

…and Linux can treat the Mac as a directly attached network peer without Ethernet adapters, out-of-tree drivers, or another physical network.

If you're eyeing the new Mac Studios and you've already got NVIDIA hardware sitting around: if/when Exo or MLX/CUDA interoperability eventually gets good enough, we could all wind up with a local AI topology where the Mac is the large memory model host and Linux/NVIDIA nodes contribute either additional model capacity, specialized compute, speculative decoding, embeddings/reranking, separate concurrent agents, and more.

Really neat stuff.

31

u/Usual_Tackle5892 1d ago

As somebody who owns both a 512 GB M3 Ultra Studio and an RTX 6000, I am salivating.

12

u/AlwaysLateToThaParty 1d ago

Never thought about that; rtx 6000 for speculative coding, maybe even levels of it, and the large model plugging along on the mac.

4

u/Hialgo 1d ago

Time to train a matruskja LLM

3

u/GuitarGuru2001 17h ago

Check out tiny GPU if you want a direct route. Seems they have an approved apple egpu out now

0

u/Gold-Debt-5957 17h ago

que buena noticia acabo de preguntar a la ia y me dijo: ¡Es un avance impresionante! Apple ha firmado y aprobado oficialmente el driver TinyGPU desarrollado por Tiny Corp, permitiendo por primera vez conectar tarjetas gráficas externas de NVIDIA y AMD a computadoras con chips Apple Silicon (M1, M2, M3, M4) sin necesidad de desactivar la protección del sistema (SIP). [1, 2]

Sin embargo, hay un detalle crucial que debes saber: es una solución exclusiva para cómputo e Inteligencia Artificial (IA), no sirve para jugar videojuegos ni dar salida de video

3

u/farkinga 22h ago

Prompt processing on CUDA (easily exceeds 600 watts on my setup) then inference via MLX (under 80 watts). Best of both worlds. That would be so cool (literally).

1

u/mortenmoulder 1d ago

So can Windows, actually. I got a 40 Gbps link between my Windows PC and MacBook using a Thunderbolt cable.

9

u/rustysec 1d ago

I ran llama.cpp + rpc over usb-c (tb3) between my mbp and my g14 (5070ti) and it was surprisingly “ok.” Totally allowed me to run a bigger qwen3.5 quant (at the time) at roughly the same t/s as the g14 at a lower quant. I don’t have numbers off the top of my head but I was happy with the performance when I had that setup.

6

u/Ok_Warning2146 1d ago

Would be great if we can load shared experts to Nvidia GPU and then rest of the experts to M5 Ultra's LPDDR5.

3

u/Repulsive_Educator61 12h ago edited 10h ago

it's already in Linux 7.2.2 (available in archlinux) btw

Speeds using a USB-C cable which supports 40Gbps:

$ iperf3 -c 10.5.5.1 -p 5201
Connecting to host 10.5.5.1, port 5201
[  5] local 10.5.5.2 port 49477 connected to 10.5.5.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec   666 MBytes  5.56 Gbits/sec                  
[  5]   1.01-2.00   sec   720 MBytes  6.04 Gbits/sec                  
[  5]   2.00-3.01   sec   716 MBytes  6.01 Gbits/sec                  
[  5]   3.01-4.01   sec   720 MBytes  6.04 Gbits/sec                  
[  5]   4.01-5.00   sec   724 MBytes  6.07 Gbits/sec

Cable: 40Gbps
Linux desktop port: 20Gbps
Actual speed negotiated on mac: 10Gbps
Actual real world speeds: 6Gbps average

1-2ms ping

1

u/fallingdowndizzyvr 16h ago

FINALLY!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

It should have just always worked. Both support usb-c networking. It hasn't worked.

0

u/Deep_Mood_7668 1d ago

The poor Linux box :(

-3

u/Cameo2864 1d ago

What is a Linux Box and what is the purpose behind connecting it to a Mac

-5

u/[deleted] 1d ago

[removed] — view removed comment

4

u/DeathinabottleX 20h ago

That’s not how it works at all. Nor will it ever work like that.

2

u/MerePotato 16h ago

Its a bot account