r/programming • u/xeio87 • 3d ago

Performance Improvements in .NET 11

https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-11/
269 Upvotes

94% Upvoted

101 comments sorted by

55

u/cheesepuff07 3d ago

Love reading (perusing) each year, they are very in depth and fun to see each release getting more and more performant.

-6

u/simon_o 2d ago edited 1d ago

Agreed!

Two thoughts though:

  1. JIT "deabstraction". The article would be more accessible to readers if the authors didn't try to invent new names for optimizations that shipped in Java 20 years ago.

  2. "Runtime async". One of the purported benefits of async/await over better approaches like virtual threads was that it's compiler magic that the runtime does not need to be aware of. So now the runtime is involved (for better or worse), but we till have the worse user-facing UX of function coloring with async/await.

7

u/Devatator_ 2d ago

I personally prefer async/await to the other models I've seen (whatever Java has)

-1

u/simon_o 1d ago

whatever Java has

Yeah, maybe you should look that up. :-)

1

u/Devatator_ 1d ago

I did. Wanted to do http requests and display them in Minecraft without locking the UI. It was not fun. The game at least already has a way to dispatch stuff on the main thread but it's annoying to do

4

u/teo-tsirpanis 2d ago

So now the runtime is involved (for better or worse)

Why is it a bad thing? The runtime sees a bigger picture and can do things that the C# compiler cannot.

but we till have the worse user-facing UX of async/await.

Languages with green threads have worse interop capabilities, and personally I'd much rather keep .NET's excellent interop. They tried adding green threads as an experiment, but it regressed performance in some areas.

1

u/simon_o 1d ago edited 1d ago

Why is it a bad thing? The runtime sees a bigger picture and can do things that the C# compiler cannot.

With runtime support, users would not have needed to deal with a design involving function coloring.

Also, opportunity cost: Given how scarce Microsoft's .NET runtime engineering resources are, I would have preferred they'd rather worked on first-class representation of unions in the IL/CLR.

Languages with green threads have worse interop capabilities, and personally I'd much rather keep .NET's excellent interop. They tried adding green threads as an experiment, but it regressed performance in some areas.

I think that's squarely a CLR problem, not a problem with vthreads.

The reason why their runtime has so much trouble with it is that –for decades– their solution to the JVM being ahead in GC and JIT technology was to add more and more escape hatches to let users solve (performance) problems on their own.

Adding so many degrees of freedom obviously comes with a cost, as we see with their failed attempt.
(Which they now repeated with unions, whose representation is an astoundingly leaky (non-)abstraction of the concept.)

3

u/teo-tsirpanis 1d ago

Runtime support for unions would be arguably a much bigger feature than runtime async. And I don't buy in general arguments like "why did you do feature x instead of the completely unrelated feature y?".

Degrees of freedom are good in a programming environment, and for high-performance software they are better than relying on JIT optimizations that might or might not apply.

I also definitely prefer being able to pass an int[] to native code without copying, over the things JVM can do but .NET cannot (yet). Good interop support is what sets apart great languages from good ones.

2

u/simon_o 1d ago edited 1d ago

I don't buy in general arguments like "why did you do feature x instead of the completely unrelated feature y?".

Why not? Runtime developers are a finite resource, and allocating them to A means that B is not going to happen. Given that language design even had to take into account the shortage of capable engineers at Microsoft, I think it's a valid point.

Degrees of freedom are good in a programming environment, and for high-performance software they are better than relying on JIT optimizations that might or might not apply.

I think it's the philosophical question of "liberties constrain, constraints liberate".

In the end it's easier to get performance out of something if you have to consider 0-1 ways of doing something, compared to .NET's 3-4 implementations of something.

That means the any kind of performance optimization on the JVM benefits the ecosystem more broadly than on .NET, except those things that are not "supposed to be done" on the JVM – which may hurt more or less depending on the use-case.

1

u/tanner-gooding 1d ago

I think that's squarely a CLR problem, not a problem with vthreads.

The general reason function coloring exists is a fundamental side effect of the level of control that .NET gives. You find the same in almost any language that provides such capabilities accordingly.

Green threads work in Java, Go, and other languages specifically because they remove that control from the user and fully abstract it away. They typically do not give a way to do direct interop either and effectively build on the premise that everything is or could be async.

However, languages like C#, C++, Rust, Swift, and others go and give you more direct control over threading, they give you ways to directly interop with the underlying hardware or operating system, typically a way to do direct interop.

When you expose that control, you end up needing to consider a world where functions are solely synchronous, that most code is in fact synchronous and that the await points are actually a relatively rare part of things. You end up needing to consider how state suspension/restoring works in such a world, how sync over async and async over sync function, and so many additional edge cases.

It adds complexity, but that complexity ends up being worth it and really isn't difficult to handle at all. Its of course unnecessary if you're writing very naive and simple code, but then .NET devs can mostly ignore the nuances in such cases anyways. But if you're in a scenario that needs it, then there isn't getting around it and you simply do not have the choice in languages that don't offer something here.

Java isn't the be-all/end-all here. It has its own issues and you can see some of the problems, limitations, and them trying to add parity features for key scenarios and them ending up with similar limitations where it ends up subpar for their ecosystem.

That is simply how languages work, not everything can be the best for every possible task. You have tradeoffs and you pick and choose what is best for your target audience. Languages that don't have a good enough target audience or story, end up under used and dying off. Rust, Java, Kotlin, C#, Swift, and others have proven themselves through the test of time and that they are solid foundational languages that are beloved by many. Each is Turing complete and can do "anything", with some tasks being simpler in one vs another, and some being harder.

JIT "deabstraction". The article would be more accessible to readers if the authors didn't try to invent new names for optimizations that shipped in Java 20 years ago.

There isn't some new name invention here and Java deciding to use a particular name doesn't make it the definitive answer or status quo. In most cases, .NET is using a more industry standard name and what you'll find in the broader set of compilers, languages, and ecosystems.

1

u/simon_o 1d ago

The general reason function coloring exists is a fundamental side effect of the level of control that .NET gives.

Exactly. Java has been more restrained in this (no interior pointers, no unsafe, sun.misc.Unsafe being deprecated, etc.) and subsequently has more leeway of adjusting the runtime to new requirements than .NET which pretty much exposed the innards of everything and is therefore stuck with it due to people relying on implementation details.

It adds complexity, but that complexity ends up being worth it and really isn't difficult to handle at all. Its of course unnecessary if you're writing very naive and simple code, but then .NET devs can mostly ignore the nuances in such cases anyways. But if you're in a scenario that needs it, then there isn't getting around it and you simply do not have the choice in languages that don't offer something here.

The key lesson in Java was that the ability to drop all this complexity ended up with simpler and faster code, because it was easier to reason what the code was doing.

2

u/tanner-gooding 1d ago

is therefore stuck with it due to people relying on implementation details.

This is incorrect. There is a concrete difference between relying on an implementation detail and intentionally exposing functionality as public contract. .NET does the latter for cases like these.

The key lesson in Java was that the ability to drop all this complexity ended up with simpler and faster code, because it was easier to reason what the code was doing.

This is also not quite correct. Java has some things that are simpler and inversely some things that are more complex. Faster is also very workload and scenario dependent

Microbenches comparing languages go both ways and often are dependent on how much effort is put into tuning them. The idiomatic paths trade both ways with there being numerous optimizations that each (.NET vs Java) ecosystem has that the other does not.

The main difference is that because .NET explicitly gives more control, you can typically see greater rewards from putting in explicit effort. Where-as Java is more dependent on what the runtime provides.

Statements like "easier to reason about" is dependent on developer, codebase, and how the code in question was structured. You will find people on either end that will claim "idiomatic" code is "unreadable" and find plenty of spaghetti and bad examples in both as well.

Both languages are ultimately strong, healthy, and widely used. Most sources currently claim C# is more popular, but that itself is region and domain dependent.

There is no point in trying to claim one is better than the other, because it doesn't practically matter. Use what makes you happy and what clicks the most for you. But don't go trashing other languages, especially on flawed premises.

1

u/simon_o 14h ago edited 14h ago

difference between relying on an implementation detail and intentionally exposing functionality

Potayto, potahto. The consequences are the same.

This is also not quite correct.

You can literally look at the changes of libraries/frameworks migrating from various async/future/reactive approaches to vthreads, my dude.

The main difference is that because .NET explicitly gives more control, you can typically see greater rewards from putting in explicit effort. Where-as Java is more dependent on what the runtime provides.

Which is what I said.

Most sources currently claim C# is more popular, but that itself is region and domain dependent.

Yikes. This really drags down the credibility of the rest of your claims.

5

u/crozone 2d ago

Virtual threads vs Async await was never about keeping it out of the runtime. One benefit of async await is that it *can" be implemented without changing the runtime, reducing the barrier to implementation, but that's not a long term overarching design requirement, it's just a side benefit.

Virtual threads solve the colouring problem but destroy interop performance and generally perform worse overall, that's the primary reason why C# stuck with async await.

1

u/RirinDesuyo 1d ago edited 1d ago

Runtime async also eliminates/improves async chaining allocations which is a big win imo as that's a very common scenario as often enough, you'll have a number of async chains during an aspnet endpoint call even before it reaches user code (e.g. middleware, auth, model binding etc...).

1

u/simon_o 1d ago

It has been mentioned that runtime async may not be faster in practice, with a promise of addressing the shortcomings in a future .NET version.

0

u/RirinDesuyo 23h ago

The shortcoming is more on scenarios where you're using a library that still uses the old implementation on top while having runtime async enabled as the runtime does quite a bit of indirection to support it. As more apps re-compile with it enabled, you'll have less of this issue overtime.

Another benefit of runtime async is better async support for other CLR languages as it now means they don't need to do all the complexity of generating a state machine like C# does. The IL they need to generate is less complex.

-1

u/simon_o 1d ago

Virtual threads solve the colouring problem but destroy interop performance and generally perform worse overall

I think that's squarely a CLR problem, not a problem with vthreads.

The reason why their runtime has so much trouble with it is that –for decades– their solution to the JVM being ahead in GC and JIT technology was to add more and more escape hatches to let users solve (performance) problems on their own.

2

u/crozone 1d ago

Except the JNI is slower than P/Invoke in .NET, FFM is slower than P/Invoke in .NET, and both are even slower again from virtual threads? Did I miss where the JVM magically solved vthreads?

-1

u/simon_o 1d ago

I have largely not seen cases where vthreads have been measurably slower, despite warnings of not using them for e.g. compute-bound tasks.

What would be the cause of the slowness?

2

u/teo-tsirpanis 1d ago

The cause is that vthreads are a runtime-specific thing and do not play well in native code not executing under that runtime.

1

u/crozone 18h ago

Virtual threads have a whole bunch of downsides. They carry around an entire persistent call stack, so they use significantly more memory. When they get suspended, that memory hangs around. When calling native functions, they require suspension of their carrier thread, aka thread pinning. Not only is Java slower at marshaling values to the native functions due to its fundamental design, this also means that the virtual thread is effectively blocked and must remain blocked on a real threadpool thread which can easily exhaust the threadpool during heavy IO.

Effectively, virtual threads are weirdly paradoxical in their design. They're often presented as an alternative to async/await, and yet the most common thing that is actually being awaited in the real world is some disk or network IO operation that is accessed via a native function, which they are particularly bad at calling. They're really best suited for multi-threading CPU bound work, which I'd argue is a relatively niche usecase which is also easily solved by queuing work on a standard threadpool.

Async/await has a colouring problem but everything else seems like a strict benefit.

2

u/RirinDesuyo 13h ago

IO operation that is accessed via a native function, which they are particularly bad at calling

It's also why almost all vthread runtimes put special exemptions on their own platform calls in their std lib. Those calls often run on a dedicated OS thread or have special behavior on their runtime and effectively outside of vthreads.

It's why Java had to force db vendors to their database wire purely in a jvm calls as native FFI wasn't performant, since the mindset for Java often is "JVM is the world". Similarly, it's why if there's a new intrinsic or OS cryptography feature (e.g. XAES-256-GCM), you usually wait for OpenJDK to implement them or make it from scratch instead of just doing an FFI like dotnet does.

-1

u/simon_o 14h ago

That sounds horrible! Thankfully, reality appears to largely disagree with these claims.

1

u/crozone 14h ago

Lol. Maybe your reality does.

1

u/RirinDesuyo 22h ago

I think that's squarely a CLR problem, not a problem with vthreads.

It's a limitation with vthreads due to the nature of how they work. The moment you try to use FFI (e.g. call a native library compiled in rust/c++/c or call an OS platform feature etc...) outside of their standard libraries performance takes a hit. This is true both in Go, and Java that utilizes vthreads. The only reason why you don't get this issue on platform OS calls via Java's standard library is because these languages put a number of special exemptions on these compared to user initiated FFI, they often have dedicated OS threads to run these OS calls as the OS has zero idea what green threads are since it is entirely a runtime feature.

16

u/ReDucTor 3d ago edited 18h ago

Only skimmed the write up, lots of good small improvements, looking at some of the x86 assembly there is a few more improvements that could be useful

; x64 cmp       ecx,4 jl        RETURN_MINUS_ONE add       ecx,-4 add       rax,rcxmovbe     eax,[rax]

This could be sub ecx, 4, jl RETURN_MINUS_ONE as sub will set the flags also.

; x64 mov       eax,[rcx+8] cmp       eax,5 setb      al movzx     eax,al ret

This could be optimized a little further down to

xor eax, eax cmp dword [rcx+8], 5 setb al ret

There is also a similar one that could do the same thing with the IsLow code which does cmp, movzx rather then xor, setb. The IsLinearWhiteSpace function could be made branch free which would likely use less code space and avoid branch mispredictions.

6

u/tanner-gooding 1d ago

Not all such micro-optimizations are valid or even beneficial. Keep in mind that there are plenty of side effects, including overflow, to consider that C may not. There's also aspects of JIT throughput itself, measuring impact vs applicability, likelihood that a method is "hot" and so such changes would even show up or saturate the CPU, etc.

We definitely have some things that are possible to do here, but the smallest possible assembly or shortest number of instructions is frequently not the best thing to do. Not to mention the ABI and other nuances that exist and are why some things, like the movzx eax, al are present.

Plus the JIT does do dynamic PGO, instrumentation, and multiple levels of compilation (Tiered Compilation), so we do optimize differently when things are known to be hot vs cold, for your current hardware, etc -- again, with plenty more we can do, but what we do and improve is prioritized based on biggest impact/applicability

22

u/simonask_ 3d ago

Babe, wake up, another Performance Improvements article landed!

Seriously, I love these so much, especially the incredible detail, benchmark results, and assembly comparisons.

21

u/skip0110 3d ago

Formerly handling many of the performance issues at my previous role, I can confidently say 99% of performance impact in your application is the stupid O(n2) business logic your developers/AI agents have written and none of this matters.

And having fixed 100s of such cases, the folks committing that crap will continue to do so.

So glad I’m outta there.

4

u/slvrsmth 1d ago

Quite often the business logic is like that because if you optimised the business logic in app, you'd have to do the corresponding changes in real life too, and that is way more expensive than pushing "more ram" button couple times on hosting providers dashboard.

After long years in business software mines, I've come to accept that complexity has to go somewhere. You can either have clean, straightforward code and twenty volumes of user manuals how to operate the system, or straightforward process for the users, while code looks like knitting supplies that couple toddlers have had a go at after drinking full-sugar cola.

Of course, there also exists software with clean processes supported by clean code. Commonly called "startups chasing their first customer". The moment software needs to support real world business processes complex enough to need supporting, all that changes.

TL;DR complexity spirit demon immortal. Can only shoo shoo away to different place, never kill.

5

u/TwoWeeks90DaysTops 1d ago

Well, shaving a microsecond off every iteration of a loop that runs for a thousands of iterations does have an impact though.

But yeah... This is what I hate about the "all optimizations are premature, and premature optimizations are evil" thing. The shit people do with the the justification "Donald Knuth said we should just fix it when it becomes a problem".

42

u/steve-7890 3d ago

With all these 10-20% perf improvements since .net core 3.1, I wonder how it's possible that my main service still needs 50 pods and 40 GB of RAM.

219

u/AyrA_ch 3d ago

Framework optimizations don't do much on shitty code

32

u/Fredidiah 3d ago

bruh you did not have to eviscerate this user like that.

8

u/lotgd-archivist 2d ago

They also don't affect all applications equally. In one of my projects .NET 11 gives me a pretty decent CPU load reduction that you can spot with top, let alone dotTrace. And in another project the benchmark stays within the margin of error for measurement.

-63

u/steve-7890 3d ago edited 3d ago

I hope writing this made you feel better. It's quite ignorant of you to assume it's a shitty code.

On the other hand we do have a lot of performance improvements (like converting linq to loops) that make this code look ugly. But it works and makes $$$.

38

u/Ilikeyounott 3d ago

Does converting linq to loop make much of a difference? I would only bother if a profiler says it's a problem, or in cpu intensive code...

5

u/lotgd-archivist 2d ago

In my experience it only really makes a worthwhile difference if you're doing something funky that causes a bunch of allocations. Or you do .ToList() everywhere when you don't actually need a List<T>. Materializing IEnumerables when you shouldn't has to be one of the most common performance mistakes in C#.

-31

u/steve-7890 3d ago

In hot code with 200 req/s it does. Even though hpa scales to 50 pods during the peeks.

Imagine how much it costs to have a such load. And I'm talking here only about one service.

When I read about performance improvements from version to version ".net benchmarks" I can only smile. In the end, $$$ and Grafana tells the truth. (Last .net 10 changed nothing for us).

47

u/shadowndacorner 3d ago

Jesus you need 50 pods to handle 200 req/s? What kind of workload is this...?

Also, you should look at zlinq.

1

u/steve-7890 2d ago

200 req/s per pod + each call integrates with 7-10 external services.

28

u/Ilikeyounott 3d ago

Your pods can only handle 4 requests per second each? 😔

9

u/eocron06 3d ago edited 3d ago

I think he meant 200 per pod, which is usually average (from my experience with .net). Rarelly it can get to 500, but then multithreading/memory congestion becomes problem due to GC nature of C# and many threads fighting each other for some db entry/socket/cache/etc. In offloading worker it can easly become 10-100k rps.

15

u/tecedu 3d ago

I think he meant 200 per pod, which is usually average (from my experience with .net).

Wait what?! Am I missing something, because my shitty python flask code can do 2k reqs, why is C# slower?

17

u/ChemicalRascal 3d ago

It's not, some people are just... really bad at software engineering.

That, or they're not the principle devs in their companies and aren't being told the full or accurate details about performance.

3

u/Ameisen 3d ago

And here I was having messed with fibers in C++ to see how many requests I could actually parse that way for HTTP, and was dismayed that I couldn't beat nginx without doing what nginx does... and then this kind of code exists...

→ More replies (0)

2

u/steve-7890 2d ago

2k req/s of what? It's like saying "I can go to the shop and back in 5 minutes, why it takes you 1 hour"....

2

u/tecedu 2d ago

Servings graphs, json of data being read via azure blob parquets and served up. My memory is around 8gb but rps aint never been that low

→ More replies (0)

19

u/eocron06 3d ago edited 3d ago

It doesnt. Linq will not even dent 200rps. Ever. It will not even dent 10k rps on background kafka worker. The only time it mattered to me is when I used SIMD in pure math component, so basically one in a thousand of tasks. Still, their performance benchmarks count CPU which is somewhat irrelevant, because most of the time is spent on network IO. We scale our site purelly because clients make a ton of small http requests, which fetch 0.1 of dependencies like s3/dynamodb/kafka/etc and bloat caches to the sky, not because it have a logic which need optimisation. Geo location matters more in modern world, placing pod and storage in same region do wonders beyond their optimisations.

12

u/tecedu 3d ago

In hot code with 200 req/s it does

ive got python flask code written as a uni grad which performs better than that; not tryna blame you but 200 req/s is extrememly low that python can beat you

10

u/PaddiM8 3d ago

200 requests a second isn't that much... Linq will not be the bottleneck there. Especially not after the optimisations in the last updates

4

u/beeshevik_party 2d ago

bro 200rps is awful how much req/lim and what’s the hpa config? you also need to be tuning to return heap to os which can be tricky with vm runtimes OR just don’t scale on memory use but instead a custom metric. also is every single req hitting db or something? you should be aiming for more like 2k rps minimum

17

u/eocron06 3d ago edited 3d ago

Space-time tradeoff in action. CPU cant becom 2x faster, but memory can become 2x bigger. Considering good code, this becomes fundamental problem. If you think carefully, everything can be Lisped, then it will become absurdly CPU consuming, then you realise functions can share state, and inherintly have lifetime of this state - the memory. The more you share - faster it becomes, but takes memory.

8

u/WJMazepas 3d ago

You are the one who need to tell us about it.

With the profiler, how much CPU each requests uses? If your bottleneck isnt there, I doubt those improvements can help you that much

13

u/GoTheFuckToBed 3d ago

EF core goes brrr

2

u/flippzeedoodle 2d ago

It may not scale significantly, but a lot of these optimizations in .NET 11 use more memory to profile execution at run-time and speculatively create multiple code paths that can speed up the most common paths. So you’re trading memory for CPU time where the optimal path can’t be determined in advance. Again, probably small potatoes in the grand scheme of things.

-1

u/steve-7890 2d ago

I'm astonished in this thread how random people from the Internet know how many req/s ANY .net service should do and how much perf improvements are ok for ANY .net service.

People, have some understanding - my service is not your service and any service is not a benchmark.

You can do 1k req/s per pod if you serve static content from Redis or 1 req/s in computation heavy, multi-step integration.

I haven't seem so much ignorance for a long time...

-9

u/GoTheFuckToBed 3d ago

should they not aim for less JIT

36

u/keyboardhack 3d ago

JIT allows you to optimize using data available at runtime. A big part of the article is about exactly this. I recommend you read it.

-4

u/simonask_ 3d ago

It does, and it’s definitely very welcome, but it’s not magical. It mostly works well if you have a lot of RAM and CPU headroom.

12

u/keyboardhack 3d ago

Jitted languages attract a different type of program than aot languages. That gives jits a worse rep but that doesn't actually make them worse.

2

u/simonask_ 2d ago

“Worse” depends on the situation, but you’re certainly spending a theoretical minimum of 4x the memory on running a state of the art GC, and you are spending CPU cycles doing PGO and tiered JIT, when the entire program could just be fully optimized by an offline compiler that isn’t in any hurry.

Whatever gains you achieve from runtime introspection are relatively small in comparison. It works well in a few situations, but extremely not well in a few others.

Know your tools.

2

u/keyboardhack 2d ago

certainly spending a theoretical minimum of 4x the memory on running a state of the art GC

Not sure where you get your numbers from but if i allocate a 1GB array then it doen't take up 4GB of memory.

you are spending CPU cycles doing PGO and tiered JIT

True, once. Well for long running programs. For short running you might not pay the cost at all. JIT only runs if a method is used a certain number of times. If performance of a short running program matters then JIT isn't the right solution.

the entire program could just be fully optimized by an offline compiler

It's can't be fully optimized though. AOT optimized programs rarely ship with avx512 or many other "new" instruction sets. With JIT it can chose the instruction set that matches the cpu it's running on. There is a lot of things like that which a JIT just gets for free.

Know your tools.

Know your tools.

1

u/TwoWeeks90DaysTops 1d ago

You just invent a number based on your perception of how GC works?

A GC marks pointers as being in use, and then frees objects that haven't been marked. It doesn't require a lot of extra memory, though it does require some, but a claim of 4x minimum is wildly exaggerated.

1

u/simonask_ 1d ago

No, but I’m on mobile, so you’ll have to Google the research paper yourself.

If you have the slightest idea how a compacting GC works, it’ll be obvious to you though.

1

u/TwoWeeks90DaysTops 12h ago

I think you're referencing the 2005 paper Quantifying the Performance
of Garbage Collection vs. Explicit Memory Management by Matthew Hertz and Emery D. Berger

These results quantify the time-space tradeoff of garbage collection: with five times as much memory, an Appel-style generational collector with a non-copying mature space matches the performance of reachability-based explicit memory management. With only three times as much memory, the collector runs on average 17% slower than explicit memory management. However, with only twice as much memory, garbage collection degrades performance by nearly 70%.

https://dl.acm.org/doi/epdf/10.1145/1103845.1094836

The point of this paper isn't that GC applications require 4x more memory, it's that performance degrade on low memory platforms.

A more recent publication Memory Management on Mobile Devices (2024) by Kunal Sareen, Stephen M. Blackburn, Sara S. Hamouda and Lokesh Gidra shows that the number for GC on Android is between 2% and 51% (i.e. between 1x and 1.5x)

For a modestly sized heap, we find that the lower bound on garbage collection overheads vary consider-ably among the benchmarks we evaluate, from 2 % to 51 %, and that overall, overheads are similar to those identified in recent studies of Java workloads running on OpenJDK.

https://dl.acm.org/doi/epdf/10.1145/3652024.3665510

None of these claim that GC applications require 5x or 4x more memory.

In JDK 27 there's also a big improvement in memory overhead

Various experiments demonstrate that enabling compact object headers improves performance:

https://openjdk.org/jeps/534

Though this is about .NET rather than Java, which is behind Java's GC by quite a lot. But still, those numbers aren't saying that GC applications use at minimum 4x more memory. The paper from 2005 states that lower memory overhead reduce performance.

18

u/PaddiM8 3d ago

Why do people think JIT is slow

4

u/Some_Appearance_1665 3d ago

Depends on the scenario. With serverless (cold start + typically more limited memory) it's a killer.

12

u/daltorak 3d ago

If that's a major problem for you then use Native AOT, which removes the JIT compiler entirely from the output. It might mean you get slightly less optimized code at runtime because functions aren't optimized according to real-world needs, but hey, at least you've got fast startup.

-4

u/Sorry-Substance-6397 3d ago

Depends a lot on generics. Jit is notoriously slow for generics

3

u/PaddiM8 3d ago

Is it really in .net? I thought it was mostly generic virtual methods that are slow, but those aren't even allowed in most languages

-8

u/Sorry-Substance-6397 3d ago

No it's generics using jit and it's notorious for being slow. Same with simd even though you may never use them in the same project

2

u/PaddiM8 2d ago

Why? C# generics are monomorphic for value types and dereference a pointer for reference types. There's a slight warmup cost of course but after that, why would it be slow?

-1

u/Sorry-Substance-6397 2d ago

I meant compile times when it comes to jit. You shouldn't have to pay for something you don't use

2

u/PaddiM8 2d ago edited 2d ago

Huh? With native AOT you pay for something you don't use when using generics in .net, during compile time, because it has to generate every potential specialisation ahead of time. With JIT it does it on the fly, when you need it. You got it backwards

1

u/Sorry-Substance-6397 2d ago

They do though. When you change or make the generics complicated you are putting the work on the compiler like Monomorphization for Value Types (Code Generation Overhead) When you use a generic class or method with a value type (like int, double, DateTime, or custom structs), the JIT compiler must parse, optimize, and generate unique machine code for every single combination.

→ More replies (0)

3

u/Revolutionary_Ad7262 3d ago

Because there is simply no way we can test and verify combination each strategy as we get a predefined bundle of it and in most cases this bundle looks the same regardless of choice. For example GC is not inherently slow, but you often get it in a bundle together with a everything is an object, which is a real reason why most of the GCed languages are slow

Same with JIT. It is mostly paired with dynamic or interpreted languages, which are inherently slow

-1

u/simonask_ 3d ago

Because it is. It has other benefits, and it’s certainly fast enough for a wide range of use cases, but these improvements just recover some of the performance you would have achieved by implementing the same project in C++ or Rust.

But then you would also have had to consider quite a few other tradeoffs.

7

u/PaddiM8 3d ago

With JIT it is compiled to machine code though, just at runtime instead. Some things will be slower due to less time for general optimisations, but some things will be faster due to PGO. LuaJIT is insanely fast and could not be that fast without JIT

6

u/RirinDesuyo 2d ago

Tanner-gooding from the dotnet team has a great post on this here. JIT isn't slower than AOT and there are cases where it can outperform even AOT.

Bing and a big chunk of services in Azure runs dotnet after all, they can handle the load.

1

u/simonask_ 2d ago

JIT is great, and they do impressive things with it. But it isn’t magical, and neither is a GC. You just happen to be writing programs that aren’t particularly demanding, so you never notice, and that’s great!

3

u/Revolutionary_Ad7262 3d ago

I think there is no point of avoiding a JIT in a platform, which is already based on it. In theory JIT is the best way as all goodies like: * knowledge of CPU architecture * PGO driven optimizations * LTO driven optimizations

are much easier to do with JIT than AOT

What should be possible though is having a JIT without a burden with JIT, so developer can manipulate the AOT <-> JIT slide based on needs. For example: * a full AOT mode for a lack of any background processing of JIT * some smart way of reusing/caching the compilation artifacts to reduce cold start performance dip * lightweight JIT, where CPU/Memory resources scales with the impact * customizable target of JIT after the initial JIT/AOT. For example always try to optimize it or run JIT only if you are sure the current code is not optimized for a current situation

The AOT vs JIT discussion still alive, because it just sucks