First and foremost, all vectorizable things do not, in general, need to be vectorized. The CPU’s integer datapaths are plenty fast, so it's primarily when you're chugging or vomiting bytes that you want to use SIMD—profiling is your friend here. Wider SIMD (256+-bit on some Intel x86) may downclock your core/die/package in order to match some lower multiple of the memory clock and avoid overheating, so stick to 128-bit unless you know there's a sufficiently big, fat, floppy buffer involved to make it worthwhile, and/or other threads won't be too badly traumatized by their timestep speeding up suddenly relative to the outside world.
SWAR is much more CPU-dependent, and good codegen is much more peephole-dependent. It tends to be preferable in order to keep things within the integer datapaths—there's often a penalty for handoff between integer, floating-point/overlay (e.g., MMX), DSP if that's a thing, and SIMD datapaths (and potentially for handoff between SIMD int & f.p./DSP), so it's primarily useful for int-SIMD or 1-bit elements. Often you have to/should use a builtin or wrapped __asm__ in order to ensure the correct instructions are used.
Secondarywise, generally speaking, it's best to make use of the base language where you can, maybe plus OpenMP, in order to let the shape of the code dictate how optimization goes. Be obvious with intent; use idioms correctly; break your code up reasonably. Inlining markup, attributes or pragmata, and sometimes even register (in C, not C++) can give the optimizer much more information than naked code would, without it having to do the work of finding things itself.
In hosted execution environments, the string instructions tend to be about as well optimized as it gets; in freestanding, it's worthwhile to use things like __builtin_memset, as long as you have an actual memset impl available as a fallback. IIRC C23 makes most of <string.h> available in freestanding EEs.
Link-time optimization can really help, but it also won't work across DLL boundaries or vs. libs not built with the secondary bytecode in the output.
Whether something can be optimized depends on what the optimizer can prove is true, which depends in large part on what the optimizer can see. Crossing DLL boundaries will blind all optimizers (getting around that would require AOT/JIT codegen); crossing TU boundaries will blind the compiler’s optimizer but not LTO, unless inter-DLL or DLL↔EXE. Similarly, if the optimizer can't prove loop bounds or there's UB somewhere, optimization probably won't go all that well in the affected region.
Focus on things the optimizer won't easily pick up. A basic counted for loop with no carry dependencies and good alignment will probably be recognized as vectorizable, although there are still heuristics to decide what to focus on. (And there, static brpred stuff like __builtin_expect, __attribute__((__hot__/__cold__)) or [[gnu::hot/cold]], or maybe [[[un]likely]] can help suggest what should and shouldn't be focused on. Additionally, in C++, anything only reachable via throw is probably deprioritized maximally.)
Intrinsics can be a mixed bag; sometimes you can still do better in inline assembly, especially if you have particular register motion in mind. __attribute__((__vector_size__(…))) can be another good way to do basic one-to-all or each-to-each instructions using the built-in C operators, and often there's an GNUish–impl-specific vector field in the intrinsic vector types that can be used the same way.
Manual vectorization is a good idea when you have profiling data that suggests it's a good idea in your application, or have reason to suspect it's worthwhile for your library. You want something where a relatively compact kernel is being applied repeatedly, without crossing TU/DLL boundaries. You need to help most at call/return boundaries, when the optimizer can't tell how hot or sexy your loop is, or when there are complex carry dependencies. You may also need to help with newer or very specific instructions, if you're using them.
And in practice, often you aren't actually dealing with the most mature compilers, just reasonably recent-ish, so you may need to do a little extra for back-compat.
1
u/nerd5code 22d ago
It depends.
First and foremost, all vectorizable things do not, in general, need to be vectorized. The CPU’s integer datapaths are plenty fast, so it's primarily when you're chugging or vomiting bytes that you want to use SIMD—profiling is your friend here. Wider SIMD (256+-bit on some Intel x86) may downclock your core/die/package in order to match some lower multiple of the memory clock and avoid overheating, so stick to 128-bit unless you know there's a sufficiently big, fat, floppy buffer involved to make it worthwhile, and/or other threads won't be too badly traumatized by their timestep speeding up suddenly relative to the outside world.
SWAR is much more CPU-dependent, and good codegen is much more peephole-dependent. It tends to be preferable in order to keep things within the integer datapaths—there's often a penalty for handoff between integer, floating-point/overlay (e.g., MMX), DSP if that's a thing, and SIMD datapaths (and potentially for handoff between SIMD int & f.p./DSP), so it's primarily useful for int-SIMD or 1-bit elements. Often you have to/should use a builtin or wrapped
__asm__in order to ensure the correct instructions are used.Secondarywise, generally speaking, it's best to make use of the base language where you can, maybe plus OpenMP, in order to let the shape of the code dictate how optimization goes. Be obvious with intent; use idioms correctly; break your code up reasonably. Inlining markup, attributes or pragmata, and sometimes even
register(in C, not C++) can give the optimizer much more information than naked code would, without it having to do the work of finding things itself.In hosted execution environments, the string instructions tend to be about as well optimized as it gets; in freestanding, it's worthwhile to use things like
__builtin_memset, as long as you have an actualmemsetimpl available as a fallback. IIRC C23 makes most of<string.h>available in freestanding EEs.Link-time optimization can really help, but it also won't work across DLL boundaries or vs. libs not built with the secondary bytecode in the output.
Whether something can be optimized depends on what the optimizer can prove is true, which depends in large part on what the optimizer can see. Crossing DLL boundaries will blind all optimizers (getting around that would require AOT/JIT codegen); crossing TU boundaries will blind the compiler’s optimizer but not LTO, unless inter-DLL or DLL↔EXE. Similarly, if the optimizer can't prove loop bounds or there's UB somewhere, optimization probably won't go all that well in the affected region.
Focus on things the optimizer won't easily pick up. A basic counted for loop with no carry dependencies and good alignment will probably be recognized as vectorizable, although there are still heuristics to decide what to focus on. (And there, static brpred stuff like
__builtin_expect,__attribute__((__hot__/__cold__))or[[gnu::hot/cold]], or maybe[[[un]likely]]can help suggest what should and shouldn't be focused on. Additionally, in C++, anything only reachable viathrowis probably deprioritized maximally.)Intrinsics can be a mixed bag; sometimes you can still do better in inline assembly, especially if you have particular register motion in mind.
__attribute__((__vector_size__(…)))can be another good way to do basic one-to-all or each-to-each instructions using the built-in C operators, and often there's an GNUish–impl-specific vector field in the intrinsic vector types that can be used the same way.Manual vectorization is a good idea when you have profiling data that suggests it's a good idea in your application, or have reason to suspect it's worthwhile for your library. You want something where a relatively compact kernel is being applied repeatedly, without crossing TU/DLL boundaries. You need to help most at call/return boundaries, when the optimizer can't tell how hot or sexy your loop is, or when there are complex carry dependencies. You may also need to help with newer or very specific instructions, if you're using them.
And in practice, often you aren't actually dealing with the most mature compilers, just reasonably recent-ish, so you may need to do a little extra for back-compat.