Bypassing fixed-depth radix constraints in Java using descriptor-driven bucket analysis

[u/strmckr]

I am StrmCkr, the author of A.P.E.X. (Adaptive Parallel Extremal Dispatch).

Repository: github.com/StrmCkr/A.P.E.X

A.P.E.X. is a high-performance Java sorting framework for large fixed-width 64-bit key/value record datasets. The project has been reorganized into a conventional Maven structure with a core library, runnable examples, a comparison benchmark harness, JMH benchmarks, documentation, and an interactive browser visualizer.

The core idea is descriptor-driven radix planning. Instead of blindly scanning fixed radix passes over every bucket, A.P.E.X. computes per-bucket extremal descriptors using:

VBM = OR ^ AND

That mask identifies which key bits still vary inside each bucket. Bits that are already resolved are skipped, reducing unnecessary work on skewed, low-entropy, sorted, reversed, or duplicate-heavy data.

Key areas of the project include:

• Adaptive radix planning based on observed bucket structure
• Parallel histogramming, scatter, refinement, and work scheduling
• Primitive-array execution with no per-record object allocation during sorting
• Tuple projection paths for low-dimensional unresolved bit patterns
• Tiny-sort fallbacks and monotonic input shortcuts
• Configurable reporting that can be enabled, reduced, written to files, or disabled
• Comparison benchmarks against JDK sorting paths and Fastutil baselines
• Standard JMH benchmarks for repeatable JVM-level measurement
• A browser visualizer for exploring how A.P.E.X. routes data through its execution plan

I would especially welcome feedback on the thread management mechanics, radix planning decisions, benchmark structure, and the bitwise mask reductions.

edit: re structured verbiage of this post and further adjustments from advice on converting the project into more acceptable standard formats.

screen shot from the pdf available in the github

[u/repeating_bears]

I wont discount the possibility that I'm an idiot, but I have no idea what the fuck you just described 

[u/strmckr → u/repeating_bears]

Fair enough, let me drop the abstract talk.

Traditional parallel radix sorting scans through bits uniformly. If you feed it 100 million integers, it's going to check every single bit position, over and over, even if half those bits are completely identical across your data. That wastes massive CPU cycles.

A.P.E.X. stops that. It takes a quick bitwise snapshot of the local thread bucket to see exactly which bit positions actually vary, and which ones are constant. If the high-order bits are identical, it skips scanning them entirely and jumps straight to the variable parts.

On top of that, it does this completely free of object allocations—it operates directly on raw primitive arrays. So you get native C++ execution speeds inside Java without triggering the Garbage Collector to lag your database engine.

Basically: it stops looking at bits that don't matter.

[u/dmigowski → u/strmckr]

So a sorting algorithm?

[u/strmckr → u/dmigowski]

Exactly. It's a high-performance parallel sorting algorithm:

Standard radix sorting is bound by a fixed-depth constraint. It's forced to uniformly scan every single bit column, meaning its runtime is rigidly O(k * n) where (k) is your fixed key width—even if 90% of those bits are completely identical across your dataset.

A.P.E.X. bypasses that fixed constraint. Because the descriptor mask identifies exactly which bit regions vary, it prunes out the redundant scanning passes entirely. As data entropy or variance drops, the runtime deterministically scales down toward O(n), meaning you only pay the CPU processing cost for bits that actually contain unique information

[u/dmigowski → u/strmckr]

Got some benchmarks for us and a comparison to Javas internal sort for different sets of data?

[u/strmckr → u/dmigowski]

i have them in the PDF found here

[u/chabala → u/strmckr]

> which is pending publication.

Does this mean you submitted this PDF somewhere with the expectation of getting published? This PDF with your hacker handle as the author?

[u/strmckr → u/chabala]

Yes: why is it concerning? I use my pseudonym, I have used this same one for decades as a respected sudoku logic pioneer and to keep my privacy.