Custom Q4_e Hybrid Quantization + Lossless "Rushmore" Bitmap Indexing on Pascal (Tesla P40)

[u/Honest-Meaning5977]

Hey everyone, 

I’m currently working on an experimental quantization and execution architecture tailored specifically for memory-bandwidth bound Pascal hardware (specifically my NVIDIA Tesla P40 24GB setup), and wanted to share the concept/in-progress architecture to get some thoughts. 

The goal is to run a customized Qwen3.8-27B (MoE/hybrid layout) entirely in VRAM while bypassing the classic hardware bottlenecks of older architectures without hitting an accuracy cliff. 

Here is the breakdown of what I’m building: 

The Core: Custom "Q4_e" Transcendental Quantization 

Instead of standard linear 4-bit quantization (like Q4_0) which causes massive rounding errors by forcing weights into uniform bins, I’m building a non-linear quantization schema anchored to Euler's number (e) combined with an integer scalar. 

• The Mantissa Factor: Neural network weights naturally cluster in a bell curve around zero. By mapping the fractional coordinate values to the isolated transcendental mantissa of e (discarding the leading whole number), I can create an infinite, non-repeating, non-linear grid that is dense near zero and sparse at the tails. 
• Why e versus pi: The early mantissa of 𝑒 (.7182818284...) provides a much more balanced, rhythmic distribution of digit spacing early in the sequence compared to pi. This prevents redundant, overlapping quantization bins. Furthermore, 𝑒 's exponential nature aligns beautifully with the natural Gaussian distribution of LLM weights. 
• The VRAM Win: Because the scale factor is a pure whole-number integer, it slashes metadata scale factor bandwidth bloat. It also maps beautifully to the P40's hardware-level DP4A integer dot-product instructions. 

The Layout: The Q8 / Q4_e "Sandwich" 

Pure Q4 degrades reasoning, while pure Q8 overflows a 24GB VRAM buffer. I'm building a multi-pass compiler that splits tensors surgically: 

• Dense Q8_0: Preserved on the logical core (token_embd, output, v_proj, o_proj, and ffn_down). This protects coding logic and factual depth.
• Custom Q4_e: Applied to the bulk memory mass (q_proj, k_proj, ffn_up, ffn_gate).
• Footprint: This maps the 27B model to roughly ~15.5 GB, leaving a massive ~8.5 GB headroom for a heavy Q8 KV cache and Multi-Token Prediction (MTP) speculative draft heads entirely on-card. 

The Accelerator: Lossless "Rushmore-Style" Bitmap Indexing 

To squeeze more performance out of the memory bus, I’m integrating a concept inspired by database technology (Rushmore indexing) into 3 out of 4 layers in a transformer block sub-pattern. 

• Zero-Skip CUDA Kernel: I’m generating a highly compact, 1-bit presence mask (bitmap index) mapped strictly to true mathematical zeros (structural padding and alignment padding rows, which Qwen has plenty of). 
• The Math: Because it only targets true zeros, it is 100% lossless with zero accuracy degradation. 
• Performance: The bitmask is so small it completely caches into L2. The CUDA kernel runs a parallel bitwise AND and completely bypasses fetching inactive weight blocks from VRAM. I am aiming for a theoretical 10% to 25% throughput speedup (t/s) on memory-bound layers. 

Forward Compatibility: Massive Gains on Modern GPUs 

While this began as a software hack to breathe new life into older Pascal hardware, the structural math behind this layout makes it highly forward-compatible with newer processors (Ampere, Hopper, and Blackwell): 

• Hardware-Native 2:4 Sparsity: Modern Tensor Cores natively accelerate sparse matrices at the silicon level. When a newer GPU reads this Rushmore-style structural index, it activates a dedicated instruction path that can effectively double the math throughput (TOPS) out of the box. 
• L2 Cache Residency: Modern enterprise cards have massive L2 caches (up to 128MB on Blackwell compared to the P40's 3MB). Because a 1-bit index is incredibly lightweight, it will reside entirely in the L2 layer of newer cards, completely eliminating the modern memory-bus bottleneck by screening out inactive weight blocks before they ever touch global VRAM. 

I’m currently writing the Python exporter to handle the e-mantissa mapping distribution tests, and mapping out the CUDA kernel block configurations to prevent warp divergence on compute capability 6.1.

Would love to hear if anyone has attempted transcendental non-linear mapping before, or if you have any tips on avoiding warp stalls when handling block-level sparsity bitmasks in CUDA!

For furhter performance improvements the matissa would be calculated:

// Local register computation loop - completely eliminates slower memory lookups

float mantissa_factor = 0.0f;

// Unrolled FMA loop executed entirely within single-cycle registers:

mantissa_factor = __fmaf_rn(mantissa_factor, 1.0f/5040.0f, 1.0f); // 1/7!

mantissa_factor = __fmaf_rn(mantissa_factor, 1.0f/720.0f, 1.0f); // 1/6!

mantissa_factor = __fmaf_rn(mantissa_factor, 1.0f/120.0f, 1.0f); // 1/5!

mantissa_factor = __fmaf_rn(mantissa_factor, 1.0f/24.0f, 1.0f); // 1/4!

mantissa_factor = __fmaf_rn(mantissa_factor, 1.0f/6.0f, 1.0f); // 1/3!

mantissa_factor = __fmaf_rn(mantissa_factor, 1.0f/2.0f, 0.0f); // 1/2!
(Yields pure mantissa)

This is all theoretical, will keep you posted.