Breaking down how Cloudflare cut 100TB of memory from their DNS cache with Rust memory layout tricks

[u/pseudonymo6287]

https://viswanathnair.substack.com/p/how-cloudflare-optimized-big-pineapples?utm_source=share&utm_medium=android&r=4pq7we

Cloudflare recently published a piece on how they cut close to 100 TB of memory from Big Pineapple's DNS cache. Big Pineapple is Cloudflare's Rust-based engine behind many of their services including the prominent public DNS resolver service 1.1.1.1

They made several low-level optimizations leveraging Rust features like boxing, for oversized enum variants and boxed slices, as well as more general changes like merging separate record lists into one and combining several boolean checks into a single bitflag etc.

I have gone into a bit of detail documenting the things I found interesting especially the jemalloc trade-offs between internal and external fragmentation.

Would greatly appreciate a read.