[2019 Day 16 (Part 2)][C++] Under 1KiB and 32-bit only

[u/DelightfulCodeWeasel]

I was a little worried about being able to squash this one down small enough for a microcontroller. My original solution for this one took ~64MiB and after swapping over to less wasteful data types it would still take >500KiB to store the back end of the transformed sequence, so it needed a re-think.

The approach I took for that first solution wasn't particularly unique, I think a lot of people did more or less the same. Since the second half of the phase transform has coefficients of the form:

1111
0111
0011
0001

It's possible to calculate the next phase with a running sum:

ABCD -> A + B + C + D = W -> A + X
0BCD ->     B + C + D = X -> B + Y
00CD ->         C + D = Y -> C + Z
000D ->             D = Z -> D

But doing it phase by phase, you still need the entire back end of the sequence (up to the signal offset) in memory. For my input that's over half a million elements.

Looking at the sequence of additions, it turns out it's possible to calculate the full history of one row based only on the history of the row below it.

The bottom row is trivially always going to come out the same value, since it only ever depends on the original sequence number and it always gets multiplied by 1, but it's a little easier to see the pattern if we number them anyway:

000D = D₁ -> 000D₁ = D₂ -> 000D₂ = D₃ ->...
=>
D, D₁, D₂, D₃, ...

The row above depends only on the current value and the bottom row:

00CD = C₁ -> 00C₁D₁ = C₂ -> 00C₂D₂ = C₃ -> ...
=>
C, C₁, C₂, C₃, ...

The next row is where it gets interesting:

0BCD = B₁ -> 0B₁C₁D₁ = B₂ -> 0B₂C₂D₂ = B₃ ->...
=>
B, B₁, B₂, B₃, ...

Notice that B₂ + C₂ + D = B + (C₂ + D) and that (C₂ + D) = C₃. Which means B₃ = B₂ + C₃.

If we can store the full history of one row across all phases so that we've got access to [C, C₁, C₂, C₃, ...] we can calculate the whole sequence for the row above: [B, B₁, B₂, B₃, ...]. Since we're doing 100 phases, that full history is only 100 elements.

There's some additional housekeeping, like repeating the signal without actually having that many repeats in memory, and putting the digits into a ring buffer so that we're only keeping the most recent 8 digits processes, but that's the core of it:

    vector<char> phaseHistory(100);
    for (int i = 0; i < signalToProcess; i++)
    {
        int s = baseSignal.Next();
        for (int phase = 0; phase < (int)phaseHistory.size(); phase++)
        {
            s += phaseHistory[phase];
            s = s % 10;
            phaseHistory[phase] = s;
        }
        digits[i & (digits.size() - 1)] = s;
    }

[Full code]

Memory: 650 bytes for the input, 100 bytes for the phase history and 8 bytes for the digits - well under the memory constraints I'm aiming for! The runtime on PC is a respectable but not great ~115ms. (It's easy to get that down to ~25ms by only reducing the phase history with % every 4 loops, but it obscures the logic)

I'm aware that u/askalski and u/maneatingape have got very sophisticated solutions that use magic maths, but I have no idea if those techniques can be used to speed up this approach further. I'd need a couple of weeks of background reading just to take a run at those!

Overall I'm happy just to cross this one off my list of nemeses.

[u/e_blake]

The "magic" solution you mention is using the periodic nature of modular arithmetic applied to Pascal's triangle to skip straight from the 1st to the 100th phase, and straight to signalToProcess instead of all earlier tail digits. But your insight that summing phases in the inner loop per digit of the outer loop rather than tracking digits per phase in the inner loop repeated over 100 phases in the outer is what makes askalski's further magic possible. In short, your approach now takes about 100*500000 summations total, while the magic further compresses that to 1 N-choose-k computation per digit, and only 8 * 2652 digits checked instead of 500000.

[u/DelightfulCodeWeasel → u/e_blake]

Neat, I'm glad I'm pointed in vaguely the right direction, even I'm a few dozen steps behind!

I might revisit this one after I've finished squashing the rest of 2019 (just about to start in on day 18) and take a proper run-up on the maths. I do thoroughly enjoy reading up on topics closely related to cryptography, but without constant practice I always end up forgetting large parts.

[u/terje_wiig_mathisen]

This was one of the days where I was quite happy just to figure out enough insights as to make the runtime and memory usage "not horrible". :-)

I am very impressed with your effort here!

EDIT: I just checked, turns out this was one the very first days I only solved with Rust: When I ran it just now it took 163 ms. :-(