During work, it can feel like a struggle to move in any direction while lost. Senses of orientation are weak and vague, while senses of difficulty, questioning, and pointlessness are overwhelming. Merely persisting takes significant effort. There is an unpleasantness about it.

Then, on arriving at some milestone where something works that didn’t work before, my experience, and importantly, my memory of the experience, rearranges itself into, “I tinkered with this until I stumbled into a fledgling understanding.”

After that, in the refinement phases, I tend to look back on the struggles as brief and needless, but that’s probably not accurate.

From zero to one

There was no evidence that I was even moving in the right direction until I identified 3 Costas arrays the proper distance apart in a data frame pulled from a recording.

Then, when I did verify that, I was instantly somewhere.

Not done, obviously. Not half-done or any known fraction of done, but I wasn’t at zero, and I wasn’t lost.

The friends we made along the way

These highlights are my notes from each knot I struggled with enough to notice when they untangled during this phase:

Windowing bug : My FFT loop only shifted by 1 sample per iteration instead of one full 160ms window, so all FFTs were nearly identical
FFT symmetry : I was searching all 1920 bins including the redundant mirror half because I didn’t understand that FFT reflects the symmetry in a bandwidth.
Silence detection fundamentally broken : My detour into averaging energy across all 1920 bins makes a -17 dB signal in 8 bins invisible, and my refusal to abandoned that approach cost a lot more time that it could have.
Truncation too short : I cut my test clip off at 30 seconds, which prevented me from finding the last value of the last Costas array until I literally looked at the values with my eyes and realized there was an incomplete array there.
costas_check numpy comparison bug : I learned that != on a numpy array returns an element-wise array, not a boolean. Then, I learned about np.array_equal, which worked until I realized I didn’t need a Costas check at all.
costas_finder returning False : Instead of a check, I needed a finder, and my finder didn’t work because False == 0 in Python, which I knew, but didn’t realize it would shadow a real match at index 0 until, again, I printed the data out in the console and looked at it with my eyes and said, out loud, “Oh, that’s False because it’s zero, not because it’s False.”
Band detection threshold too high : Math, right? While trying to distinguish noise from signal, std was inflated by a few very bright outlier bins, pushing my threshold above the actual signal. Eventually, I replaced mean + N*std with median + N*MAD (median absolute deviation), which, as a note back in time to myself in seventh grade, “Yes, you’re going to actually use this someday.”
Architecture redesign : When I did finally give up my flawed approach of using the silent part of the transmission cycle to orient my signal-finding, I spent a lot time trying to make small, targeted changes toward Costas array alignment. That didn’t go well. I had to abandon my approach to fixing the approach I abandoned. Deleting all of my code and starting over again from scratch worked better.

Conclusion

Later, I’d like to come back and clean up this writing, break it up with some graphics and code snippets so it’s not a wall of text, and add links to explanations of terms, but right now I think the best thing for me to do is focus on keeping momentum going toward decoding.