Strict Socratic mode
To begin, I used a stock set of instructions to put my LLM tool into a mode where it would not write any code or solve any problems, but instead only ask questions to help guide me toward sorting things out myself.
Socratic mode has been a perfect fit for this learning project so far.
I’m still comfortable with my complete ban on all LLM involvement in text writing, and Socratic mode gives me the same clarity with coding.
Reference recording
My test to determine how well my FT8 decoder works will be to use my decoder on a reference recording, then compare my results to the results from the JT9 binary utility multi-mode decoder written with the help of Joseph Hooton Taylor Jr., co-creator of the FT8 protocol.
From previous work, I have a record.py program that I used to make the reference recording with this command:
python3 record.py 14.074 usb --duration 20 --output ft8-challenge-reference.wavThese arguments specify that the recorder should:
- capture on frequency 14.074 MHz, which is a standard for FT8 transmissions
- use upper side band (usb) demodulation
- record 20 minutes of audio
- save the results in a file with a specific name
This worked as designed.
I’ve put a short sample of the recording here if you’re interested in hearing what a raw FT8 transmission sounds like.
Set your system volume on high and your personal expectations on low.
Like, super-low.
It’s mostly static with a faint, high-pitched whine in the background.
Decoding the reference
WSJT-X and JT9 are built to decode live audio at 12 MHz, not recordings at 40 Mhz.
So first, I used the sox utility to convert the sample rate with:
sox ft8-challenge-reference.wav -r 12000 ft8-challenge-reference-2.wavThen, in order to get JT9 to decode the 20-minutes of audio, I wrote this little script to chop the reference recording into pieces and decode each piece:
for ((i=0; i<1185; i+=15)); do
echo "begin $i to $((i+18)) seconds"
sox ft8-challenge-reference-2.wav ft8-$i.wav trim $i 18
jt9 -8 ft8-$i.wav
echo "end $i to $((i+18)) seconds"
rm ft8-$i.wav
done > reference-decode.txtThis loops through:
- writing a status message about how far along it is
- slicing off an 18-second-long piece of the recording and saving it as a file
- decoding that file with JT9 in FT8 mode
- writing a status message with an update on the progress
- deleting the file that step 2 created
Then, finally, saving all that into a file, which, in my case, looks like this:
begin 0 to 18 seconds
000000 -17 -1.5 1188 ~ K6IRK HH2JR -16
<DecodeFinished> 0 1 0
end 0 to 18 seconds
begin 15 to 33 seconds
000000 -6 -1.9 2896 ~ EA3PV K4RUM -02
000000 -7 -1.8 1985 ~ AE0US VP2MAA RR73
000000 -10 -1.8 1150 ~ K8GGG W4EIS EM13
000000 -14 -2.0 563 ~ JA7FLK N5YPJ DL99
000000 -14 -1.9 808 ~ JK1GTT TI5RR -08
<DecodeFinished> 0 5 0
end 15 to 33 seconds
begin 30 to 48 seconds
000000 -18 -1.5 1188 ~ K6IRK HH2JR -16
000000 -14 -1.9 1015 ~ CQ KA1CH FN41
<DecodeFinished> 0 2 0
end 30 to 48 seconds
begin 45 to 63 seconds
000000 0 -1.9 1985 ~ VA3YVE VP2MAA -14
000000 0 -2.0 2895 ~ EA3PV K4RUM RR73
000000 -12 -2.0 563 ~ JA7FLK N5YPJ DL99
000000 -17 -1.8 1150 ~ K8GGG W4EIS EM13
<DecodeFinished> 0 4 0Why are people sending and receiving these messages?
These are ham radio operators exchanging call signs and locations.
If you’re interested, the set of links at the bottom of the Wikipedia page on QSO is a good place to start.
Stripped of logging lines, each row is a decoded signal, like this:
000000 -17 -1.5 1188 ~ K6IRK HH2JR -16
000000 -6 -1.9 2896 ~ EA3PV K4RUM -02
000000 -7 -1.8 1985 ~ AE0US VP2MAA RR73
000000 -10 -1.8 1150 ~ K8GGG W4EIS EM13
000000 -14 -2.0 563 ~ JA7FLK N5YPJ DL99
000000 -14 -1.9 808 ~ JK1GTT TI5RR -08
000000 -18 -1.5 1188 ~ K6IRK HH2JR -16
000000 -14 -1.9 1015 ~ CQ KA1CH FN41
...There happen to be 199 decoded messages in the reference recording.
For each, we get:
- UTC time : always 000000 in my case because I split my recording into pieces and didn’t decode a live stream of audio
- dB : a measure of the signal-to-noise ratio (SNR), given in a logarithmic scale
- time-offset : a count of seconds between the beginning of the recording and the beginning of the decoded message
- transmission frequency : a measure, in hertz, of where, relative to the monitored frequency, the transmission happened
- ~ : A tilde, to separate information about the decoded message from the message itself
- message : the decoded payload of the transmission
For now, I’ll focus on getting my decoded messages to match the decoded messages from JT9.
I wish it were not so, but there is also, very confusingly, communication mode named JT9.
JT9, the mode, is an experimental branch of JT65, which is itself an experiment designed specifically for bouncing radio signals off the moon and detecting them back on earth, which has an absolutely hilarious history with my favorite Wikipedia-hosted diagram of all time.
Comparing results
Eventually, my decoder will take the same set of chopped up reference recordings and produce it’s own list of messages.
When that happens, I’ll need a way to compare my decoded messages to the known correct decoded messages.
This is complicated by the fact that, for a segment of audio with multiple messages, those messages can appear in any order.
My first approach to a comparison script was based on diff, which is a utility for comparing files line-by-line.
That won’t work well when the results can appear in any order.
After a brief detour into multiset comparison, I decided that, for this experiment, since the order of the decoded messages doesn’t really matter that much, I’ll sort all 199 known correct decoded messages alpha-numerically.
Then, I’ll sort all decoded messages from my code alpha-numerically.
Finally, I’ll use diff to compare the two.
Wrap
I went down the path of writing a python and regex utility to strip the metadata out of the decoded messages and sort them, but sed or awk can probably do this much more simply, and that’s when I ran out of time.
Writing this account of these first steps has taken roughly as long as the work itself.
I will lazily track that ratio without judgment for a while.
Maybe later, I’ll decide to adjust.