RNN in C - is this BPTT finally right?

Sorry. I haven’t written any C code in over 20 years, so I’ve forgotten half of it…

Fair enough — C-only, core only:

Your recurrent BPTT part looks mostly fine.

The broken part is the softmax/output-gradient path.

Do not do this:

y[t][i] = exp(sum);
ssum += sum;
/* ... */
softmax_out = expf(y[t][i]) / softsum[t];

Do this shape instead:

z[t][i] = sum;        /* raw logit */
p[t][i] = softmax(z); /* probability */

Then backward starts with:

for (int i = 0; i < Y_S; i++)
    dy[i] = p[t][i];

dy[target[t]] -= 1.0f;

After that, the BPTT recurrence is basically the right structure:

dh = Why_T * dy + dh_next;

dh_raw = (1.0f - h[t + 1] * h[t + 1]) * dh;

dWhy += dy     * h[t + 1];
dby  += dy;

dWxh += dh_raw * x[t];
dWhh += dh_raw * h[t];
dbh  += dh_raw;

dh_next = Whh_T * dh_raw;

So: recurrent part close; softmax part not right yet.