{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiboji2o2nxuepskncpw6hgl5d3mr5j3g5dsadbfxg5qqkjttxgs7q",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mnesyg55hrx2"
  },
  "path": "/t/rnn-in-c-is-this-bptt-finally-right/176455#post_4",
  "publishedAt": "2026-06-03T08:48:41.000Z",
  "site": "https://discuss.huggingface.co",
  "textContent": "Sorry. I haven’t written any C code in over 20 years, so I’ve forgotten half of it…\n\n* * *\n\nFair enough — C-only, core only:\n\nYour recurrent BPTT part looks mostly fine.\n\nThe broken part is the softmax/output-gradient path.\n\nDo not do this:\n\n\n    y[t][i] = exp(sum);\n    ssum += sum;\n    /* ... */\n    softmax_out = expf(y[t][i]) / softsum[t];\n\n\nDo this shape instead:\n\n\n    z[t][i] = sum;        /* raw logit */\n    p[t][i] = softmax(z); /* probability */\n\n\nThen backward starts with:\n\n\n    for (int i = 0; i < Y_S; i++)\n        dy[i] = p[t][i];\n\n    dy[target[t]] -= 1.0f;\n\n\nAfter that, the BPTT recurrence is basically the right structure:\n\n\n    dh = Why_T * dy + dh_next;\n\n    dh_raw = (1.0f - h[t + 1] * h[t + 1]) * dh;\n\n    dWhy += dy     * h[t + 1];\n    dby  += dy;\n\n    dWxh += dh_raw * x[t];\n    dWhh += dh_raw * h[t];\n    dbh  += dh_raw;\n\n    dh_next = Whh_T * dh_raw;\n\n\nSo: recurrent part close; softmax part not right yet.",
  "title": "RNN in C - is this BPTT finally right?"
}