Context
Section 14 SGDs the embedding table from seed ones on the logits.
Section 10 already has VJPs for the NLL composition. Section 11
already allows one SGD composition per tensor. Neither slice put
NLL and every decoder weight in one program.
Mean reduction of [B, T] NLL is unspecified. Next-token shift
inside the graph is unspecified.
Decision
Specify the composition in
15-nll-decoder-train.md:
- Differentiated body = decoder let-spine + Section 9 NLL. Primal is
per-token NLL. Seed
onesis sum of token NLLs, not mean. token_idsandtargetsare discreteI64parameters.- After projection, apply Section 11 SGD to
{token_embedding, wq, wk, wv, wo, lm_head}. Bind and discard mask / scale cotangents. - The wrapper returns the constructed tuple
(nll, emb', wq', wk', wv', wo', lm')under the existing Section 12 multi-result ABI.
No new operator.
Consequences
storylm0_nll_step.kinais the host numeric instance.- AdamW, mean NLL, in-graph target shift, sampling, and KV-cache remain later sections.