Kina.ai

Specification

ADR: StoryLM-0 NLL ∘ decoder train step

Context

Section 14 SGDs the embedding table from seed ones on the logits. Section 10 already has VJPs for the NLL composition. Section 11 already allows one SGD composition per tensor. Neither slice put NLL and every decoder weight in one program.

Mean reduction of [B, T] NLL is unspecified. Next-token shift inside the graph is unspecified.

Decision

Specify the composition in 15-nll-decoder-train.md:

No new operator.

Consequences