Kina.ai

Specification

ADR: StoryLM-0 loss operations

Context

storylm0_forward.kina emits [B, T, V] logits. Phase 1 gather cannot select a class from that rank-3 tensor. log(softmax(x)) is the wrong numerical form for a training loss.

Decision

Add two primitives, specified in 09-log-softmax-and-take-along-axis.md:

NLL is the composition -log_softmax.take_along_axis. No third primitive. AD VJPs are a later decision (36-storylm0-loss-ad.md).

Consequences