Context
storylm0_forward.kina emits [B, T, V] logits. Phase 1 gather cannot
select a class from that rank-3 tensor. log(softmax(x)) is the wrong
numerical form for a training loss.
Decision
Add two primitives, specified in
09-log-softmax-and-take-along-axis.md:
log_softmax— same typing assoftmax; stable max-shift +log(sum(exp)).take_along_axis— pick one element along a compile-time axis; indices have the values shape with that axis removed.
NLL is the composition -log_softmax.take_along_axis. No third primitive.
AD VJPs are a later decision
(36-storylm0-loss-ad.md).
Consequences
- The bootstrap
.kinasurface and the S-expression core both grow two forms. - Host MLIR goldens and negative type tests land with the ops.
- Device
take_along_axisand AD parity remain follow-on work.