Kina.ai

Specification

ADR: StoryLM-0 loss-op VJPs

Context

Section 9 compiles log_softmax and take_along_axis and leaves their reverse-mode rules unspecified. The restricted compiler AD pass (transform_ad) also lacks a VJP for the existing neg unary, which NLL uses. There is no user-facing scatter in the language.

Phase 2 AD-as-an-effect is a different design. This decision is only for the bootstrap compiler pass that already differentiates softmax, relu, and rank-2/3 matmul.

Decision

Specify and implement the VJPs in 10-log-softmax-and-take-along-axis-ad.md:

Do not change @main tuple lowering in this decision. Numeric checks are reference-formula + finite-difference + optional PyTorch, plus compile / golden / structural IR tests.

Consequences