Context
Section 9 compiles log_softmax and take_along_axis and leaves their
reverse-mode rules unspecified. The restricted compiler AD pass
(transform_ad) also lacks a VJP for the existing neg unary, which NLL
uses. There is no user-facing scatter in the language.
Phase 2 AD-as-an-effect is a different design. This decision is only for
the bootstrap compiler pass that already differentiates softmax, relu,
and rank-2/3 matmul.
Decision
Specify and implement the VJPs in
10-log-softmax-and-take-along-axis-ad.md:
neg:g_a = -g_v.log_softmax:g_x = g - sum(g) * exp(y)withythe primal.take_along_axis: no index adjoint; values cotangent is an IR-internalScatterAlongAxis(compare/select over the destination shape). No surfacescatter.
Do not change @main tuple lowering in this decision. Numeric checks are
reference-formula + finite-difference + optional PyTorch, plus compile /
golden / structural IR tests.
Consequences
- The typed IR grows one constructor that the parser never produces.
- Host MLIR goldens for
ad_neg,ad_log_softmax, andad_nllland with the pass. - Device
take_along_axis, Kina-executed.sograds, and Phase 2 handler AD remain follow-on work.