Kina.ai

Specification

ADR: StoryLM-0 decoder-wide AdamW

Context

Section 16 locks AdamW on one [4, 8] parameter. Section 15 and Section 24 update every decoder weight with SGD. Common LLM practice uses AdamW on the full trainable set.

A tuple-map or fused adamw would invent a parameter-bundle ABI. Section 16 already says to write one composition per tensor.

Decision

Specify decoder-wide AdamW as the existing composition in 25-storylm0-decoder-adamw.md: Section 15 reverseMode, then Section 16 independently on embed, Wq, Wk, Wv, Wo, and lm_head. No new operator. The host train instance uses the Section 24 corpus, lr = 0.2, λ = 0.01, 40 epochs.

Consequences