Context
AGENTS.md step 4 asks for KV-cache update semantics after
seeded sampling. llm_generate.kina doubles a fixed-shape
matrix inside while; that is not an append. A while carry
cannot change tensor shape. A fused cache primitive would
invent mutability and a write index that Phase 0 does not
define.
Phase 0 already specifies concat. Section 5 already specifies
rank-4 matmul. StoryLM-0 attention already uses Kᵀ layout
[B, H, Dh, S] and V layout [B, H, S, Dh].
Decision
Specify one decode-step cache update as the existing
composition in
19-kv-cache.md:
concat on the sequence axis, then q_new @ k_next and
softmax @ v_next. No new operator. The host holds the
returned cache and calls again. Do not grow S inside
while. Do not add scatter-into-T_max.
Consequences
storylm0_kv_step.kinaand a.sonumeric check land without a new compiler operator. Host staticconcatlowers throughinsert_sliceso one-shot-bufferize can emit a.so.- Visitors can see a typed cache append; they still cannot
grow
Sinsidewhile. - The host generation loop is Section 20.
Prefill handoff is Section 21.
T_maxscratch is Section 22.