Kina.ai

Specification

ADR: StoryLM-0 head-parallel attention

Context

Phase 3 Section 5 locked vocab-parallel lm_head and named head-parallel attention as the next StoryLM-0 axis. H = 2 divides evenly across a 2-device logical mesh.

@mesh / @sharded are still unimplemented. Rank-4 attention, slice, and concat already exist.

Decision

Specify 2-way head-parallel attention as the existing composition in 06-storylm0-head-parallel.md: slice Q / Kᵀ / V on the head axis, attend with H = 1, concat the context. No new operator. Do not add @mesh. Wo and lm_head stay replicated.

Consequences