Kina.ai

Specification

ADR: StoryLM-0 QKV projection shards

Context

Phase 3 Section 6 locked head-parallel attention after a replicated QKV projection and named column-parallel Wq / Wk / Wv as the next StoryLM-0 axis. H · Dh = 4 divides evenly across a 2-device logical mesh.

@mesh / @sharded are still unimplemented. Rank-3 batched matmul, slice, reshape, transpose, and Section 6 per-head attention already exist.

Row-parallel Wo would need an all-reduce after the output projection. That collective is unspecified.

Decision

Specify 2-way column-parallel QKV as the existing composition in 07-storylm0-proj-shards.md: slice Wq / Wk / Wv on the H·Dh axis, project each head, attend with H = 1, concat the context. No new operator. Do not add @mesh. Wo and lm_head stay replicated.

Consequences