Context
Phase 3 Section 6 locked head-parallel attention after a
replicated QKV projection and named column-parallel
Wq / Wk / Wv as the next StoryLM-0 axis. H · Dh = 4
divides evenly across a 2-device logical mesh.
@mesh / @sharded are still unimplemented. Rank-3
batched matmul, slice, reshape, transpose, and
Section 6 per-head attention already exist.
Row-parallel Wo would need an all-reduce after the output
projection. That collective is unspecified.
Decision
Specify 2-way column-parallel QKV as the existing
composition in
07-storylm0-proj-shards.md:
slice Wq / Wk / Wv on the H·Dh axis, project each
head, attend with H = 1, concat the context. No new
operator. Do not add @mesh. Wo and lm_head stay
replicated.
Consequences
storylm0_tp_proj.kinaand a.sonumeric check land without a new compiler operator.- Visitors can see per-head projections that match the unsharded forward; they still cannot parse a mesh.
- A combined vocab+heads instance, row-parallel
Wo, and collective lowering remain follow-on work.