Context
Phase 3 Section 5 locked vocab-parallel lm_head and named
head-parallel attention as the next StoryLM-0 axis. H = 2
divides evenly across a 2-device logical mesh.
@mesh / @sharded are still unimplemented. Rank-4
attention, slice, and concat already exist.
Decision
Specify 2-way head-parallel attention as the existing
composition in
06-storylm0-head-parallel.md:
slice Q / Kᵀ / V on the head axis, attend with
H = 1, concat the context. No new operator. Do not add
@mesh. Wo and lm_head stay replicated.
Consequences
storylm0_tp_heads.kinaand a.sonumeric check land without a new compiler operator.- Visitors can see per-head attention that matches the unsharded forward; they still cannot parse a mesh.
- Projection-weight shards are Section 7. A combined vocab+heads instance and collective lowering remain follow-on work.