StoryLM-0 train → decode slice. Not a Phase 1 exit-criteria replacement. Not a new primitive. Uses Section 24 host SGD and Section 23 rollout. Weights and tokenization stay on the host.
1. What this section commits to
Section 23 rolls 32 tokens from the patterned fixture. Section 24 trains the toy cycle but only locks eight greedy ids and does not write a checkpoint the rollout can reload. Companion 47’s generation acceptance still wants a fixed tiny checkpoint whose 32-token greedy text is a taught continuation.
This section locks that handoff:
- Run the Section 24 SGD recipe (40 epochs,
lr = 0.1, three sliding windows, sum-NLL). - Persist the trained weights as a versioned host fixture
(
storylm0-checkpoint-v1). - Section 23 rollout from prompt
[1, 2, 3, 4]must emit the locked 36-id cycle (prompt ++ 32 generated).
There is no train keyword, no new .kina program, and no
change to the patterned storylm0-checkpoint-v0 regression.
2. Why not a primitive
Section 15 SGD, Section 23 re-forward, and host NPZ I/O are already specified. Fusing “train then generate” into one op would invent a checkpoint ABI inside the language. The host owns the fixture; Kina functions stay pure.
3. Operands (numeric instance)
Widths match Section 23: B = 1, T = 36, H = 2,
Dh = 2, D = 4, V = 8. Host tensors.
| Field | Value |
|---|---|
| Init | Section 23 patterned recipe (storylm0-checkpoint-v0) |
| Corpus | Section 24 storylm0-corpus-v0 |
| Train | 40 epochs, lr = 0.1, Section 15 sum-NLL SGD |
| Prompt | [1, 2, 3, 4] (once upon a time) |
| Pad | 0 |
| Generate | 32 |
| Sample | seed 42, T = 0.8, Uniform(ε, 1-ε) as Section 23 |
Locked greedy sequence (prompt ++ 32 generated):
[1, 2, 3, 4, 5, 6] × 6(once upon a time the end repeated six times). The first
twelve ids match Section 24’s eight-token lock.
4. Operational meaning
w0 = load_patterned_fixture() # Section 23 v0
w = host_sgd_loop(w0, corpus) # Section 24
save_npz(w) # storylm0-checkpoint-v1
tokens = greedy_rollout(prompt, w, 32) # Section 23 algebra
assert tokens == locked_36A second seeded sample run on the same Uniform stream must match itself. Changing a future id must not alter earlier logit rows (causal non-leak).
The patterned v0 fixture remains the Section 23 regression input. This section does not replace it.
5. Typing (derived)
No new typing. Each forward / argmax / Gumbel-max step is
existing Section 23 surface.
6. Surface instance
No new .kina program. Reuse storylm0_nll_step.kina and
storylm0_rollout.kina.
Fixture (host-owned, not a compiler ABI):
tests/compiler/fixtures/storylm0/
trained_manifest.json format storylm0-checkpoint-v1
trained_checkpoint.npz embed, wq, wk, wv, wo, lm_head, scale
vocab.json shared eight-class vocabscripts/export_trained_checkpoint.py regenerates the NPZ
from the Section 24 recipe (NumPy). Checked-in bytes must
match that export.
No new tokens. @mesh does not appear.
7. Tests
scripts/export_trained_checkpoint.py— deterministic NumPy export; bytes match the committed NPZ.scripts/validate_trained_rollout.py— loads v1, compilesstorylm0_rollout.kinato a.so, and checks (a) 32 greedy tokens, twice, equal the locked cycle, (b) 32 sampled tokens, twice, match Section 18 onRandomState(42), (c) causal non-leak on the trained forward (NumPy; the same algebra in PyTorch when present), and (d) decoded text isonce upon a time the end× 6.just validate-trained-rolloutruns the validator.- Section 23
just validate-rolloutstays on v0.
8. Out of scope
- Mean NLL as the train objective (Section 26 composition stays available; Section 24 stays sum-NLL).
- Decoder-wide AdamW as the v1 recipe (Section 25 remains a separate loop).
- A real corpus, tokenizer, or production checkpoint ABI.
- Top-k / top-p, in-place write, growing
Sinsidewhile. - Differentiating through the host loop.
- Phase 1 exit criteria.
9. Open issues
- AdamW-trained fixture. Optional second NPZ from
Section 25 if a prose lock is needed at
lr = 0.2. - Prefill-
T/ in-place write. Unchanged. - Top-k / top-p. Still need a specified sort.
reduce_meanVJP. Still out of scope (Section 26).