Phase 1, Section 24: StoryLM-0 host train loop

StoryLM-0 train slice. Not a Phase 1 exit-criteria replacement. Not a new primitive. Uses Section 15 NLL ∘ SGD and Section 23 rollout. Tokenization and the corpus stay on the host.

1. What this section commits to

Section 23 can load a fixture and roll 32 tokens. Those weights are the patterned recipe, so greedy text is not a taught continuation. Section 15 already updates every trainable weight in one step. This section locks a host loop around that step.

The instance is:

• Init from the Section 23 patterned fixture.
• A three-window toy corpus over the eight-class vocab, sliding through once upon a time the end and wrapping to once.
• 40 epochs, lr = 0.1, Section 15 sum-NLL SGD on {embed, Wq, Wk, Wv, Wo, lm_head}.
• After training, Section 23 greedy from prompt [1, 2, 3, 4] must emit the locked cycle [1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6].

There is no train keyword, no mean NLL, and no AdamW on the decoder. Next-token shift stays a host targets tensor (Section 15 §2).

2. Why not a primitive

reverseMode, Section 11 SGD, and the Section 23 host forward are already specified. A fused train op would invent an optimizer ABI and a data loader. Looping the existing step is the functional encoding of “fit the tiny fixture.”

3. Operands (numeric instance)

Widths match Section 15: B = 1, T = 4, H = 2, Dh = 2, D = 4, V = 8. Host tensors. Vocab is the Section 23 JSON (<pad> once upon a time the end <unk>).

Corpus windows:

lr : Tensor[F32, [1, 1], @host] is 0.1. Each epoch visits the three windows in that order. 40 epochs.

Init weights are the Section 23 fixture recipe. Mask and scale stay host constants.

4. Operational meaning

w = load_patterned_fixture()
for epoch in 0 .. 39:
    for (ids, targets) in corpus:
        (nll, w) = storylm0_nll_step(ids, targets, w, lr)
sequence = greedy_rollout(prompt=[1, 2, 3, 4], w, steps=8)

greedy_rollout is Section 23 (T = 36 re-forward + argmax). The first eight generated ids, prefixed by the prompt, must equal

[1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6]

(once upon a time the end once upon a time the end).

Corpus sum of per-token NLL (three windows, seed ones semantics) must fall from the init value to < 0.5. Changing a future id on the trained forward must not alter earlier logit rows.

Weights are still a toy fixture, not a creative-writing checkpoint.

5. Typing (derived)

Each step is the existing Section 15 wrapper type (nll, emb', wq', wk', wv', wo', lm'). The host rebinds those results as the next call’s weights. Shapes do not change across epochs.

6. Surface instance

No new .kina program. scripts/validate_train.py loads storylm0_nll_step.kina and storylm0_rollout.kina from the same host driver. The corpus is

tests/compiler/fixtures/storylm0/corpus.json

format storylm0-corpus-v0.

No new tokens. The loop sits outside any reverseMode function.

7. Tests

• scripts/validate_train.py compiles both examples to .so files, runs the §4 loop, and checks (a) Kina weight updates match NumPy Section 15 SGD on the first step, (b) corpus sum-NLL after 40 epochs is < 0.5 and below init, (c) Kina greedy 8-token continuation matches the locked cycle and NumPy, and (d) the causal non-leak on the trained T = 36 forward (NumPy; the same algebra in PyTorch when present).

8. Out of scope

• Mean NLL as this loop’s objective (see Section 26).
• Decoder-wide AdamW (see Section 25).
• A real corpus, tokenizer, or production checkpoint ABI.
• Growing S inside while, or an in-place write.
• Top-k / top-p.
• Differentiating through the epoch loop.
• Phase 1 exit criteria.

9. Open issues

1. Mean NLL. Specified in Section 26. This loop stays sum-NLL.
2. Decoder-wide AdamW. Specified in Section 25.
3. Trained 32-token fixture. Specified in Section 27.
4. Prefill-T family / in-place write. Unchanged.