Phase 1, Section 22: `T_max` padded scratch

StoryLM-0 decode slice. Not a Phase 1 exit-criteria replacement. Not a new primitive. Uses Phase 0 concat, broadcast, and slice on the Section 19 / 21 cache tensors.

1. What this section commits to

Sections 19–21 keep the live cache at its exact length S. A host that wants a fixed-shape buffer across steps does not get an in-place write or a Phase 2 state effect. It pads the live Kᵀ / V to a compile-time T_max with zeros, and slices the live prefix back out before the next static-S decode.

There is no T_max keyword, write index, or scatter-assign. T_max is a static shape. The unused suffix is zeros and is never attended: decode still consumes the sliced live prefix, not the padded tensor.

This instance uses the Section 21 prompt (T = 3) and T_max = 5 (prompt plus two generated tokens).

2. Why not a primitive

concat, broadcast, and slice are already specified. A fused scratch op would invent mutability and a write index that Phase 0 does not define. Padding with zeros is the functional encoding of “preallocate T_max.”

3. Operands (numeric instance)

Layouts match storylm0_forward.kina: B = 1, H = 2, Dh = 2, T_max = 5. Host tensors. The pad tail is a host scalar 0 broadcast to the leftover extent (T_max - S).

Pad after prefill (S = 3):

Name	Type	Role
`k_live`	`Tensor[F32, [1, 2, 2, 3], @host]`	live `Kᵀ`
`v_live`	`Tensor[F32, [1, 2, 3, 2], @host]`	live `V`
`zero`	`Tensor[F32, [1, 1, 1, 1], @host]`	scalar `0`

Pad result:

k_pad : Tensor[F32, [1, 2, 2, 5], @host]
v_pad : Tensor[F32, [1, 2, 5, 2], @host]

Unpad is slice of that result: k_pad axis 3 [0, 3) → [1, 2, 2, 3]; v_pad axis 2 [0, 3) → [1, 2, 3, 2]. Those slices are views of the pad buffer (same extract_slice lowering as Section 21 last_logits).

After decode_s3 the live length is S = 4. pad_s4 concats a leftover of extent 1 onto [1, 2, 2, 4] / [1, 2, 4, 2] and again yields T_max = 5.

4. Operational meaning

k_tail = broadcast(zero, [B, H, Dh, T_max - S])
v_tail = broadcast(zero, [B, H, T_max - S, Dh])
k_pad  = concat[axis=3] k_live k_tail
v_pad  = concat[axis=2] v_live v_tail
k_live = slice[axis=3, start=0, stop=S] k_pad
v_live = slice[axis=2, start=0, stop=S] v_pad

Host loop for this instance (Section 21 handoff, then pad):

(k, v, z)     = prefill(prompt, mask, …)
(k5, v5)      = pad_s3(k, v, zero)           # S=3 → T_max=5
z_last        = slice[axis=1, start=2, stop=3] z
t3            = argmax(z_last, axis=2)
(k, v)        = unpad_s3(k5, v5)             # views of the prefix
(k4, v4, z4)  = decode_s3(t3, k, v, …)
(k5, v5)      = pad_s4(k4, v4, zero)         # S=4 → T_max=5
t4            = argmax(z4, axis=2)
sequence      = prompt ++ [t3, t4]

The suffix of each pad is exactly zero. The prefix equals the live cache. Attention never sees the suffix.

argmax is Section 17. decode_s3 is Section 20 / 21. Host padded broadcast maps size-1 source dims to affine 0 (same rule as Section 18).

5. Typing (derived)

Each line is existing broadcast / concat / slice typing. Cache ranks stay 4. Only the static sequence axis changes, and each S is a different compiled fn.

6. Surface instance

storylm0_tmax.kina binds pad_s3, unpad_s3, and pad_s4. @main calls pad_s3 with host placeholders. The numeric check also loads storylm0_prefill.kina from the same host driver (prefill, last-logit slice, decode_s3, argmax).

No new tokens. The steps sit outside any reverseMode function.

7. Tests

storylm0_tmax.kina — §6 program; golden accepted by mlir-opt; pad_s3 result is (tensor<1x2x2x5xf32>, tensor<1x2x5x2xf32>).
scripts/validate_tmax.py compiles that example and storylm0_prefill.kina to .so files, injects the Section 21 prompt [1, 4, 2] and a zero scalar, runs the §4 loop, and checks (a) pad prefix equals the live cache, (b) pad suffix is zero, (c) unpad views match the live prefix, and (d) tokens match Section 21 ([1, 4, 2, 5, 3]; NumPy, and the same algebra in PyTorch when present).

8. Out of scope

In-place write, scatter-assign, or a write index.
Growing S inside while.
Phase 2 state / mutation.
Changing storylm0_forward.kina.
The companion’s 32-token checkpoint test (now Section 23).
Sampled rollout (now Section 23).
Differentiating through pad / unpad.
Device pad / unpad.

9. Open issues

In-place write. Still unspecified. This section does not add one.
Prefill T family. One pad / unpad signature per (S, T_max) pair, same static-S stance as Section 20.
Sampled rollout after prefill. Specified as the T=36 host loop in Section 23.
Packed cache layout. A single tensor holding K and V is unspecified.