Phase 1, Section 11: Functional SGD

StoryLM-0 update slice. Not a Phase 1 exit-criteria replacement. Not a new primitive. Uses Section 2 / Phase 0 elementwise ops and broadcast.

1. What this section commits to

A training step needs a pure parameter update. AdamW, momentum, and optimizer state are later. This section locks the first update as plain SGD:

w' = w - broadcast(lr, shape(w)) * g

w and g have the same dtype, shape, and placement. lr is a tensor that is broadcastable to shape(w) under the existing Phase 0 §11 rule (already implemented as broadcast).

There is no sgd keyword, IR constructor, or FFI. The compiler does not grow a new typing rule beyond restating the composition.

2. Why not a primitive

The arithmetic is already specified: broadcast, mul, sub. A fused sgd op would invent semantics (rounding, fused multiply-add, in-place update) that Phase 0 does not define. Functional SGD is the composition above. AdamW stays a later composition (or a later primitive, if a future section justifies one).

3. Typing (derived)

Γ ⊢ w  : Tensor[T, σ]
Γ ⊢ g  : Tensor[T, σ]
Γ ⊢ lr : Tensor[T, ρ]
T floating (implemented frem set)
broadcastable(ρ, σ)
w.placement = g.placement = lr.placement
─────────────────────────────────────────
Γ ⊢ w - lr.broadcast(σ) * g : Tensor[T, σ]

mul and sub still require equal shapes after the broadcast. A direct lr * g is a type error when ρ ≠ σ.

The StoryLM-0 numeric instance (LM-head layout from storylm0_forward.kina):

w, g : Tensor[F32, [4, 8], @host]
lr   : Tensor[F32, [1, 1], @host]
w'   : Tensor[F32, [4, 8], @host]

[1, 1] is a ranked stand-in for a scalar step size so the host .so ABI stays a ranked memref. Rank-0 Tensor[T, []] is already broadcastable to any σ and is allowed; it is not the numeric-check instance.

4. Operational meaning

lr_b[i] = lr[0, 0]     (after broadcast to σ)
w'[i]   = w[i] - lr_b[i] * g[i]

No weight decay, momentum, Nesterov, clipping, or in-place store. Each call produces a new tensor.

Several parameters update independently with the same rule. This slice does not add a tuple-map combinator; write one composition per tensor.

5. Surface

S-expression	Bootstrap `.kina`
`(sub w (mul (broadcast [4 8] lr) g))`	`w - lr.broadcast([4, 8]) * g`

No new tokens.

6. Relation to reverse-mode AD

Section 12 returns g from @main / a wrapper fn. This section’s examples still take w and g as explicit arguments. Naming those gradients inside .kina is Section 13.

7. Tests

sgd_step.kina — host LM-head update; golden accepted by mlir-opt.
Shape-mismatch lr * g without broadcast is rejected.
scripts/validate_sgd.py compiles the example to a .so, injects patterned w / g / lr, and matches w - lr * g (NumPy; PyTorch when present).

8. Out of scope

AdamW, momentum, weight decay, gradient accumulation.
In-place or buffer-updating updates.
Device-only kernels (host and device use existing elementwise lowering; the numeric instance is host).
argmax, sampling, KV-cache.
Tuple projection (now Section 13).

9. Open issues

Tuple projection. Specified in Section 13.
AdamW. Specified as composition in Section 16.
Per-parameter learning rates / schedules. Host-provided lr tensors; no clock or step counter in this section.