Single-branch grammar-constrained generation as an Effection operation
Creates a fresh branch (or forks from opts.parent), prefills the prompt,
generates to EOG, and prunes the branch. Uses Branch's async
iterator — single-branch generation doesn't need batched commit.
When parent is provided, the prompt is prefilled as a delta (with turn
separator) on a fork of the parent. This is the attention scratchpad
pattern: the fork sees the parent's context, attends to the prompt
content, generates a result, and is pruned — zero net KV cost.
The branch is always cleaned up via try/finally, even on error or
scope cancellation.
Single-branch grammar-constrained generation as an Effection operation
Creates a fresh branch (or forks from
opts.parent), prefills the prompt, generates to EOG, and prunes the branch. Uses Branch's async iterator — single-branch generation doesn't need batched commit.When
parentis provided, the prompt is prefilled as a delta (with turn separator) on a fork of the parent. This is the attention scratchpad pattern: the fork sees the parent's context, attends to the prompt content, generates a result, and is pruned — zero net KV cost.The branch is always cleaned up via try/finally, even on error or scope cancellation.