Consolidated mutable state for a single branch. More...

#include <lloyal/branch.hpp>

Public Attributes
llama_context *	ctx = nullptr
	Llama context (not owned, must outlive branch)

const llama_model *	model = nullptr
	Llama model (not owned, must outlive branch)

llama_seq_id	seq_id = NO_LEASE
	KV cache sequence identifier (NO_LEASE when inactive)

llama_pos	position = 0
	Current decode position in the sequence.

llama_pos	fork_head = 0
	Parent's position at fork time (0 for root branches)

SamplerChainHandle	sampler_chain = 0
	Handle into BranchStore's sampler registry.

GrammarHandle	grammar = 0
	Handle into BranchStore's grammar registry.

CachedSamplingParams	cached_params
	Params used to create current chain (for memoization)

boundaries::BoundaryTracker *	boundary_tracker = nullptr
	Token boundary detector (owned, optional)

std::vector< llama_logit_bias >	logit_bias
	Static token biases, cloned on fork.

std::function< void(llama_token_data_array &)>	steer_fn
	Dynamic logit callback, NOT cloned on fork.

MetricsHandle	metrics = 0
	Handle into BranchStore's metrics registry.

llama_token	last_token = -1
	Last token returned by sample()

std::vector< llama_token_data >	last_candidates
	Filtered candidates from last sample()

std::vector< float >	logits_snapshot
	Captured logit distribution (n_vocab floats)

bool	has_logits = false
	True only after force_snapshot_logits(), prefill(), or step()

std::vector< llama_token_data >	candidates_buffer
	Reusable scratch buffer for sampling (avoids O(n_vocab) allocs per sample call).

int	n_batch = DEFAULT_N_BATCH
	Batch size for decode operations.

int	n_vocab = 0
	Vocabulary size (cached for buffer pre-allocation)

uint16_t	generation = 0
	Slot generation counter (for ABA prevention)

bool	in_use = false
	True when slot is allocated to an active branch.

BranchHandle	parent = INVALID_HANDLE
	Parent branch (INVALID_HANDLE if root)

std::vector< BranchHandle >	children
	Child branches forked from this one.

Detailed Description

Consolidated mutable state for a single branch.

Each branch encapsulates all state needed for independent generation:

Forkable state (cloned by fork()):

KV cache sequence (via llama_memory_seq_cp)
Sampler chain (handle into BranchStore registry)
Grammar constraints (handle into BranchStore registry)
Boundary tracker (token boundary detection)
Metrics (handle into BranchStore registry)
Logits snapshot (captured distribution for deferred sampling)
Logit bias (static token-level adjustments)

Non-forkable state (NOT cloned by fork()):

steer_fn (dynamic callback — may capture references, unsafe to copy)

Sampling application order in sample(): Grammar → Logit Bias → Steer → Sampler Chain

Definition at line 275 of file branch.hpp.

Member Data Documentation

◆ boundary_tracker

boundaries::BoundaryTracker* lloyal::branch::BranchState::boundary_tracker = nullptr

Token boundary detector (owned, optional)

Examples: /home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 288 of file branch.hpp.

◆ cached_params

CachedSamplingParams lloyal::branch::BranchState::cached_params

Params used to create current chain (for memoization)

Examples: /home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 286 of file branch.hpp.

◆ candidates_buffer

std::vector<llama_token_data> lloyal::branch::BranchState::candidates_buffer

Reusable scratch buffer for sampling (avoids O(n_vocab) allocs per sample call).

Memory footprint per branch for large vocab models (128k tokens):

logits_snapshot: n_vocab * 4 bytes (~512KB)
candidates_buffer: n_vocab * sizeof(llama_token_data) (~1.5–2MB)
last_candidates: typically ~40 entries with top_k=40 (~480 bytes)

Todo:
Move to per-thread or SessionContext scratch arena for deep search trees.