KV Tenancy

Vacancy manager for seq_ids — the scarce KV cache resource. More...

Namespaces
namespace	lloyal::kv::tenancy

Variables
constexpr llama_seq_id	lloyal::kv::NO_LEASE = static_cast<llama_seq_id>(-1)
	Sentinel value indicating a branch has no KV residency.

Detailed Description

Vacancy manager for seq_ids — the scarce KV cache resource.

A llama context has a fixed pool of seq_ids (typically 1–256). Each seq_id represents an independent recurrent state in the KV cache. Tenancy tracks which sequences are leased (owned by a branch) and which are vacant (available for allocation), providing symmetric acquire/release lifecycle.

Key invariant

A seq_id in the vacant pool is always clean — it has no KV cache tags. evict() strips tags before releasing; fork()'s debug assert verifies this on acquisition.

LIFO reuse

Vacant seq_ids are stored as a LIFO stack. This is a locality heuristic (recently freed seqs may still be warm in cache), not a correctness requirement.

Variable Documentation

NO_LEASE

constexpr llama_seq_id lloyal::kv::NO_LEASE = static_cast<llama_seq_id>(-1)

constexpr

Sentinel value indicating a branch has no KV residency.

-1 is chosen because 0 is a valid seq_id. Used as the default seq_id on freshly allocated (but not yet leased) branch slots.

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp, and /home/runner/work/liblloyal/liblloyal/include/lloyal/kv.hpp.

Definition at line 206 of file kv.hpp.