|
| bool | remove_range (llama_context *ctx, llama_seq_id seq, llama_pos p0, llama_pos p1) |
| | Remove token range from KV cache sequence.
|
| |
| llama_pos | pos_max (llama_context *ctx, llama_seq_id seq) |
| | Get maximum position in KV cache sequence.
|
| |
| void | seq_cp (llama_context *ctx, llama_seq_id src, llama_seq_id dst, llama_pos p0=0, llama_pos p1=-1) |
| | Copy KV cache from one sequence to another.
|
| |
| void | seq_keep (llama_context *ctx, llama_seq_id seq) |
| | Keep only one sequence, removing all others.
|
| |
| size_t | state_size (llama_context *ctx, llama_seq_id seq) |
| | Get size needed to serialize sequence state.
|
| |
| size_t | state_save (llama_context *ctx, llama_seq_id seq, uint8_t *dst, size_t size) |
| | Save sequence state to buffer.
|
| |
| size_t | state_load (llama_context *ctx, llama_seq_id seq, const uint8_t *src, size_t size) |
| | Restore sequence state from buffer.
|
| |
| size_t | global_state_size (llama_context *ctx) |
| | Get size needed to serialize global state.
|
| |
| size_t | global_state_save (llama_context *ctx, uint8_t *dst, size_t size) |
| | Save global state to buffer.
|
| |
| size_t | global_state_load (llama_context *ctx, const uint8_t *src, size_t size) |
| | Restore global state from buffer.
|
| |
| void | log_build_info (llama_context *ctx) |
| | Log KV cache build info and current state.
|
| |
| void | clear_all (llama_context *ctx) |
| | Clear all KV cache (complete reset)
|
| |
| void | clear_metadata (llama_context *ctx) |
| | Clear KV cache metadata only (fast reset)
|
| |
| void | clear_and_reseed (llama_context *ctx, const std::vector< llama_token > &original_sinks, const std::vector< llama_token > &tail, int32_t n_batch) |
| |
| size_t | write_file (llama_context *ctx, llama_seq_id seq, const std::string &filepath, const std::vector< llama_token > &tokens) |
| | Write KV state to file with self-describing format.
|
| |
| FileData | read_file (llama_context *ctx, llama_seq_id seq, const std::string &filepath) |
| |
| bool lloyal::kv::remove_range |
( |
llama_context * |
ctx, |
|
|
llama_seq_id |
seq, |
|
|
llama_pos |
p0, |
|
|
llama_pos |
p1 |
|
) |
| |
|
inline |
Remove token range from KV cache sequence.
Removes tokens in the range [p0, p1) from the specified sequence's KV cache.
- Warning
- LANDMINE — a PARTIAL range is only meaningful on attention layers.
The signature offers p0/p1, so rewinding a sequence to an earlier point looks like a supported operation. On attention it is: cells are per position, and removing a tail removes them. On a RECURRENT or hybrid model there is no per-position cell — the carrier has already folded every prior token into fixed-size state, so there is nothing at [p0, p1) to remove.
llama.cpp will rewind a recurrent sequence only where the model keeps per-token snapshots, bounded by n_rs_seq, and returns false otherwise. Qwen3.5's Gated DeltaNet — the default model — reports rs_seq = 0, so every partial rewind there fails. NEVER discard this function's return value: a caller that ignores it gets attention rolled back, the carrier untouched, and no indication the two now disagree. That is a silent wrong answer, the worst failure mode available.
Whole-sequence removal (0, -1) is safe everywhere and is what tenancy uses. To correct a lineage, do NOT rewind it: replay the content onto a fresh sequence. Replay re-runs the folds, which is why it is the portable form and keeps working when the layer type changes underneath you.
- Parameters
-
| ctx | Llama context (must not be null) |
| seq | Sequence ID (use 0 for single-sequence mode) |
| p0 | Start position (inclusive) |
| p1 | End position (exclusive), use -1 for "to end" |
- Returns
- true if successful, false if context is null or operation failed
- Warning
- CRITICAL: Call this BEFORE next llama_decode(), not after. Calling after decode may cause undefined behavior.
- Examples
- /home/runner/work/liblloyal/liblloyal/include/lloyal/kv.hpp, and /home/runner/work/liblloyal/liblloyal/include/lloyal/logits.hpp.
Definition at line 98 of file kv.hpp.
1.9.8