lloyal::decode::Scratch Struct Reference

Reusable scratch buffers for multi-sequence batch construction. More...

#include <lloyal/decode.hpp>

Public Member Functions
void	resize (int32_t n, int32_t n_pos_per_embd=1)
llama_batch	as_batch (int32_t n_tokens)
	ABI-sensitive: writes llama_batch fields directly (no common_batch_* wrapper exists for external-buffer batches).
llama_batch	as_embd_batch (int32_t n_tokens, const float *rows)
	The embedding-rail twin of as_batch(): embd points at CALLER-owned rows (never copied — that is the point), token is null.

Public Attributes
std::vector< llama_token >	tokens_
std::vector< llama_pos >	pos_
std::vector< int32_t >	n_seq_id_
std::vector< llama_seq_id >	seq_id_single_
std::vector< llama_seq_id * >	seq_id_ptrs_
std::vector< int8_t >	logits_

Detailed Description

Reusable scratch buffers for multi-sequence batch construction.

Holds pre-allocated vectors that back the llama_batch pointers. Reuse a single Scratch across calls to avoid per-decode allocation.

Definition at line 406 of file decode.hpp.

Member Function Documentation

as_batch()

llama_batch lloyal::decode::Scratch::as_batch ( int32_t  n_tokens )

inline

ABI-sensitive: writes llama_batch fields directly (no common_batch_* wrapper exists for external-buffer batches).

Audit on llama.cpp submodule bumps.

Definition at line 429 of file decode.hpp.

as_embd_batch()

llama_batch lloyal::decode::Scratch::as_embd_batch ( int32_t  n_tokens, const float *  rows  )

inline

The embedding-rail twin of as_batch(): embd points at CALLER-owned rows (never copied — that is the point), token is null.

A legacy llama_batch carries tokens or rows, not both. tokens_ is unused on this path.

Same ABI sensitivity as as_batch() — audit together.

Definition at line 446 of file decode.hpp.

resize()

void lloyal::decode::Scratch::resize ( int32_t  n, int32_t  n_pos_per_embd = 1  )

inline

Parameters

n	Tokens (or embedding rows) in the batch
n_pos_per_embd	Positions per entry — 1 for token batches, 4 for M-RoPE embedding batches, where pos_ is section-major and holds n * n_pos_per_embd entries.

Definition at line 418 of file decode.hpp.

Member Data Documentation

logits_

std::vector<int8_t> lloyal::decode::Scratch::logits_

Definition at line 412 of file decode.hpp.

n_seq_id_

std::vector<int32_t> lloyal::decode::Scratch::n_seq_id_

Definition at line 409 of file decode.hpp.

pos_

std::vector<llama_pos> lloyal::decode::Scratch::pos_

Definition at line 408 of file decode.hpp.

seq_id_ptrs_

std::vector<llama_seq_id*> lloyal::decode::Scratch::seq_id_ptrs_

Definition at line 411 of file decode.hpp.

seq_id_single_

std::vector<llama_seq_id> lloyal::decode::Scratch::seq_id_single_

Definition at line 410 of file decode.hpp.

tokens_

std::vector<llama_token> lloyal::decode::Scratch::tokens_

Definition at line 407 of file decode.hpp.

---

The documentation for this struct was generated from the following file:

• include/lloyal/decode.hpp