lloyal::decode::TokenSource Class Reference

A token span as a stream of one TEXT segment, so text is placed by the rail like any other input. More...

#include <lloyal/decode.hpp>

Inheritance diagram for lloyal::decode::TokenSource:

Public Member Functions
	TokenSource (std::span< const llama_token > tokens)
size_t	size () override
	Number of segments in this prefill.
size_t	cells () const override
	KV cells this whole sequence will consume, known BEFORE anything decodes.
Segment	at (size_t) override
	Segment i. Invalidates any previously returned segment.
void	positions (size_t, llama_pos, llama_pos *) override
	Fill positions for an EMBD segment, given the absolute base the store chose.
Public Member Functions inherited from lloyal::decode::SegmentSource
virtual	~SegmentSource ()=default
virtual int32_t	input_of (size_t) const
	The input segment i belongs to (an image, a video, a recording), or -1 for prompt text that belongs to none.
virtual size_t	total_of (int32_t) const
	KV cells input input takes, summed over its segments: what progress on it counts towards.

Detailed Description

A token span as a stream of one TEXT segment, so text is placed by the rail like any other input.

Definition at line 740 of file decode.hpp.

Constructor & Destructor Documentation

TokenSource()

lloyal::decode::TokenSource::TokenSource ( std::span< const llama_token >  tokens )

inlineexplicit

Definition at line 742 of file decode.hpp.

Member Function Documentation

at()

Segment lloyal::decode::TokenSource::at ( size_t  i )

inlineoverridevirtual

Segment i. Invalidates any previously returned segment.

Implements lloyal::decode::SegmentSource.

Definition at line 745 of file decode.hpp.

cells()

size_t lloyal::decode::TokenSource::cells ( ) const

inlineoverridevirtual

KV cells this whole sequence will consume, known BEFORE anything decodes.

Exists so a caller can decide ADMISSION before touching a branch. decode_segments is not atomic: once the first segment dispatches the branch is mutated, so discovering mid-walk that the sequence does not fit costs the branch. Refusing up front costs nothing.

Text can already be measured by tokenizing it, but a source that encodes — a vision projector, an audio tower — knows a row count its caller cannot derive from the bytes it holds. Without this, such input is the one thing that bypasses a context-pressure gate.

Cells, not positions: a KV budget is spent in cells, and under M-RoPE a segment costs far more cells than it advances position. The unit matches DecodeSegmentsResult::cells, so a caller can compare what it was quoted against what it was charged.

Must equal the sum over at(0 .. size()-1) of tokens.size() for TEXT and n_rows for EMBD. A source that cannot know the count before encoding must not estimate: this is a budget promise, and an under-quote is spent out of someone else's budget.

Implements lloyal::decode::SegmentSource.

Definition at line 744 of file decode.hpp.

positions()

void lloyal::decode::TokenSource::positions ( size_t  i, llama_pos  base, llama_pos *  out  )

inlineoverridevirtual

Fill positions for an EMBD segment, given the absolute base the store chose.

out has n_rows * n_pos_per_embd entries, section-major: [s0...][s1...].... Never called for a TEXT segment.

The base is passed rather than exposed, because how it applies is model-specific — M-RoPE freezes one section and leaves another at zero, so a caller-side rebase would need the producer's model taxonomy.

Implements lloyal::decode::SegmentSource.

Definition at line 750 of file decode.hpp.

size()

size_t lloyal::decode::TokenSource::size ( )

inlineoverridevirtual

Number of segments in this prefill.

Implements lloyal::decode::SegmentSource.

Definition at line 743 of file decode.hpp.

---

The documentation for this class was generated from the following file:

• include/lloyal/decode.hpp