A segment of a stream once it is in the KV: what moved, for the caller's books. More...

#include <lloyal/decode.hpp>

Public Attributes
size_t	index = 0
	Its index in the source.

Segment::Kind	kind = Segment::Kind::Text

int32_t	cells = 0
	KV cells it took: tokens for TEXT, rows for EMBD.

llama_pos	n_pos = 0
	Position it advanced: the token count for TEXT, `n_pos` for EMBD.

bool	has_logits = false
	Whether its last entry computed logits.

Detailed Description

A segment of a stream once it is in the KV: what moved, for the caller's books.

Definition at line 909 of file decode.hpp.

Member Data Documentation

◆ cells

int32_t lloyal::decode::LandedSegment::cells = 0

KV cells it took: tokens for TEXT, rows for EMBD.

Definition at line 914 of file decode.hpp.

◆ has_logits

bool lloyal::decode::LandedSegment::has_logits = false

Whether its last entry computed logits.

Placed one segment per dispatch, every TEXT segment does, and an EMBD segment only when it ends the stream; placed whole, only the segment that ends the stream does.

Definition at line 920 of file decode.hpp.

◆ index

size_t lloyal::decode::LandedSegment::index = 0

Its index in the source.

Definition at line 911 of file decode.hpp.

◆ kind

Segment::Kind lloyal::decode::LandedSegment::kind = Segment::Kind::Text

Definition at line 912 of file decode.hpp.

◆ n_pos

llama_pos lloyal::decode::LandedSegment::n_pos = 0

Position it advanced: the token count for TEXT, n_pos for EMBD.

Definition at line 916 of file decode.hpp.

The documentation for this struct was generated from the following file:

include/lloyal/decode.hpp