How a context reads what is placed into it: the one read of its traits that placement decides from. More...

#include <lloyal/decode.hpp>

Public Member Functions
bool	whole () const
	With no memory, attention that is not causal, or pooling, a dispatch sees only its own entries, so a stream placed in pieces would be read in pieces.

bool	takes_bidirectional () const
	Whether a bidirectional block can ride inside one mixed batch.

Public Attributes
bool	keeps_memory = true
	It keeps what it was given (a KV cache), so a later dispatch attends to an earlier one.

bool	causal = true
	Its attention is causal.

bool	pooled = false
	It pools a sequence into one output.

Detailed Description

How a context reads what is placed into it: the one read of its traits that placement decides from.

Definition at line 927 of file decode.hpp.

Member Function Documentation

◆ takes_bidirectional()

bool lloyal::decode::StreamReading::takes_bidirectional ( ) const

inline

Whether a bidirectional block can ride inside one mixed batch.

llama.cpp encodes a context without memory with causal attention off for the whole batch; a context that keeps memory and attends causally cannot, since its causal mode is context-wide.

Definition at line 945 of file decode.hpp.

◆ whole()

bool lloyal::decode::StreamReading::whole ( ) const

inline

With no memory, attention that is not causal, or pooling, a dispatch sees only its own entries, so a stream placed in pieces would be read in pieces.

Any other context (a reasoning model's: memory, causal, no pooling) is placed a segment at a time.

Definition at line 939 of file decode.hpp.

Member Data Documentation

◆ causal

bool lloyal::decode::StreamReading::causal = true

Its attention is causal.

Definition at line 931 of file decode.hpp.

◆ keeps_memory

bool lloyal::decode::StreamReading::keeps_memory = true

It keeps what it was given (a KV cache), so a later dispatch attends to an earlier one.

Definition at line 929 of file decode.hpp.

◆ pooled

bool lloyal::decode::StreamReading::pooled = false

It pools a sequence into one output.

Definition at line 933 of file decode.hpp.

The documentation for this struct was generated from the following file:

include/lloyal/decode.hpp

Public Member Functions

Public Attributes