Input item for decode::embd — embedding rows for one sequence. More...

#include <lloyal/decode.hpp>

Public Attributes
const float *	rows = nullptr
	n_rows x n_embd_inp floats, caller-owned, valid for the call's duration

int32_t	n_rows = 0

int32_t	n_embd_inp = 0
	Row width — llama_model_n_embd_inp(model), the INPUT dim (not n_embd)

const llama_pos *	pos = nullptr
	Section-major positions: n_rows * n_pos_per_embd entries, laid out [s0...][s1...]... Under M-RoPE the producer decides what each section means; this layer only slices them.

int32_t	n_pos_per_embd = 1
	1 (plain positions) or 4 (M-RoPE)

llama_seq_id	seq_id = 0

bool	non_causal = false
	Bracket the whole decode in non-causal attention (Gemma-class projectors; requires n_ubatch >= n_rows since the block cannot split)

bool	output_logits = false
	When true, compute logits for the LAST row of the last sub-chunk.

Detailed Description

Input item for decode::embd — embedding rows for one sequence.

The embedding-rail counterpart of ScatterItem. rows is CALLER-owned and never copied: an encoder's output buffer is pointed at directly.

Definition at line 612 of file decode.hpp.

Member Data Documentation

◆ n_embd_inp

int32_t lloyal::decode::EmbdItem::n_embd_inp = 0

Row width — llama_model_n_embd_inp(model), the INPUT dim (not n_embd)

Definition at line 617 of file decode.hpp.

◆ n_pos_per_embd

int32_t lloyal::decode::EmbdItem::n_pos_per_embd = 1

1 (plain positions) or 4 (M-RoPE)

Definition at line 623 of file decode.hpp.

◆ n_rows

int32_t lloyal::decode::EmbdItem::n_rows = 0

Definition at line 615 of file decode.hpp.

◆ non_causal

bool lloyal::decode::EmbdItem::non_causal = false

Bracket the whole decode in non-causal attention (Gemma-class projectors; requires n_ubatch >= n_rows since the block cannot split)

Definition at line 627 of file decode.hpp.

◆ output_logits

bool lloyal::decode::EmbdItem::output_logits = false

When true, compute logits for the LAST row of the last sub-chunk.

Rows are an interior prefix otherwise — a subsequent llama_decode resets the output buffer, so only a rows-terminal prefill needs this.

Definition at line 631 of file decode.hpp.

◆ pos

const llama_pos* lloyal::decode::EmbdItem::pos = nullptr

Section-major positions: n_rows * n_pos_per_embd entries, laid out [s0...][s1...]... Under M-RoPE the producer decides what each section means; this layer only slices them.

Definition at line 621 of file decode.hpp.

◆ rows

const float* lloyal::decode::EmbdItem::rows = nullptr

n_rows x n_embd_inp floats, caller-owned, valid for the call's duration

Definition at line 614 of file decode.hpp.

◆ seq_id

llama_seq_id lloyal::decode::EmbdItem::seq_id = 0

Definition at line 624 of file decode.hpp.

The documentation for this struct was generated from the following file:

include/lloyal/decode.hpp

Public Attributes