lloyal::branch::DecodeSegmentsResult Struct Reference

What a heterogeneous prefill consumed. More...

#include <lloyal/branch.hpp>

Public Attributes
int64_t	cells = 0
llama_pos	advance = 0

Detailed Description

What a heterogeneous prefill consumed.

cells is KV cells added (the pressure cost); advance is how far the branch position moved. They differ when an EMBD segment's n_pos < n_rows.

Definition at line 419 of file branch.hpp.

Member Data Documentation

advance

llama_pos lloyal::branch::DecodeSegmentsResult::advance = 0

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 421 of file branch.hpp.

cells

int64_t lloyal::branch::DecodeSegmentsResult::cells = 0

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 420 of file branch.hpp.

---

The documentation for this struct was generated from the following file:

• include/lloyal/branch.hpp