|
liblloyal 1.0.0
Branched Inference for llama.cpp
|
A chunk of item indices produced by bin_pack() More...
#include <lloyal/decode.hpp>
Public Attributes | |
| std::vector< int32_t > | indices |
| Indices into the original items array. | |
| bool | oversized = false |
| True → single item exceeding n_batch. | |
A chunk of item indices produced by bin_pack()
Normal chunks contain items whose total tokens fit in n_batch. Oversized chunks contain a single item whose tokens exceed n_batch (caller must dispatch via decode::many with auto-chunking).
Definition at line 460 of file decode.hpp.
| std::vector<int32_t> lloyal::decode::PackedChunk::indices |
Indices into the original items array.
Definition at line 461 of file decode.hpp.
| bool lloyal::decode::PackedChunk::oversized = false |
True → single item exceeding n_batch.
Definition at line 462 of file decode.hpp.