liblloyal 1.0.0
Branched Inference for llama.cpp
Loading...
Searching...
No Matches
lloyal::decode::PackedChunk Struct Reference

A chunk of item indices produced by bin_pack() More...

#include <lloyal/decode.hpp>

Public Attributes

std::vector< int32_t > indices
 Indices into the original items array.
 
bool oversized = false
 True → single item exceeding n_batch.
 

Detailed Description

A chunk of item indices produced by bin_pack()

Normal chunks contain items whose total tokens fit in n_batch. Oversized chunks contain a single item whose tokens exceed n_batch (caller must dispatch via decode::many with auto-chunking).

Definition at line 460 of file decode.hpp.

Member Data Documentation

◆ indices

std::vector<int32_t> lloyal::decode::PackedChunk::indices

Indices into the original items array.

Definition at line 461 of file decode.hpp.

◆ oversized

bool lloyal::decode::PackedChunk::oversized = false

True → single item exceeding n_batch.

Definition at line 462 of file decode.hpp.


The documentation for this struct was generated from the following file: