lloyal::branch::CachedSamplingParams Struct Reference

Concrete sampling params snapshot for memoization. More...

#include <lloyal/branch.hpp>

Public Member Functions
bool	operator== (const CachedSamplingParams &) const =default

Public Attributes
float	temperature = 0.8f
int32_t	top_k = 40
float	top_p = 0.95f
float	typical_p = 1.0f
float	min_p = 0.05f
float	penalty_repeat = 1.0f
float	penalty_freq = 0.0f
float	penalty_present = 0.0f
int32_t	penalty_last_n = 64
uint32_t	seed = 0

Detailed Description

Concrete sampling params snapshot for memoization.

Captures resolved parameter values at chain creation time. Used by set_sampler_params() to skip rebuilding the chain when params haven't changed. Satisfies SamplingParamsLike for re-creation.

Definition at line 215 of file branch.hpp.

Member Function Documentation

operator==()

bool lloyal::branch::CachedSamplingParams::operator== ( const CachedSamplingParams &  ) const

default

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Member Data Documentation

min_p

float lloyal::branch::CachedSamplingParams::min_p = 0.05f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 220 of file branch.hpp.

penalty_freq

float lloyal::branch::CachedSamplingParams::penalty_freq = 0.0f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 222 of file branch.hpp.

penalty_last_n

int32_t lloyal::branch::CachedSamplingParams::penalty_last_n = 64

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 224 of file branch.hpp.

penalty_present

float lloyal::branch::CachedSamplingParams::penalty_present = 0.0f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 223 of file branch.hpp.

penalty_repeat

float lloyal::branch::CachedSamplingParams::penalty_repeat = 1.0f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 221 of file branch.hpp.

seed

uint32_t lloyal::branch::CachedSamplingParams::seed = 0

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 225 of file branch.hpp.

temperature

float lloyal::branch::CachedSamplingParams::temperature = 0.8f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 216 of file branch.hpp.

top_k

int32_t lloyal::branch::CachedSamplingParams::top_k = 40

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 217 of file branch.hpp.

top_p

float lloyal::branch::CachedSamplingParams::top_p = 0.95f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 218 of file branch.hpp.

typical_p

float lloyal::branch::CachedSamplingParams::typical_p = 1.0f

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/branch.hpp.

Definition at line 219 of file branch.hpp.

---

The documentation for this struct was generated from the following file:

• include/lloyal/branch.hpp