model_registry.hpp File Reference

Thread-Safe Model Cache. More...

#include "common.hpp"
#include <functional>
#include <llama/llama.h>
#include <memory>
#include <mutex>
#include <string>
#include <unordered_map>

Go to the source code of this file.

Classes
struct	lloyal::ModelKey
	Model cache key combining file path and GPU configuration. More...
struct	lloyal::ModelKeyHash
class	lloyal::ModelRegistry

Namespaces
namespace	lloyal
	Boundary Tracker Stub for OSS liblloyal.
namespace	lloyal::detail

Macros
#define	LLOYAL_NO_SANITIZE_OVERFLOW
	Hash functor for ModelKey.

Detailed Description

Thread-Safe Model Cache.

Provides weak-pointer cache to avoid reloading same model multiple times. Uses inline static members (C++17) for header-only implementation with static state.

Cache key: (canonPath, n_gpu_layers, use_mmap) Cache value: weak_ptr to llama_model (auto-cleanup when last context releases)

Thread-safe via std::mutex for all cache operations.

Definition in file model_registry.hpp.

Macro Definition Documentation

LLOYAL_NO_SANITIZE_OVERFLOW

#define LLOYAL_NO_SANITIZE_OVERFLOW

Hash functor for ModelKey.

Computes combined hash of path, GPU layers, and mmap flag for use in std::unordered_map. Uses XOR with golden ratio constant for good distribution.

Examples

/home/runner/work/liblloyal/liblloyal/include/lloyal/model_registry.hpp.

Definition at line 56 of file model_registry.hpp.