|
liblloyal 1.0.0
Branched Inference for llama.cpp
|
#include <lloyal/model_registry.hpp>
Static Public Member Functions | |
| static std::shared_ptr< llama_model > | acquire (const std::string &fsPath, const llama_model_params ¶ms) |
| Acquire a model from cache, or load from disk on cache miss. | |
Definition at line 115 of file model_registry.hpp.
|
inlinestatic |
Acquire a model from cache, or load from disk on cache miss.
Thread-safe. On a cache hit, promotes the existing weak_ptr to a shared_ptr (zero-cost). On a cache miss, loads the model from disk while holding the mutex.
| fsPath | Filesystem path to model file (file:// prefix auto-stripped) |
| params | Model load parameters (GPU layers, mmap, etc.) |
Definition at line 199 of file model_registry.hpp.