Function createContext

createContext(
options: ContextOptions,
loadOptions?: LoadOptions,
): Promise<SessionContext>

Create a new inference context

Loads the appropriate native binary (with automatic GPU fallback) and creates an inference context for the specified model.

Parameters

options: ContextOptions
Context creation options
OptionalloadOptions: LoadOptions
Optional binary loading options (GPU variant selection)

Returns Promise<SessionContext>

Promise resolving to SessionContext instance

Example: Basic usage

const ctx = await createContext({
  modelPath: './model.gguf',
  nCtx: 2048,
  nThreads: 4
});

try {
  const tokens = await ctx.tokenize("Hello");
  await ctx.decode(tokens, 0);
  const token = ctx.sample({ temperature: 0.7 });
} finally {
  ctx.dispose();
}

Example: With GPU variant selection

// Request CUDA - falls back to CPU if unavailable
const ctx = await createContext(
  { modelPath: './model.gguf', nCtx: 4096 },
  { gpuVariant: 'cuda' }
);

Example: Using environment variable

// Set LLOYAL_GPU=cuda before running
// createContext will automatically use CUDA if available
const ctx = await createContext({ modelPath: './model.gguf' });