lloyal::chat_out Namespace Reference

Chat output parsing (tool calls, reasoning, content) More...

Classes
struct	ParseResult
	Result from parsing model output. More...
struct	ToolCall
	A single tool call extracted from model output. More...

Functions
ParseResult	parse (const std::string &output, common_chat_format format, common_reasoning_format reasoning_format, bool is_partial=false, const std::string &generation_prompt="", const std::string &parser_data="")=delete
	The shape before the engine pin moved took a reasoning format third.
ParseResult	parse (const std::string &output, common_chat_format format, bool is_partial=false, const std::string &generation_prompt="", const common_peg_arena &parser={})
	Parse model output with explicit format.
ParseResult	parse (const llama_model *model, const std::string &output, bool is_partial=false)
	Parse model output with auto-detected format from model template.

Detailed Description

Chat output parsing (tool calls, reasoning, content)

Wraps llama.cpp's common_chat_parse() to extract structured content from model output. Pairs with lloyal::chat_in – use the format from chat_in::FormatResult to select the correct parser.

Function Documentation

parse() [1/3]

ParseResult lloyal::chat_out::parse ( const llama_model *  model, const std::string &  output, bool  is_partial = false  )

inline

Parse model output with auto-detected format from model template.

Convenience overload that detects the format from the model's template. More expensive than the explicit-format overload since it initializes templates and applies them to detect the format.

Parameters

model	Llama model pointer
output	The raw model output text to parse
is_partial	True if output is incomplete (streaming)

Returns

ParseResult with content, reasoning_content, and tool_calls

Note

Prefer the explicit-format overload when you already have a FormatResult.

See also

lloyal::chat_in::format()

Definition at line 207 of file chat_out.hpp.

parse() [2/3]

ParseResult lloyal::chat_out::parse ( const std::string &  output, common_chat_format  format, bool  is_partial = false, const std::string &  generation_prompt = "", const common_peg_arena &  parser = {}  )

inline

Parse model output with explicit format.

Uses the format detected by chat_in::format() to apply the correct parser. For most formats, this delegates to common_chat_parse() which handles 25+ model-specific output formats (DeepSeek, Mistral, Hermes, etc.).

Parameters

output	The raw model output text to parse
format	The chat format (from chat_in::FormatResult.format)
is_partial	True if output is incomplete (streaming)
generation_prompt	Generation prompt prefill text (e.g. "<think>")
parser	The PEG parser chat_in::format built for this conversation (chat_in::FormatResult.parser). It is an object graph, not a string: keep the FormatResult and hand its parser back here. How reasoning is parsed is decided when the parser is built, from the reasoning_format the template was applied with.

Returns

ParseResult with content, reasoning_content, and tool_calls

Note

This function never throws. On error, returns raw output as content.

Warning

For PEG format models (COMMON_CHAT_FORMAT_PEG_*), parser must be the one from chat_in::FormatResult::parser. A default arena parses nothing.

See also

lloyal::chat_in::format()

auto fmt = chat_in::format(model, inputs);
// ... generate tokens ...
auto parsed = chat_out::parse(output_text, fmt.format, false,
                               fmt.generation_prompt, fmt.parser);
if (!parsed.tool_calls.empty()) {
  // Handle tool calls
}

Cold Restart with Thinking Models

When storing assistant messages for potential cold restart (re-formatting the full conversation from scratch), parse output to separate reasoning from content. This is important for thinking models (Qwen3, DeepSeek-R1, etc.) where the raw output contains <think>...</think> blocks — storing raw output as content would re-inject thinking tags as literal text when re-formatted.

// After generating tokens, detokenize the raw output
std::string raw_output = detokenized_text;
 
// Parse: separates reasoning from content
auto parsed = chat_out::parse(raw_output, fmt.format, false,
    fmt.generation_prompt, fmt.parser);
 
// Store with separate fields for correct re-formatting on cold restart
json msg = {{"role", "assistant"}, {"content", parsed.content}};
if (!parsed.reasoning_content.empty()) {
  msg["reasoning_content"] = parsed.reasoning_content;
}
messages.push_back(msg);

Definition at line 155 of file chat_out.hpp.

parse() [3/3]

ParseResult lloyal::chat_out::parse ( const std::string &  output, common_chat_format  format, common_reasoning_format  reasoning_format, bool  is_partial = false, const std::string &  generation_prompt = "", const std::string &  parser_data = ""  )

delete

The shape before the engine pin moved took a reasoning format third.

The enum converts to bool, so an unmigrated call would compile and read DEEPSEEK as is_partial. Refuse it at compile time instead; reasoning is decided when the parser is built.