OllamaReplyInfo Struct Reference

#include <ollamareply.h>

Public Attributes
std::chrono::nanoseconds	duration
std::chrono::nanoseconds	loadDuration
std::chrono::nanoseconds	promptEvalDuration
int	promptEvalTokenCount
int	tokenCount
std::chrono::nanoseconds	totalDuration

Detailed Description

The OllamaReplyInfo class represents information about a reply from an LLM.

When an LLM generates a completion, the server generally will return some information about the completion, including the duration of the completion, the number of tokens received, and the duration of the prompt evaluation. This struct encapsulates such information. If any one of these fields is not available, it will be set to its default value.

Definition at line 20 of file ollamareply.h.

Member Data Documentation

duration

std::chrono::nanoseconds OllamaReplyInfo::duration

The time spent generating the reply.

Definition at line 37 of file ollamareply.h.

loadDuration

std::chrono::nanoseconds OllamaReplyInfo::loadDuration

The time spent loading the model.

Definition at line 25 of file ollamareply.h.

promptEvalDuration

std::chrono::nanoseconds OllamaReplyInfo::promptEvalDuration

The time spent evaluating the prompt.

Definition at line 31 of file ollamareply.h.

promptEvalTokenCount

int OllamaReplyInfo::promptEvalTokenCount

The number of tokens in the prompt.

Definition at line 28 of file ollamareply.h.

tokenCount

int OllamaReplyInfo::tokenCount

The number of tokens in the reply.

Definition at line 34 of file ollamareply.h.

totalDuration

std::chrono::nanoseconds OllamaReplyInfo::totalDuration

The total time from when the request was received by the server to when the reply was returned.

Definition at line 22 of file ollamareply.h.

---

The documentation for this struct was generated from the following file:

• ollamareply.h