Package com.tnsai.llm.providers

Class LlamaCppServerClient

java.lang.Object

com.tnsai.llm.providers.AbstractLLMClient

com.tnsai.llm.providers.AbstractOpenAICompatibleClient

com.tnsai.llm.providers.LlamaCppServerClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class LlamaCppServerClient
extends AbstractOpenAICompatibleClient

llama.cpp server LLM client.

llama-server is the HTTP server binary shipped with the llama.cpp project. It exposes an OpenAI-compatible chat-completions endpoint at http://localhost:8080/v1 by default — same wire format as every other OpenAI-shape provider in this module. The differences from cloud providers are operational:

• No API key required by default. llama.cpp's server runs ungated when self-hosted; the Authorization header is only sent when an apiKey is explicitly supplied (e.g. when llama-server is started with --api-key or fronted by an auth proxy).
• Model = the GGUF file the server loaded. The --model flag at server-start determines what's served; probe GET /v1/models to discover the identifier and pass it as model.

llama.cpp's small footprint makes it the lightest possible LLM server — runs on Raspberry Pi 5-class ARM hardware and Apple Silicon laptops equally well. Pairs naturally with edge-class sandbox profiles where Ollama and vLLM are too heavy.

Configuration via environment variables:

• LLAMACPP_BASE_URL — Optional override, defaults to http://localhost:8080/v1.
• LLAMACPP_API_KEY — Optional bearer token for llama-server instances started with --api-key or behind an auth proxy; omitted by default.

This client follows the OllamaClient / LMStudioClient / VLLMClient pattern: it accepts the apiKey parameter for parity with the other providers but does not call requireApiKey — passing null is fine for the self-hosted-default deployment.

See Also:

• llama.cpp server docs

Constructor Summary

Constructors

Constructor	Description
LlamaCppServerClient(String model)
LlamaCppServerClient(String model, float temperature)
LlamaCppServerClient(String model, float temperature, Float topP, Integer maxTokens)
LlamaCppServerClient(String model, float temperature, Integer maxTokens)
LlamaCppServerClient(String baseUrl, String model, float temperature, Float topP, Integer maxTokens, String apiKey)
LlamaCppServerClient(String baseUrl, String model, float temperature, Integer maxTokens, String apiKey)

Method Summary

Methods inherited from class com.tnsai.llm.providers.AbstractOpenAICompatibleClient

chat, streamChat

Methods inherited from class com.tnsai.llm.providers.AbstractLLMClient

getBaseUrl, getCallTimeoutMs, getMaxTokens, getModel, getTemperature, getTopP

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getProviderName, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

LlamaCppServerClient

public LlamaCppServerClient(String model)

LlamaCppServerClient

public LlamaCppServerClient(String model,
 float temperature)

LlamaCppServerClient

public LlamaCppServerClient(String model,
 float temperature,
 Integer maxTokens)

LlamaCppServerClient

public LlamaCppServerClient(String model,
 float temperature,
 Float topP,
 Integer maxTokens)

LlamaCppServerClient

public LlamaCppServerClient(String baseUrl,
 String model,
 float temperature,
 Integer maxTokens,
 String apiKey)

LlamaCppServerClient

public LlamaCppServerClient(String baseUrl,
 String model,
 float temperature,
 Float topP,
 Integer maxTokens,
 String apiKey)