Package com.tnsai.llm.providers

Class HuggingFaceClient

java.lang.Object

com.tnsai.llm.providers.AbstractLLMClient

com.tnsai.llm.providers.AbstractOpenAICompatibleClient

com.tnsai.llm.providers.HuggingFaceClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class HuggingFaceClient
extends AbstractOpenAICompatibleClient

HuggingFace Inference API client using OpenAI-compatible endpoint.

Uses the /v1/chat/completions endpoint provided by HuggingFace's Text Generation Inference (TGI) servers, which is fully OpenAI-compatible, so this client is a thin subclass of AbstractOpenAICompatibleClient with no behavioural overrides (usage-token capture now lives in the base; cold-start "model loading" 503s are retried by the shared shouldRetry).

Supported features:

• Chat completions with system/user/assistant messages
• Tool calling (same format as OpenAI)
• Streaming via SSE (same format as OpenAI)
• Configurable base URL for Dedicated Inference Endpoints

Configuration via environment variables:

• HUGGINGFACE_API_KEY - Required API token (hf_... format)
• HUGGINGFACE_BASE_URL - Optional, defaults to https://api-inference.huggingface.co/v1
• HUGGINGFACE_ENDPOINT - Optional, alias for HUGGINGFACE_BASE_URL (for dedicated endpoints)

Example models: meta-llama/Llama-3.2-3B-Instruct, mistralai/Mistral-7B-Instruct-v0.3

See Also:

• HuggingFace API Documentation

Constructor Summary

Constructors

Constructor	Description
HuggingFaceClient(String model)
HuggingFaceClient(String model, float temperature)
HuggingFaceClient(String model, float temperature, Float topP, Integer maxTokens)
HuggingFaceClient(String model, float temperature, Integer maxTokens)
HuggingFaceClient(String baseUrl, String model, float temperature, Float topP, Integer maxTokens, String apiKey)
HuggingFaceClient(String baseUrl, String model, float temperature, Integer maxTokens, String apiKey)

Method Summary

Methods inherited from class com.tnsai.llm.providers.AbstractOpenAICompatibleClient

chat, streamChat

Methods inherited from class com.tnsai.llm.providers.AbstractLLMClient

getBaseUrl, getCallTimeoutMs, getMaxTokens, getModel, getTemperature, getTopP

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getProviderName, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

HuggingFaceClient

public HuggingFaceClient(String model)

HuggingFaceClient

public HuggingFaceClient(String model,
 float temperature)

HuggingFaceClient

public HuggingFaceClient(String model,
 float temperature,
 Integer maxTokens)

HuggingFaceClient

public HuggingFaceClient(String model,
 float temperature,
 Float topP,
 Integer maxTokens)

HuggingFaceClient

public HuggingFaceClient(String baseUrl,
 String model,
 float temperature,
 Integer maxTokens,
 String apiKey)

HuggingFaceClient

public HuggingFaceClient(String baseUrl,
 String model,
 float temperature,
 Float topP,
 Integer maxTokens,
 String apiKey)