Package com.tnsai.llm.providers

Class VLLMClient

java.lang.Object

com.tnsai.llm.providers.AbstractLLMClient

com.tnsai.llm.providers.AbstractOpenAICompatibleClient

com.tnsai.llm.providers.VLLMClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class VLLMClient
extends AbstractOpenAICompatibleClient

vLLM LLM client.

vLLM is a high-throughput self-hosted inference server (typically invoked as python -m vllm.entrypoints.openai.api_server). It exposes an OpenAI-compatible HTTP server at http://localhost:8000/v1 by default — same wire format as every other OpenAI-shape provider in this module. The differences from cloud providers are operational:

• No API key required by default. vLLM runs ungated when self-hosted; the Authorization header is only sent when an apiKey is explicitly supplied (e.g. when vLLM is started with --api-key or fronted by an auth proxy).
• Model name = whatever the user loaded with vLLM's --model flag. Probe via GET /v1/models to discover the currently-served model identifier and pass it as model.

Configuration via environment variables:

• VLLM_BASE_URL — Optional override, defaults to http://localhost:8000/v1.
• VLLM_API_KEY — Optional bearer token for vLLM instances started with --api-key or behind an auth proxy; omitted by default.

This client follows the OllamaClient / LMStudioClient pattern: it accepts the apiKey parameter for parity with the other providers but does not call requireApiKey — passing null is fine for the self-hosted-default deployment.

See Also:

• vLLM OpenAI-compatible server docs

Constructor Summary

Constructors

Constructor	Description
VLLMClient(String model)
VLLMClient(String model, float temperature)
VLLMClient(String model, float temperature, Float topP, Integer maxTokens)
VLLMClient(String model, float temperature, Integer maxTokens)
VLLMClient(String baseUrl, String model, float temperature, Float topP, Integer maxTokens, String apiKey)
VLLMClient(String baseUrl, String model, float temperature, Integer maxTokens, String apiKey)

Method Summary

Methods inherited from class com.tnsai.llm.providers.AbstractOpenAICompatibleClient

chat, streamChat

Methods inherited from class com.tnsai.llm.providers.AbstractLLMClient

getBaseUrl, getCallTimeoutMs, getMaxTokens, getModel, getTemperature, getTopP

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getProviderName, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

VLLMClient

public VLLMClient(String model)

VLLMClient

public VLLMClient(String model,
 float temperature)

VLLMClient

public VLLMClient(String model,
 float temperature,
 Integer maxTokens)

VLLMClient

public VLLMClient(String model,
 float temperature,
 Float topP,
 Integer maxTokens)

VLLMClient

public VLLMClient(String baseUrl,
 String model,
 float temperature,
 Integer maxTokens,
 String apiKey)

VLLMClient

public VLLMClient(String baseUrl,
 String model,
 float temperature,
 Float topP,
 Integer maxTokens,
 String apiKey)