Package com.tnsai.llm.providers

Class ReplicateClient

java.lang.Object

com.tnsai.llm.providers.AbstractLLMClient

com.tnsai.llm.providers.ReplicateClient

All Implemented Interfaces:

com.tnsai.llm.LLMClient

public class ReplicateClient
extends AbstractLLMClient

Replicate LLM client.

Replicate is a model marketplace; users run any community-deployed model through a single HTTP API at https://api.replicate.com/v1. Unlike every other provider in this module, Replicate is not OpenAI-shaped — the request takes an arbitrary input JSON object and the response is delivered through a predict/poll lifecycle:

1. POST a prediction → returns {id, status, urls.get}
2. Poll urls.get until status is succeeded, failed, or canceled
3. Read the output field (model-dependent shape)

v1 scope (this client)

This v1 deliberately targets the common chat-model case on Replicate (Llama, DeepSeek, Mixtral, etc.) with a minimal feature surface:

• Model identifier: "owner/name" (e.g. "meta/meta-llama-3-70b-instruct"). The latest published version is auto-resolved by Replicate via the POST /v1/models/{owner}/{name}/predictions endpoint — no need to pin a version hash. The "owner/name:version" form is also accepted (the :version suffix is preserved so callers who pinned can still pin).
• Input shape: assumes {"prompt": "..."} — the de facto convention for chat models on Replicate. System prompt + history + user message are concatenated into a single prompt string with role markers.
• Output shape: handles both string (joined text) and string[] (token list — concatenated). Other shapes (image URLs, structured JSON) are not the target use case for the LLMClient SPI.
• Polling: exponential backoff starting at 500ms, doubling up to 30s, with a 10-minute overall timeout.
• Streaming: not implemented; streamChat returns the final concatenated output as a single chunk. Replicate has an SSE streaming endpoint for some models, but it's model-by-model and not a structural fit for this v1.
• Tool calling: not supported. Replicate models don't expose an OpenAI-style tools API; tools are silently ignored.

Configuration via environment variable:

• REPLICATE_API_KEY — Required. Replicate's own docs call this REPLICATE_API_TOKEN; this module uses the _API_KEY suffix to match the framework convention enforced by ProviderEnvVarConsistencyTest. The token value is the same.

See Also:

• Replicate HTTP API documentation

Constructor Summary

Constructors

Constructor	Description
ReplicateClient(String model)
ReplicateClient(String model, float temperature)
ReplicateClient(String model, float temperature, Float topP, Integer maxTokens)
ReplicateClient(String model, float temperature, Integer maxTokens)
ReplicateClient(String baseUrl, String model, float temperature, Float topP, Integer maxTokens, String apiKey)
ReplicateClient(String baseUrl, String model, float temperature, Integer maxTokens, String apiKey)

Method Summary

Modifier and Type	Method	Description
com.tnsai.llm.ChatResponse	chat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)
Stream<String>	streamChat(String message, Optional<String> systemPrompt, Optional<List<Map<String,Object>>> history, Optional<List<Map<String,Object>>> tools)

Methods inherited from class com.tnsai.llm.providers.AbstractLLMClient

getBaseUrl, getCallTimeoutMs, getMaxTokens, getModel, getTemperature, getTopP

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.tnsai.llm.LLMClient

chat, chat, chat, chat, chat, getCapabilities, getProviderName, listModels, streamChat, streamChat, streamChatWithHandler, streamChatWithHandler, streamChatWithSpec, streamChatWithSpec

Constructor Details

ReplicateClient

public ReplicateClient(String model)

ReplicateClient

public ReplicateClient(String model,
 float temperature)

ReplicateClient

public ReplicateClient(String model,
 float temperature,
 Integer maxTokens)

ReplicateClient

public ReplicateClient(String model,
 float temperature,
 Float topP,
 Integer maxTokens)

ReplicateClient

public ReplicateClient(String baseUrl,
 String model,
 float temperature,
 Integer maxTokens,
 String apiKey)

ReplicateClient

public ReplicateClient(String baseUrl,
 String model,
 float temperature,
 Float topP,
 Integer maxTokens,
 String apiKey)

Method Details

chat

public com.tnsai.llm.ChatResponse chat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)

streamChat

public Stream<String> streamChat(String message,
 Optional<String> systemPrompt,
 Optional<List<Map<String,Object>>> history,
 Optional<List<Map<String,Object>>> tools)