Package com.tnsai.llm

Class ParallelLLM

java.lang.Object

com.tnsai.llm.ParallelLLM

public final class ParallelLLM
extends Object

Parallel LLM execution utilities. Provides fan-out, scatter-gather, and consensus patterns for LLM calls.

Example usage:

 // Fan-out to multiple LLMs
 List<LLMClient> llms = List.of(deepseek, qwen, gemini);
 List<String> responses = ParallelLLM.fanOut(prompt, llms);

 // Consensus from multiple LLMs
 String bestResponse = ParallelLLM.consensus(prompt, llms, scorer);

 // Race - first response wins
 String fastest = ParallelLLM.race(prompt, llms, 30, TimeUnit.SECONDS);
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	ParallelLLM.LLMResult	Result of a parallel LLM call.
static final record	ParallelLLM.ParallelResults	Aggregated results from multiple LLMs.

Method Summary

Modifier and Type	Method	Description
static Optional<ParallelLLM.LLMResult>	consensus(String prompt, List<LLMClient> llms, Function<String,Double> scorer)	Gets responses from multiple LLMs and selects the best one using a scoring function.
static Optional<String>	ensemble(String prompt, List<LLMClient> llms, LLMClient aggregator)	Gets responses from multiple LLMs and combines them using an aggregator LLM.
static ParallelLLM.ParallelResults	fanOut(String prompt, List<LLMClient> llms)	Sends the same prompt to multiple LLMs in parallel and collects all responses.
static ParallelLLM.ParallelResults	fanOut(String prompt, List<LLMClient> llms, long timeout, TimeUnit unit)	Sends the same prompt to multiple LLMs in parallel with timeout.
static Optional<ParallelLLM.LLMResult>	majorityVote(String prompt, List<LLMClient> llms)	Gets responses from multiple LLMs and selects by majority voting.
static Optional<ParallelLLM.LLMResult>	race(String prompt, List<LLMClient> llms, long timeout, TimeUnit unit)	Sends prompt to multiple LLMs, returns the first successful response.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

fanOut

public static ParallelLLM.ParallelResults fanOut(String prompt,
 List<LLMClient> llms)

Sends the same prompt to multiple LLMs in parallel and collects all responses.

Parameters:

prompt - The prompt to send

llms - List of LLM clients

Returns:

Aggregated results from all LLMs

fanOut

public static ParallelLLM.ParallelResults fanOut(String prompt,
 List<LLMClient> llms,
 long timeout,
 TimeUnit unit)

Sends the same prompt to multiple LLMs in parallel with timeout.

Parameters:

prompt - The prompt to send

llms - List of LLM clients

timeout - Timeout value

unit - Timeout unit

Returns:

Aggregated results from all LLMs

race

public static Optional<ParallelLLM.LLMResult> race(String prompt,
 List<LLMClient> llms,
 long timeout,
 TimeUnit unit)

Sends prompt to multiple LLMs, returns the first successful response.

Parameters:

prompt - The prompt to send

llms - List of LLM clients

timeout - Timeout value

unit - Timeout unit

Returns:

First successful response, or empty if all fail/timeout

consensus

public static Optional<ParallelLLM.LLMResult> consensus(String prompt,
 List<LLMClient> llms,
 Function<String,Double> scorer)

Gets responses from multiple LLMs and selects the best one using a scoring function.

Parameters:

prompt - The prompt to send

llms - List of LLM clients

scorer - Function to score each response (higher is better)

Returns:

Best response based on scoring

majorityVote

public static Optional<ParallelLLM.LLMResult> majorityVote(String prompt,
 List<LLMClient> llms)

Gets responses from multiple LLMs and selects by majority voting. Uses similarity to find the most common response pattern.

Parameters:

prompt - The prompt to send

llms - List of LLM clients

Returns:

Most common response pattern

ensemble

public static Optional<String> ensemble(String prompt,
 List<LLMClient> llms,
 LLMClient aggregator)

Gets responses from multiple LLMs and combines them using an aggregator LLM.

Parameters:

prompt - The original prompt

llms - List of LLM clients for initial responses

aggregator - LLM client to synthesize final response

Returns:

Synthesized response