Package com.tnsai.tools.audio

Class SpeechToTextTools

java.lang.Object

com.tnsai.tools.audio.SpeechToTextTools

public class SpeechToTextTools
extends Object

Function-shape speech-to-text tools — non-OpenAI providers, complementing MediaTools.whisperTranscribe(String, String) which already covers OpenAI Whisper. Wraps three popular alternatives behind a uniform @Tool surface so an LLM can pick by latency / cost / feature set at call time:

Method	Backend	Auth	Strength
deepgramTranscribe(java.lang.String, java.lang.String, java.lang.String)	Deepgram Nova-2	DEEPGRAM_API_KEY	Fastest, cheapest, sync
assemblyaiTranscribe(java.lang.String, java.lang.String)	AssemblyAI Universal-2	ASSEMBLYAI_API_KEY	Best accuracy, async (polled)
replicateWhisper(java.lang.String, java.lang.String)	Replicate Whisper large-v3	REPLICATE_API_TOKEN	Same key as FLUX

All three return a uniform {provider, model, text, language?} JSON envelope so callers don't need per-provider parsers.

assemblyaiTranscribe(java.lang.String, java.lang.String) synchronously polls AssemblyAI's async job endpoint (upload → submit → poll → fetch). The poll loop has a 5-minute hard cap so the @Tool call returns deterministically; longer transcripts should batch-upload via AssemblyAI's webhook flow (out of scope for the @Tool surface, where the LLM expects a synchronous reply).

Implementation deliberately uses OkHttp + Jackson only — no vendor SDK is pulled in, matching the rest of tnsai-tools.

Cost notes (April 2026 list pricing — providers may change)

• Deepgram Nova-2: $0.0043 / minute (pay-as-you-go)
• AssemblyAI Universal-2: $0.27 / hour (~$0.0045 / minute)
• Replicate Whisper large-v3: ~$0.0007 / minute on Nvidia T4 (cheapest)

Since:

0.8.4

See Also:

• for the OpenAI Whisper / TTS path
• for the matching TTS side

Constructor Summary

Constructors

Constructor	Description
SpeechToTextTools()	Default no-arg ctor — the only one BuiltInTool.SPEECH_TO_TEXT_TOOLS ever calls.

Method Summary

Modifier and Type	Method	Description
String	assemblyaiTranscribe(String audioPath, String languageCode)
String	deepgramTranscribe(String audioPath, String language, String model)
String	replicateWhisper(String audioPath, String model)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SpeechToTextTools

public SpeechToTextTools()

Default no-arg ctor — the only one BuiltInTool.SPEECH_TO_TEXT_TOOLS ever calls.

Method Details

deepgramTranscribe

@Tool(name="deepgram_transcribe",
      description="Transcribe a local audio file using Deepgram Nova-2 (sync). Returns a JSON object with `text` (the full transcript) and `confidence`. {audioPath} accepts MP3 / WAV / M4A / OGG / FLAC, max 50 MB. {language} is an ISO language code (default `en`). {model} is `nova-2` (default), `nova-3`, etc. Requires DEEPGRAM_API_KEY.")
public String deepgramTranscribe(@ToolParam(description="Absolute path to the audio file (max 50 MB)")
 String audioPath,
 @ToolParam(description="ISO language code; default `en`",required=false,defaultValue="en")
 String language,
 @ToolParam(description="Deepgram model \u2014 `nova-2` (default), `nova-3`, `enhanced`, `base`",required=false,defaultValue="nova-2")
 String model)
                          throws IOException

Throws:

IOException

assemblyaiTranscribe

@Tool(name="assemblyai_transcribe",
      description="Transcribe a local audio file using AssemblyAI Universal-2. The method uploads the file, submits the transcript job, then polls until completion (5-minute timeout). Returns a JSON object with `text`, `language`, and `audio_duration` (seconds). {audioPath} accepts MP3 / WAV / M4A / OGG / FLAC, max 50 MB. {languageCode} optional ISO code (auto-detect if blank). Requires ASSEMBLYAI_API_KEY.")
public String assemblyaiTranscribe(@ToolParam(description="Absolute path to the audio file (max 50 MB)")
 String audioPath,
 @ToolParam(description="Optional ISO language code (e.g. `en`, `tr`); blank = auto-detect",required=false,defaultValue="")
 String languageCode)
                            throws IOException

Throws:

IOException

replicateWhisper

@Tool(name="replicate_whisper",
      description="Transcribe a local audio file using OpenAI Whisper hosted on Replicate. Cheaper than OpenAI\'s own Whisper endpoint. Returns a JSON object with `text` and `language`. {audioPath} accepts MP3 / WAV / M4A / OGG / FLAC, max 50 MB. {model} picks Whisper variant: `large-v3` (default, best), `large-v2`, `medium`, `small`. Requires REPLICATE_API_TOKEN (same key as FLUX image generation).")
public String replicateWhisper(@ToolParam(description="Absolute path to the audio file (max 50 MB)")
 String audioPath,
 @ToolParam(description="Whisper model \u2014 `large-v3` (best, default), `large-v2`, `medium`, `small`",required=false,defaultValue="large-v3")
 String model)
                        throws IOException

Throws:

IOException