Package com.tnsai.agents.validation.validators

Class LLMCapabilityValidator

java.lang.Object
com.tnsai.agents.validation.validators.LLMCapabilityValidator
All Implemented Interfaces:
Validator
public final class LLMCapabilityValidator extends Object implements Validator
Two-headed LLM capability validator.
  • AGENT-V003 (function calling) — fires when the agent will pass tools to the LLM (external tools registered, OR a role declares an ActionType.LLM action) but LLMCapabilities.supportsFunctionCalling() reports false.
  • AGENT-V004 (declarative capability requirements, issue #238) — for each role that declares @RoleSpec(requires = {LLMCapability.X, ...}), fires once per requested capability the LLM reports as unsupported. Lets consumers express "this role needs streaming / structured output / vision" at design time and catch the mismatch before first chat instead of at first streamChatWithTools/chatWithFormat/multimodal call.

The two checks share an LLM-introspection prologue (one llm.getCapabilities() call) but emit independent issues; a single agent can surface both at once if both signals fire.

AGENT-V003 — function calling

The agent passes tools to the LLM when one of two signals is true:

  • External tools are registered on the builder (toolPojos / dynamicTools / builtInTools) — surfaced via ValidationContext.hasExternalTools()
  • Any role has at least one action with type ActionType.LLM — LLM-as-dispatcher picks among the role's own actions

If neither signal fires, the LLM is used purely for chat / text generation and function-calling support is irrelevant. If at least one signal fires AND the LLM reports it does not support function calling, the agent will throw LLMCapabilityException on the first chat that wants to call a tool — typically minutes or hours into production.

Severity

Emitted as Severity.WARNING, not ERROR. The issue table in #85 specifies ERROR but the codebase convention (see LlmPresenceValidator for AGENT-V002) is to soft-launch new validators as warnings: a model whose getCapabilities() reports no function-calling may still work (model registry incomplete, custom provider with conservative defaults, etc.), and warning-level keeps existing builds compiling. Switch to ERROR in a follow-up PR once consumers have had time to either fix the underlying mismatch or suppress with .relaxValidation("AGENT-V003").

Conservative skip rules

  • No LLM configured → AGENT-V002 owns that case; this validator silent
  • LLMClient.getCapabilities() returns null (some mocks / test doubles don't implement it) → silent. Cannot distinguish "doesn't support FC" from "doesn't know yet"
  • Capabilities reports FC supported → silent regardless of model registry coverage

AGENT-V004 — declarative capability requirements

Roles annotate their LLM needs via @RoleSpec(requires = {LLMCapability.STREAMING, ...}). The validator iterates ValidationContext.roles(), reads each role's RoleSpec.requires(), and emits a separate AGENT-V004 WARNING per capability the LLM does not support. Roles without @RoleSpec or with an empty requires array contribute no signal — keeps the check opt-in by declaration.

Conservative skip rules (shared)

  • No LLM configured → AGENT-V002 owns that case; this validator silent
  • LLMClient.getCapabilities() returns null or throws → silent. Cannot distinguish "doesn't support FC" from "doesn't know yet"
  • Role's @RoleSpec.requires() array empty → V004 silent for that role regardless of LLM
Since:
0.8.4 (issue #85 phase 1); V004 added 0.8.5 (issue #238)

  • Constructor Details

    • LLMCapabilityValidator

      public LLMCapabilityValidator()

  • Method Details

    • id

      public String id()
      Description copied from interface: Validator
      Short, stable id for logging and test assertions (e.g. "llm-presence").
      Specified by:
      id in interface Validator

    • validate

      public List<ValidationIssue> validate(ValidationContext ctx)
      Description copied from interface: Validator
      Inspect the agent configuration and return any issues detected. Returning an empty list means "no problems found by this validator".
      Specified by:
      validate in interface Validator
      Parameters:
      ctx - frozen view of the agent under construction
      Returns:
      zero-or-more issues; never null