Package com.tnsai.intelligence.rag.binding

Interface SourceLoader

All Known Implementing Classes:
HttpKnowledgeSourceLoader, LocalFileSourceLoader
public interface SourceLoader
Loads documents for a single KnowledgeSourceConfig declaration into the per-Role RAG memory store at binding-init time.

One SourceLoader implementation per Knowledge.Type. TnsAI bundles FILE and HTTP/HTTPS URL loaders. DATABASE remains an application-provided SPI extension. No loader serves MEMORY or GRAPH, which load nothing; discovering one fails.

Note what this seam can and cannot express. load(com.tnsai.rag.KnowledgeSourceConfig) runs once, when a Role's binding is built, and returns the documents it found; the framework indexes those and searches the index thereafter. A corpus that must be queried per request — a remote vector database, a search API — therefore cannot be a SourceLoader at all, no matter what the annotation offers to configure it with. Those are served by com.tnsai.rag.RetrievalEngineProvider, which is invoked per retrieval rather than per binding.

Loaders are looked up by handles() at binding-init time. A declared source whose type has no registered loader is skipped with a warning, and the binding still builds from whichever sources did load; only when every enabled source is unloadable does binding fail. The line is drawn there so that installing one optional loader at a time stays possible — but it does mean a missing loader costs part of the corpus without failing anything, so read the warning.

Since:
0.5.4 (issue #172, Phase 1)

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static final record 
    SourceLoader.LoadedDocument
    One document loaded by a SourceLoader — the unit ingested into the vector store.

  • Method Summary

    Modifier and Type
    Method
    Description
    com.tnsai.annotations.Knowledge.Type
    handles()
    The Knowledge.Type this loader handles.
    List<SourceLoader.LoadedDocument>
    load(com.tnsai.rag.KnowledgeSourceConfig source)
    Loads documents from the given source declaration.
    default Map<String,String>
    metadata(com.tnsai.rag.KnowledgeSourceConfig source, SourceLoader.LoadedDocument document)
    Returns optional metadata for one loaded document.

  • Method Details

    • handles

      com.tnsai.annotations.Knowledge.Type handles()
      The Knowledge.Type this loader handles. RoleRagBinding routes each declared source to the loader whose handles() matches.
      Returns:
      the supported type

    • load

      List<SourceLoader.LoadedDocument> load(com.tnsai.rag.KnowledgeSourceConfig source)
      Loads documents from the given source declaration.

      Each returned SourceLoader.LoadedDocument carries an id (used as the entry id in the vector store), the textual content, and an optional source label for retrieval-time attribution (e.g. the file path, URL, or DB row id).

      The return type is a list because a loader decides the unit of retrieval: it may split one source into as many documents as it likes, and everything downstream — topK, minScore, dedupeThreshold, contextWindow — is defined over whatever it returns. The bundled LocalFileSourceLoader keeps small files intact and emits stable, format-aware chunks for larger files (TAN-5327).

      Successfully loaded empty sources return an empty list. The bundled FILE and HTTP loaders throw RetrievalException with reason SOURCE_LOADING when a source cannot be read; they never publish a partial corpus as an empty success. Invalid source configuration and content surface as CONFIGURATION at the binding boundary.

      The returned list defines ingest order. Ordered consumers observe documents in exactly this order, so implementations must return a deterministic list for a stable source snapshot.

      Parameters:
      source - the normalized knowledge-source declaration
      Returns:
      loaded documents in deterministic ingest order, or empty when the source was successfully read but contained no documents

    • metadata

      default Map<String,String> metadata(com.tnsai.rag.KnowledgeSourceConfig source, SourceLoader.LoadedDocument document)
      Returns optional metadata for one loaded document.

      The default implementation lets loaders opt out of document metadata. Hierarchical loaders can declare relationships with HierarchyMetadata.PARENT_ID; binding-owned metadata such as the source fence is added by RoleRagBinding. Temporal loaders provide TemporalMetadata.TIMESTAMP as a canonical ISO-8601 instant such as 2026-07-26T00:00:00Z.

      Parameters:
      source - the normalized declaration that produced the document
      document - one document returned from load(KnowledgeSourceConfig)
      Returns:
      document metadata, or an empty map
      Since:
      0.13.0