Package com.tnsai.intelligence.rag.binding

Class FileSourceReconciler

java.lang.Object

com.tnsai.intelligence.rag.binding.FileSourceReconciler

public final class FileSourceReconciler
extends Object

Reloads a FILE knowledge source against the manifest of its previous load and reports which files and chunks changed.

A reconcile walks the source with the same code as a cold load, so caps, selectors, ignore files, ordering and outcome checks are identical. The prior manifest only replaces the per-file read: a file whose stat key is unchanged is not read, and a file whose bytes and pipeline fingerprint are unchanged is not extracted, chunked or enriched again. Reconciling against FileManifest.empty() is a cold load. On a file system without an inode change time, a change that keeps a file's size, modification time and file key is not seen until its bytes are read again; see FileManifest.

The reconciler holds no mutable state; callers keep the returned manifest and pass it to the next call.

Since:

0.18.7

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	FileSourceReconciler.Reconciled	Outcome of one reconcile: the load result, the manifest for the next reconcile, and the diff against the prior manifest.

Constructor Summary

Constructors

Constructor	Description
FileSourceReconciler(FileIngestionService ingestionService)	Creates a reconciler that loads through ingestionService and keeps the extracted document of each ingested file in its manifests.

Method Summary

Modifier and Type	Method	Description
FileSourceReconciler.Reconciled	reconcile(com.tnsai.rag.KnowledgeSourceConfig source, FileManifest prior, Consumer<FileIngestionService.IngestionProgress> observer)	Loads source with prior as the reuse oracle.
FileSourceReconciler	withoutOriginals()	Returns a reconciler on the same pipeline whose manifests keep each file's chunks but not the extracted document text, for callers that index chunks only.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FileSourceReconciler

public FileSourceReconciler(FileIngestionService ingestionService)

Creates a reconciler that loads through ingestionService and keeps the extracted document of each ingested file in its manifests.

Parameters:

ingestionService - the FILE pipeline and its admission caps

Method Details

withoutOriginals

public FileSourceReconciler withoutOriginals()

Returns a reconciler on the same pipeline whose manifests keep each file's chunks but not the extracted document text, for callers that index chunks only.

Returns:

a reconciler that keeps no extracted documents

reconcile

public FileSourceReconciler.Reconciled reconcile(com.tnsai.rag.KnowledgeSourceConfig source,
 FileManifest prior,
 Consumer<FileIngestionService.IngestionProgress> observer)

Loads source with prior as the reuse oracle.

A file whose read fails keeps its prior entry and is reported in FileSourceReconciler.Reconciled.failedPaths(), neither changed nor removed. A load that discards what it collected, such as a failed directory walk, is aborted: it returns prior unchanged with an empty diff.

Parameters:

source - canonical FILE-source configuration

prior - manifest of the previous load, or FileManifest.empty()

observer - synchronous progress observer

Returns:

the load result, the next manifest and the diff against prior