Package com.tnsai.rag

Class IngestionFingerprint

java.lang.Object
com.tnsai.rag.IngestionFingerprint
public final class IngestionFingerprint extends Object
Stable SHA-256 of the ingestion pipeline that produced one indexed file.

Content bytes alone are not enough to decide whether a file can stay indexed. A parser, chunker, selector, or KnowledgeUnit upgrade must rebuild the same bytes so retrieval cannot serve chunks from a previous pipeline.

KnowledgeUnit.AUTO is omitted from the payload, so declaring a unit is by itself never a reason an undeclared source re-ingests: a non-AUTO unit is a field of its own rather than a CHUNKER_VERSION bump, so only sources that set the member are invalidated by this member.

ChunkContext.NONE is omitted the same way: opting into contextual retrieval is a trailing field of its own, appended after unit when both are set. Field order is part of the hash.

That is a claim about these members alone, and it is the only claim that holds. It is not a promise that a given source keeps a hash it had in some earlier release: any other pipeline change — a parser, a selector, or a CHUNKER_VERSION bump — re-ingests every source regardless of what unit or chunkContext says. Pinning the promise to a version number is what makes it false the first time something else in the pipeline moves.

Since:
0.14.0

  • Field Details

  • Method Details

    • compute

      public static String compute(String sourceName, String sourcePath, String fileIdentity, String contentHash, String declaredFormat, String detectedFormat, String extractorId, String extractorVersion, List<String> include, List<String> exclude, int maxDepth, boolean followSymlinks)
      Computes the pipeline fingerprint for one admitted file.
      Parameters:
      sourceName - declared knowledge-source name
      sourcePath - declared source path
      fileIdentity - normalized file identity inside the source
      contentHash - SHA-256 of the admitted source bytes
      declaredFormat - declared DocumentFormat
      detectedFormat - effective concrete format after detection
      extractorId - extractor implementation class name
      extractorVersion - extractor ContentExtractor.version()
      include - declared include selectors
      exclude - declared exclude selectors
      maxDepth - admitted directory walk depth
      followSymlinks - whether the walk follows symbolic links
      Returns:
      lowercase hex SHA-256

    • compute

      public static String compute(String sourceName, String sourcePath, String fileIdentity, String contentHash, String declaredFormat, String detectedFormat, String extractorId, String extractorVersion, List<String> include, List<String> exclude, KnowledgeUnit unit, int maxDepth, boolean followSymlinks)
      Computes the pipeline fingerprint including a declared slicing unit.

      KnowledgeUnit.AUTO is omitted from the canonical payload, so this overload and the twelve-argument form produce the same hash for an AUTO source. Equality is between these two overloads in the same build — not with any particular released version.

      Since:
      0.15.0

    • compute

      public static String compute(String sourceName, String sourcePath, String fileIdentity, String contentHash, String declaredFormat, String detectedFormat, String extractorId, String extractorVersion, List<String> include, List<String> exclude, int maxDepth, boolean followSymlinks, ChunkContext chunkContext, String chunkContextTemplate, String chunkContextModel)
      Computes the pipeline fingerprint including chunk-context identity.

      ChunkContext.NONE omits the extra fields so a source that does not opt in stays byte-identical to the twelve-argument form in the same build.

      Since:
      0.15.0

    • compute

      public static String compute(String sourceName, String sourcePath, String fileIdentity, String contentHash, String declaredFormat, String detectedFormat, String extractorId, String extractorVersion, List<String> include, List<String> exclude, KnowledgeUnit unit, int maxDepth, boolean followSymlinks, ChunkContext chunkContext, String chunkContextTemplate, String chunkContextModel)
      Computes the pipeline fingerprint including unit and chunk-context identity.

      Conditional trailing fields keep a pinned order: unit then chunkContext/chunkContextTemplate/ chunkContextModel. Defaults are omitted so an undeclared source matches the twelve-argument form in the same build.

      Since:
      0.15.0

    • compute

      public static String compute(String sourceName, String sourcePath, String fileIdentity, String contentHash, String declaredFormat, String detectedFormat, String extractorId, String extractorVersion, List<String> include, List<String> exclude, KnowledgeUnit unit, int maxDepth, boolean followSymlinks, ChunkContext chunkContext, String chunkContextTemplate, String chunkContextModel, String select)
      Computes the pipeline fingerprint including the source's select.

      A blank select is omitted, so a source that selects nothing matches the sixteen-argument form in the same build. A non-blank one is the last field, after the chunk-context fields.

      Since:
      0.18.0