Package com.tnsai.rag

Class StructuralPath

java.lang.Object
com.tnsai.rag.StructuralPath
public final class StructuralPath extends Object
Builds and selects structural paths.

A path starts at a format root and adds one step per level. An object key is .name when it is an identifier ([A-Za-z_][A-Za-z0-9_]*) and ['name'] otherwise, with \\, \', \b, \f, \n, \r, \t and \uXXXX escapes; an array element is [n], counted from 0.

The roots the built-in extractors use, with the counts they start from:

  • JSON: $, as in $.products[0].name or $['a b'].
  • JSON Lines: $line[n] per record, from 1.
  • YAML: $document[n] per document, from 1.
  • Markdown: $frontmatter; $section[n] per heading, from 1 ($section[0] is the text before the first heading), with .heading, and .paragraph[l] and .code[l] at the line they start on.
  • XML: /name[k] per element, from 1 among same-named siblings, as in /catalog[1]/book[2], and /$text[k] for text.
  • HTML: $html.tag[k].
  • CSV and TSV: $row[n] per data row, from 1.
  • Plain text: $text. Source code: $source, and $symbol['name'] per symbol.
  • Office documents: $heading[n], $paragraph[n], $table[n].

A selector keeps the blocks at and below it: a path is selected when it equals the selector or continues it with ., [ or /. $.products selects $.products[4].name but not $.productsArchive.

Since:
0.18.0

  • Method Details

    • property

      public static String property(String name)
      The step for an object key.
      Parameters:
      name - the key
      Returns:
      .name, or ['name'] with escapes

    • index

      public static String index(int index)
      The step for an array element.
      Parameters:
      index - the element index
      Returns:
      [index]

    • selects

      public static boolean selects(String selector, String path)
      Whether selector selects path.
      Parameters:
      selector - the selector; blank selects every path
      path - a structural path
      Returns:
      true when the path equals the selector or continues it with ., [ or /

    • select

      public static ExtractedDocument select(ExtractedDocument document, String selector)
      Keeps the blocks selector selects. The kept document's content is their contents, one block per line run, in order.
      Parameters:
      document - an extracted document
      selector - the selector; blank keeps the document as it is
      Returns:
      the selected document
      Throws:
      IllegalArgumentException - when the selector selects no block

    • requireBlankForEmpty

      public static void requireBlankForEmpty(String selector, String documentId)
      Fails a non-blank selector over an empty document, which has no block to select. A blank selector keeps the empty document.
      Parameters:
      selector - the selector
      documentId - the empty document's id
      Throws:
      IllegalArgumentException - when the selector is not blank