Package com.tnsai.utils

Class TextSanitizer

java.lang.Object
com.tnsai.utils.TextSanitizer
public final class TextSanitizer extends Object
Cleans text payloads before they leave the process toward an external API.

Exists to defend against malformed Java Strings that JSON encoders and HTTP clients refuse to serialize. The canonical failure mode is a lone UTF-16 surrogate — the high or low half of a surrogate pair without its partner. These arise when user input is truncated mid-character (e.g. a prompt cut at a byte budget, a chat message copy-pasted from a broken source) and cause Jackson/OkHttp to throw MalformedInputException or silently drop the request depending on the provider.

Replacement strategy: each unpaired surrogate is swapped for REPLACEMENT_CHAR (U+FFFD), matching the JVM's own behaviour when it encounters invalid bytes during decode. Paired surrogates (valid non-BMP code points like emoji) are left intact.

Since:
0.3.0

  • Field Details

    • REPLACEMENT_CHAR

      public static final char REPLACEMENT_CHAR
      Unicode replacement character used for every lone surrogate removed.
      See Also:

  • Method Details

    • sanitizeLoneSurrogates

      public static String sanitizeLoneSurrogates(String input)
      Returns input with every unpaired UTF-16 surrogate replaced by REPLACEMENT_CHAR. Null input returns null; empty input returns the same reference. Inputs with no surrogate issues are returned unchanged (same reference) — the allocation is avoided on the hot path.
      Parameters:
      input - raw text that may contain unpaired surrogates
      Returns:
      sanitized text safe for JSON / UTF-8 serialization

    • containsLoneSurrogate

      public static boolean containsLoneSurrogate(String input)
      Fast check: does the string contain any unpaired surrogate? Lets callers short-circuit allocation when the input is already clean.
      Parameters:
      input - the text to scan (may be null)
      Returns:
      true if at least one unpaired high or low surrogate is present