Package com.tnsai.quality.security.moderation

Class PatternBasedModerator

java.lang.Object
com.tnsai.quality.security.moderation.PatternBasedModerator
All Implemented Interfaces:
ContentModerator
public class PatternBasedModerator extends Object implements ContentModerator
Pattern-based content moderator using regex matching.

Provides fast, offline content moderation based on configurable regex patterns. Suitable for filtering known harmful patterns without external API dependencies.

Default Patterns

The default configuration includes patterns for:

  • Common profanity and slurs
  • Violent threats
  • Self-harm references
  • Spam indicators
  • PII patterns (email, phone, SSN)

Usage Example


 // Default patterns
 ContentModerator moderator = PatternBasedModerator.withDefaults();

 // Custom configuration
 ContentModerator custom = PatternBasedModerator.builder()
     .addPattern(Category.TOXIC, "badword1|badword2", 0.9)
     .addPattern(Category.SPAM, "(buy now|free offer)", 0.7)
     .blockThreshold(0.8)
     .warnThreshold(0.5)
     .build();

 // Check content
 ModerationResult result = moderator.checkInput(userMessage);
 if (result.isBlocked()) {
     logger.warn("Blocked content: {}", result.getReason());
     return result.getSafeResponse();
 }

  • Method Details

    • withDefaults

      public static PatternBasedModerator withDefaults()
      Creates a moderator with default patterns.
      Returns:
      default moderator

    • strict

      public static PatternBasedModerator strict()
      Creates a strict moderator with lower thresholds.
      Returns:
      strict moderator

    • builder

      public static PatternBasedModerator.Builder builder()
      Creates a moderator builder.
      Returns:
      new builder

    • checkInput

      public ModerationResult checkInput(String userInput)
      Description copied from interface: ContentModerator
      Checks user input for policy violations.

      Called before processing user messages. Should detect:

      • Harmful or offensive content
      • Prompt injection attempts
      • PII that shouldn't be processed
      • Spam or off-topic content
      Specified by:
      checkInput in interface ContentModerator
      Parameters:
      userInput - the user's input message
      Returns:
      moderation result

    • checkOutput

      public ModerationResult checkOutput(String modelOutput)
      Description copied from interface: ContentModerator
      Checks model output for policy violations.

      Called before returning responses to users. Should detect:

      • Harmful or offensive content generated by the model
      • Hallucinated harmful instructions
      • PII leakage from training data
      • Content that bypassed input filtering
      Specified by:
      checkOutput in interface ContentModerator
      Parameters:
      modelOutput - the model's response
      Returns:
      moderation result

    • checkToolResult

      public ModerationResult checkToolResult(String toolName, String result)
      Description copied from interface: ContentModerator
      Checks tool execution results for policy violations.

      Called after tool execution, before using results. Should detect:

      • Sensitive data in tool results
      • Harmful content from external sources
      • Unexpected or dangerous outputs
      Specified by:
      checkToolResult in interface ContentModerator
      Parameters:
      toolName - the name of the tool
      result - the tool's output
      Returns:
      moderation result

    • getName

      public String getName()
      Description copied from interface: ContentModerator
      Gets the name of this moderator.
      Specified by:
      getName in interface ContentModerator
      Returns:
      moderator name