Package com.tnsai.quality.security.moderation

Class ModerationResult

java.lang.Object
com.tnsai.quality.security.moderation.ModerationResult
public final class ModerationResult extends Object
Result of content moderation check.

Contains moderation status, category flags, and confidence scores.

Moderation Categories

CategoryDescription
HATEHate speech and discrimination
HARASSMENTPersonal attacks, bullying
VIOLENCEViolent content or threats
SELF_HARMSelf-harm related content
SEXUALExplicit sexual content
ILLEGALIllegal activities
TOXICGenerally toxic content
SPAMSpam or promotional content

Actions

  • ALLOW: Content is safe, proceed normally
  • WARN: Content is suspicious, log and proceed with caution
  • BLOCK: Content violates policies, reject request

  • Method Details

    • allowed

      public static ModerationResult allowed()
      Returns an allowed result.
      Returns:
      allowed result

    • warn

      public static ModerationResult warn(String reason, Map<ModerationResult.Category,Double> categoryScores)
      Returns a warning result.
      Parameters:
      reason - reason for warning
      categoryScores - category confidence scores
      Returns:
      warning result

    • block

      public static ModerationResult block(String reason, Map<ModerationResult.Category,Double> categoryScores, List<String> flaggedContent)
      Returns a blocked result.
      Parameters:
      reason - reason for blocking
      categoryScores - category confidence scores
      flaggedContent - specific content that was flagged
      Returns:
      blocked result

    • builder

      public static ModerationResult.Builder builder()
      Creates a builder for complex results.
      Returns:
      new builder

    • getAction

      public ModerationResult.Action getAction()
      Gets the recommended action.
      Returns:
      action to take

    • isAllowed

      public boolean isAllowed()
      Checks if content is allowed.
      Returns:
      true if allowed

    • isBlocked

      public boolean isBlocked()
      Checks if content is blocked.
      Returns:
      true if blocked

    • hasWarning

      public boolean hasWarning()
      Checks if content has warnings.
      Returns:
      true if warned

    • getCategoryScores

      public Map<ModerationResult.Category,Double> getCategoryScores()
      Gets the category confidence scores.
      Returns:
      map of category to score

    • getScore

      public double getScore(ModerationResult.Category category)
      Gets the score for a specific category.
      Parameters:
      category - category to check
      Returns:
      score (0.0 - 1.0) or 0.0 if not flagged

    • isFlagged

      public boolean isFlagged(ModerationResult.Category category)
      Checks if a category was flagged.
      Parameters:
      category - category to check
      Returns:
      true if flagged

    • getFlaggedCategories

      public Set<ModerationResult.Category> getFlaggedCategories()
      Gets all flagged categories.
      Returns:
      set of flagged categories

    • getReason

      public String getReason()
      Gets the reason for the action.
      Returns:
      reason or null if allowed

    • getFlaggedContent

      public List<String> getFlaggedContent()
      Gets the flagged content snippets.
      Returns:
      list of flagged content

    • getSafeResponse

      public String getSafeResponse()
      Gets a safe response to return when blocked.
      Returns:
      safe response message

    • getDetailedSafeResponse

      public String getDetailedSafeResponse()
      Gets a detailed safe response.
      Returns:
      detailed safe response

    • toString

      public String toString()
      Overrides:
      toString in class Object