Package com.tnsai.models.chat

package com.tnsai.models.chat
Multimodal content parts for LLM communication.

This package contains data structures for representing multimodal content (text and images) when communicating with Large Language Models that support vision capabilities.

Content Part Types

TypeDescriptionUse Case
TextPartText contentStandard text messages
ImagePartImage contentVision/multimodal requests

Multimodal Message Flow

 ┌──────────────────────────────────────────────────────────────┐
 │                  Multimodal Content Flow                      │
 ├──────────────────────────────────────────────────────────────┤
 │                                                               │
 │  User Input                                                   │
 │      │                                                        │
 │      ▼                                                        │
 │  ┌─────────────────────────────┐                             │
 │  │   List<ContentPart>         │                             │
 │  │  ┌─────────┐ ┌─────────┐   │                             │
 │  │  │TextPart │ │ImagePart│   │                             │
 │  │  └─────────┘ └─────────┘   │                             │
 │  └──────────────┬──────────────┘                             │
 │                 │                                             │
 │                 ▼                                             │
 │            Vision LLM                                         │
 │                 │                                             │
 │                 ▼                                             │
 │           Text Response                                       │
 │                                                               │
 └──────────────────────────────────────────────────────────────┘

Usage Example


 // Create multimodal content for vision model
 List<ContentPart> parts = List.of(
     new TextPart("What is in this image?"),
     ImagePart.fromBase64(base64EncodedImage, "image/jpeg")
 );

 // Send to vision-capable LLM (e.g., Gemini)
 String response = geminiClient.chat(parts);

Image Part Creation


 // From Base64 encoded image
 ImagePart image = ImagePart.fromBase64(base64Data, "image/png");

 // From URL
 ImagePart urlImage = ImagePart.fromUrl("https://example.com/image.jpg");

 // From file
 byte[] imageBytes = Files.readAllBytes(Path.of("photo.jpg"));
 String base64 = Base64.getEncoder().encodeToString(imageBytes);
 ImagePart fileImage = ImagePart.fromBase64(base64, "image/jpeg");

Supported by

  • GeminiClient (Gemini 1.5/2.0)
  • OpenAI GPT-4o Vision
  • Anthropic Claude 3.x Vision
See Also: