中文 ｜ English

Project Source Code

Sora2 Text-to-Video / Image-to-Video Dify Plugin

📖 Project Overview

This is a comprehensive Dify plugin based on JXINCM Sora-2 API that supports both text-to-video and image-to-video generation modes. Generate high-quality videos from text descriptions or create animated videos from image URLs, with real-time progress tracking. The plugin offers rich features including landscape/portrait orientation, watermark control, and multiple model selection.

✨ Key Features

• 🎬 Dual Mode Video Generation: Support both text-to-video and image-to-video modes
• 📸 Multi-Image Support: Image-to-video mode supports multiple image URLs
• 🔄 Smart Mode Switching: Automatically switch generation mode based on image URL input
• 📐 Multiple Orientations: Support landscape and portrait video formats
• 🎯 High-Quality Output: Powered by JXINCM Sora-2 and Sora-2-Pro models
• 🔗 Complete Result Return: Returns video URL, thumbnail, and GIF preview
• 🔄 Real-time Progress Tracking: Display full process status from queued to completed
• 🛡️ Comprehensive Error Handling: User-friendly error messages and solutions
• 🌐 Bilingual Support: Supports both English and Chinese interface

🏗️ Project Architecture

jxincm_sora2/
├── manifest.yaml              # Plugin manifest file
├── main.py                   # Plugin entry point
├── requirements.txt          # Python dependencies
├── README.md                # English documentation
├── README_CN.md             # Chinese documentation
├── PRIVACY.md               # Privacy policy
├── icon.svg                 # Plugin icon
├── provider/                # Service provider configuration
│   ├── jxincm.yaml          # JXINCM provider config
│   └── jxincm_provider.py   # Provider implementation
└── tools/                   # Tool implementation
    ├── text2video.yaml     # Video generation tool config
    └── text2video.py       # Video generation tool implementation

🚀 Quick Start

1. Get JXINCM API Key

1. Visit JXINCM Official Website
2. Register and login to your account
3. Get your API Key

2. Install Dependencies

pip install -r requirements.txt

3. Install Plugin in Dify

1. Upload the plugin folder to Dify plugin directory
2. Enable the plugin in Dify management interface
3. Configure JXINCM API Key

🔧 Usage

Mode 1: Text-to-Video

1. Add "Text to Video" tool in Dify workflow
2. Configure JXINCM API Key in plugin settings
3. Input video description prompt
4. Select video orientation:
   • portrait: Vertical (suitable for mobile short videos)
   • landscape: Horizontal (suitable for widescreen playback)
5. Select video size: large (high quality)
6. Select model:
   • sora-2: Standard quality model
   • sora-2-pro: High quality model
7. Set video duration: Fixed at 15 seconds
8. Choose whether to add watermark
9. Choose whether to make it private
10. Leave image URL parameter empty
11. Run the tool to generate video

Example:

Prompt: "A golden retriever playing in a sunny park with children running around"
Orientation: portrait
Size: large
Model: sora-2
Duration: 15 seconds
Watermark: No
Private: Yes
Image URLs: (leave empty)
→ Generate text-to-video

Mode 2: Image-to-Video

1. Add "Text to Video" tool in Dify workflow
2. Configure JXINCM API Key in plugin settings
3. Input animation description prompt
4. Input image URLs (supports the following formats):
   • Single URL: https://example.com/image.jpg
   • Multiple URLs (comma-separated): https://example.com/img1.jpg, https://example.com/img2.jpg
   • Multiple URLs (newline-separated):

     https://example.com/img1.jpg
     https://example.com/img2.jpg
     

5. Configure other parameters (orientation, model, etc.)
6. Run the tool to generate animated video

Example:

Prompt: "make animate"
Orientation: portrait
Image URLs: https://filesystem.site/cdn/20250612/VfgB5ubjInVt8sG6rzMppxnu7gEfde.png
→ Generate image-to-video

Prompt Suggestions

For best video generation results, we recommend:

Text-to-Video Prompts:

• Detailed Description: Provide specific information about scenes, actions, camera movements, lighting
• Clear Expression: Use concise and clear language
• Camera Direction: Specify camera movements like "slow push-in", "orbit shot"

Example:

A golden retriever playing in a sunny park with children running around,
camera slowly pushing in and orbiting, bright natural lighting

Image-to-Video Prompts:

• Action Description: Describe how elements in the image should animate
• Keep It Simple: Usually short action commands work best
• Stay Consistent: Prompt should relate to the image content

Example:

make animate
bring the scene to life
add natural dynamic effects

⚙️ Technical Implementation

Core Workflow

Video Generation Flow:

1. Parameter Parsing: Parse input parameters, determine generation mode (text or image)
2. Task Creation: Submit video generation request to JXINCM API
3. Status Polling: Monitor task status via periodic API calls
4. Progress Tracking: Display generation progress (queued → processing → completed)
5. Result Extraction: Auto-extract video URL, thumbnail, GIF preview
6. Result Return: Return complete video information

API Call Pattern

# 1. Create video task (using JSON format)
POST https://api.jxincm.cn/v1/video/create
Headers:
  - Authorization: Bearer {api_key}
  - Content-Type: application/json
Body:
  {
    "prompt": "description",
    "model": "sora-2",
    "orientation": "portrait",
    "size": "large",
    "duration": 15,
    "watermark": false,
    "private": true,
    "images": []  # Empty for text-to-video, contains URLs for image-to-video
  }

# 2. Poll for status
GET https://api.jxincm.cn/v1/video/query?id={video_id}
Response: {
  "id": "sora-2:task_xxx",
  "status": "completed",
  "progress": 100,
  "detail": {
    "url": "https://.../video.mp4",
    "thumbnail": "https://.../thumbnail.jpg",
    "gif": "https://.../preview.gif"
  }
}

Key Implementation Details

Task Creation:

# Build request body
payload = {
    "prompt": prompt,
    "model": model,
    "orientation": orientation,
    "size": size,
    "duration": 15,
    "watermark": watermark,
    "private": private,
    "images": image_urls if image_urls else []  # Auto-switch mode
}

Progress Polling:

while attempt < max_attempts:
    response = conn.request("GET", f"/v1/video/query?id={video_id}")
    status = data.get("status")
    progress = data.get("progress")

    if status == "completed":
        # Extract video URL, thumbnail, GIF
        video_url = detail.get("url")
        thumbnail_url = detail.get("thumbnail")
        gif_url = detail.get("gif")

🔍 Troubleshooting

Common Issues

1. Invalid API Key

   • Check if API Key format is correct
   • Confirm API Key is valid and has sufficient quota
   • Verify JXINCM platform account status

2. Generation Timeout

   • Check network connection stability
   • Video generation typically takes 2-10 minutes
   • Try simplifying prompt description
   • Retry later if server is busy

3. Invalid Image URL

   • Ensure image URL is publicly accessible
   • Check if image format is supported (JPG, PNG, WebP, etc.)
   • Verify URL format is correct

4. Prompt Rejected

   • Avoid sensitive or inappropriate content
   • Use more general descriptions
   • Follow content policy guidelines

Error Codes

• 401: Invalid or unauthorized API Key
• 429: API call rate limit exceeded
• 500: Internal server error
• Timeout: Request timeout (network or server issue)

📊 Performance Metrics

• Request Timeout: 30 seconds (API calls), 10 minutes (total polling)
• Average Generation Time: 2-10 minutes (varies by complexity and queue length)
• Video Duration: 15 seconds (fixed)
• Supported Format: MP4
• Video Orientation:
  • portrait (vertical)
  • landscape (horizontal)
• Video Size: large (high quality)
• Model Selection:
  • sora-2 (standard quality)
  • sora-2-pro (high quality)
• Generation Modes:
  • Text-to-video (no image URLs)
  • Image-to-video (with image URLs)
• Output Content:
  • Video URL (main playback link)
  • Thumbnail (preview image)
  • GIF preview (animated preview)
• Polling Interval: 5 seconds (real-time progress updates)

🔒 Privacy & Security

Please refer to PRIVACY.md for detailed information about data handling and privacy policy.

Key points:

• No local storage of prompts or videos
• Temporary processing only during generation
• API Key stored securely in Dify environment
• Data processed by JXINCM according to their privacy policy
• Image URLs provided by users, plugin does not store or cache them

📋 Development Standards

This plugin follows Dify plugin development best practices:

• ✅ Generator response processing
• ✅ Real-time progress tracking
• ✅ Automatic URL extraction
• ✅ Complete error handling mechanism
• ✅ Bilingual support (English/Chinese)
• ✅ Standard JXINCM API integration
• ✅ Dual mode support (text/image)

🤝 Contributing

Welcome to submit Issues and Pull Requests to improve this plugin!

📄 License

This project is licensed under the MIT License.

🔗 Related Links

• JXINCM Official Website
• Dify Official Documentation
• Plugin GitHub Repository

📦 Release Notes

0.0.4 (2025-11-01) 🆕

• API Migration: Migrated from 302.AI to JXINCM API
• Dual Mode Support: Support both text-to-video and image-to-video modes
• Multi-Image Support: Image-to-video mode supports multiple image URLs
• Parameter Optimization: Simplified parameter configuration, fixed 15-second duration
• Enhanced Output: Returns video URL, thumbnail, and GIF preview
• Progress Display: Complete task creation and progress tracking information
• Smart Mode: Auto-switch generation mode based on image URL parameter

0.0.1 (2025-10-02)

• Initial release with text-to-video generation