Volcengine AI Plugin for Dify

Powerful Visual AI Capabilities for Dify

简体中文 | English

📖 Overview

The Volcengine AI plugin integrates Volcengine (火山引擎)'s Jimeng AI series models into Dify, providing professional-grade visual AI capabilities including text-to-image, image-to-image, text-to-video, and image-to-video generation.

✨ Features

7 Professional AI Visual Tools

🎨 Image Generation Tools

1. HighAesGeneralV30LZT2I - High Aesthetic General 3.0 Text-to-Image

   • Next-generation text-to-image model built on ByteDance's visual large model
   • Supports direct output at resolutions below 2048
   • Significant improvements in image-text understanding, structural accuracy, and aesthetics

2. I2IPortraitPhoto - Portrait Photo Generation

   • Generates diverse realistic portrait photos based on facial features
   • Supports text-guided style transformation
   • Perfect for interactive entertainment, photo effects, and e-commerce marketing

3. SeededitV30 - Smart Drawing 3.0 (Image Editing)

   • Advanced image-to-image editing model
   • Accurately follows editing instructions while preserving image content
   • Excels at handling real images and maintaining fine details

4. JimengHighAESGeneralV21L - Jimeng High Aesthetic General 2.1

   • Accurately interprets text to generate aesthetically unique images
   • Supports Chinese and English character rendering
   • Achieves "what you see is what you think" creative expression

🎬 Video Generation Tools

5. JimengVGFMT2VL20 - Jimeng Video Generation 2.0 (Text-to-Video)

   • Pure text-to-video generation
   • Deep understanding of text instructions
   • Supports multi-dimensional parameter fine-tuning

6. JimengVGFMI2VL20 - Jimeng Video Generation 2.0 (Image-to-Video)

   • First-frame image-to-video generation
   • Professional-level video generation experience
   • Excellent visual fluency

7. JimengVGFMT2VL30Pro - Jimeng Video Generation 3.0 Pro ⭐ Latest

   • Multi-shot narrative capabilities: Complex camera transitions, character movements, object motion trajectories
   • Dual modes: Text-to-video and image-to-video
   • Flexible parameters: 6 aspect ratios (16:9, 9:16, 1:1, 4:3, 3:4, 21:9), 2 duration options (5s/10s)
   • Multiple input methods: File upload, URL, Base64
   • High-quality output: Enhanced semantic understanding and video quality

🚀 Quick Start

Prerequisites

1. Register a Volcengine account and complete real-name authentication

   • Account Registration Guide
   • Real-Name Authentication Guide

2. Activate services in the Volcengine console

   • Visit Volcengine Console
   • Activate the visual AI services you need

3. Get API credentials

   • Obtain AccessKeyID and AccessKeySecret
   • How to Get AccessKey

Installation

1. Install from Dify Plugin Marketplace

   • Navigate to Dify Plugin Marketplace
   • Search for "Volcengine AI" or "即梦"
   • Click "Install"

2. Configure credentials

   • Go to Tools > 即梦 > Authorize
   • Enter your AccessKeyID and AccessKeySecret

Usage

The Volcengine AI tools can be used in:

• ✅ Chatflow applications
• ✅ Workflow applications
• ✅ Agent applications

Adding Tools to Your Workflow

1. Open your Chatflow or Workflow application
2. Add a tool node
3. Select "即梦" from the tool list
4. Choose the specific tool you want to use
5. Configure the parameters

📚 Tool Details

Text-to-Video Example (JimengVGFMT2VL30Pro)

Parameters:
  prompt: "A cute cat playing in a garden, sunny day, camera slowly zooming in"
  seed: -1
  frames: 121  # 5 seconds
  aspect_ratio: "16:9"

Image-to-Video Example (JimengVGFMT2VL30Pro)

Parameters:
  image_file: [Upload image file]
  seed: -1
  frames: 241  # 10 seconds
  aspect_ratio: "9:16"  # Perfect for TikTok, Instagram Stories

Multi-shot Narrative Example

Parameters:
  prompt: "Thousands of horses galloping"
  scene_switch: "Shot 1: Wide view (0-3s), Shot 2: Medium push-in (3-7s), Shot 3: Close-up details (7-10s)"
  frames: 241
  aspect_ratio: "21:9"  # Cinematic ultrawide

🔧 Technical Features

• ✅ Complete SDK Integration: Based on Volcengine's official SDK
• ✅ Asynchronous Task Processing: Intelligent polling mechanism for long-duration tasks
• ✅ Parameter Validation: Comprehensive mutual exclusion validation
• ✅ Multi-language Support: Chinese, English, Portuguese
• ✅ Error Handling: Friendly error messages and exception handling
• ✅ Secure Authentication: HMAC-SHA256 signature authentication

🎯 Use Cases

• 🎨 Content Creation: Social media, short videos, marketing materials
• 🎬 Video Production: Commercials, promotional videos, product demos
• 🖼️ Image Design: Posters, illustrations, product images
• 📸 Photo Processing: Portrait beautification, style transformation
• 🤖 AI Applications: Intelligent customer service, virtual assistants

🙏 Acknowledgments

This plugin is built upon the original Volcengine AI plugin developed by @parkerwen.

Special thanks to parkerwen for contributing an excellent plugin foundation to the Dify community, including:

• Complete SDK integration
• Standardized code structure
• Detailed documentation
• Multiple practical AI tools

Building on this foundation, we have added the Jimeng AI Video Generation 3.0 Pro tool to further enrich the plugin's functionality.

Original Author: parkerwen
Extended Development: shaoruidong

📄 License

MIT License

🔗 Links

• Volcengine Official Website
• Jimeng AI Video Generation 3.0 Pro API Documentation
• Dify Plugin Development Documentation
• Plugin Source Code
• Report Issues

📝 Version History

v0.0.1 (2025-01-22)

Added

• ✨ New Jimeng AI Video Generation 3.0 Pro tool
• ✨ Support for text-to-video and image-to-video dual modes
• ✨ Multi-shot narrative and camera transition descriptions
• ✨ Multiple aspect ratio options (6 types)
• ✨ Video duration control (5s/10s)
• ✨ Three image input methods

Optimized

• 🔧 Improved parameter mutual exclusion validation logic
• 🔧 Enhanced error messages
• 🔧 Better asynchronous task processing
• 🔧 Enhanced parameter documentation

Fixed

• 🐛 Fixed file type parameter validation issue in workflows
• 🐛 Fixed SSL recursion error (video download failure doesn't affect overall flow)

---

Made with ❤️ for the Dify community