Generating coherent comic with rich story using ChatGPT and Stable Diffusion
Project Overview
The document explores the integration of generative AI technologies, particularly ChatGPT and Stable Diffusion, in the educational realm, focusing on their application in comic creation by generating storylines and visuals while preserving artistic styles. It introduces an innovative evaluation method for assessing the quality of generated content, showcasing significant advancements in character fidelity and art style through meticulous fine-tuning techniques. The findings underscore the transformative potential of generative AI in creative education, enabling students to extend unfinished comics and develop new narratives featuring established characters. Overall, the research highlights the promising role of AI in enhancing creativity and artistic expression within educational contexts, suggesting that such technologies can foster engagement and innovation among learners.
Key Applications
Generating comics with ChatGPT and Stable Diffusion
Context: Creating comic book pages with rich storylines and visuals, targeting both comic enthusiasts and creators.
Implementation: Utilizing ChatGPT to generate comic page scripts and Stable Diffusion to create visual representations based on the scripts.
Outcomes: Achieved high fidelity in character representation and art style, with a novel story evaluation metric introduced.
Challenges: Fine-tuning models for specific characters and ensuring the generated story aligns well with existing narratives.
Implementation Barriers
Technical
Limitations in generating specific characters and maintaining style fidelity in AI-generated content.
Proposed Solutions: Employ fine-tuning techniques like LoRA and ControlNet to enhance model performance on specific character generation.
Evaluation
Lack of established metrics for evaluating AI-generated stories and visuals.
Proposed Solutions: Introducing a new metric (story score) to assess AI-generated narratives in relation to established works.
Project Team
Ze Jin
Researcher
Zorina Song
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Ze Jin, Zorina Song
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai