FashionXchange

  • Tech Stack: Python, PyTorch, PyTorch Lightning, HuggingFace, Accelerate, StableDiffusion, Segment Anything Model, GroundingDINO
  • Github URL: Project Link

An application with Streamlit for text-based modification of outfits of people in images using Grounding DINO, Segment Anything Model and Stable Diffusion.

Improved the Stabilit-AI stable diffusion inpainting model by finetuning with DeepFashion dataset and custom masking strategy targetting the outfits.

The model was finetuned leveraging Low Rank Adaptation (LoRA) and prior preservation loss resulting in superior pose preservation, reduced distortion, and improved results for complex modifications.

Demo of the application