OctoAI DockerCon AI/ML Workshop¶
This repository holds instructions to build and deploy, "OctoShop", a multi-modal use case that leverages multiple models: Stable Diffusion XL, CLIP Interrogator, Llama2 etc. in order to manipulate images in creative ways using just words!
Co-Authors¶
- Thierry Moreau - Co-Founder, DevRel @OctoML LinkedIn
- Luis Vega - ML Staff Engineer @OctoML LinkedIn
- Sameer Farooqui - Product Marketing Manager @ OctoML LinkedIn
- Ajeet Raina, Developer Advocate @ Docker LinkedIn
GitHub Sources¶
The code and iPython notebook sources for this workshop are contained under: https://github.com/octoml/dockercon23-octoai
Introduction¶
In this lab:
- You'll learn to "containerize" a model using Python and Dockerfiles.
- You'll learn to deploy that model container on an OctoAI compute endpoint.
- You'll learn to build the OctoShop multi-modal, GenAI-powered image processing pipeline.
- You'll learn to optimize consistency between input and output images using ControlNet and face preservation tools.
- You'll learn to build an engaging and interactive Discord bot that applies unique image transformations to any image of your choice!
High Level Walkthrough¶
You'll be able to go from this logo of Moby:
To this hyper-realistic rendition of Moby powered by SDXL:
You'll learn to exploit the power of Llama 2 to bring Moby to space with some clever prompt engineering:
You'll be able to achieve better consistency through the power of ControlNet as exemplified by this hyper-realistic rendition is much more consistent with the original Docker logo:
Finally you'll unlock the ability to leverate AI-powered face swap in order to take the Mona Lisa to the space age!
Lab Overview¶
Pre-Requisites¶
- Discord setup
- DockerHub setup
- OctoAI setup
- AWS instance setup
Part 1 - Baseline OctoShop Pipeline¶
- How to build a SXDL model container using Docker
- How to launch an OctoAI SDXL inference endpoint
- Testing a model endpoint in an iPython notebook
- Test an "OctoShop" multi-endpoint pipeline (CLIP - Llama 2 - SDXL) in an iPython notebook
- Deploy your own "Baseline OctoShop" discord bot
- Share your creation with your workshop co-participants!
Part 2 - Advanced OctoShop Pipeline (ControlNet, Face Preservation)¶
- Extending the SDXL model container with a ControlNet
- Updating the model container of your OctoAI inference endpoint
- Testing your new SDXL+ControlNet endpoint in an iPython notebook
- Test a multi-endpoint pipeline (CLIP - Llama 2 - SDXL+ControlNet - FaceSwap) in an iPython notebook
- Deploy your own "Advanced OctoShop" discord bot
- Share your creation with your workshop co-participants!