Skip to content

OctoAI DockerCon AI/ML Workshop

This repository holds instructions to build and deploy, "OctoShop", a multi-modal use case that leverages multiple models: Stable Diffusion XL, CLIP Interrogator, Llama2 etc. in order to manipulate images in creative ways using just words!

Co-Authors

  • Thierry Moreau - Co-Founder, DevRel @OctoML LinkedIn
  • Luis Vega - ML Staff Engineer @OctoML LinkedIn
  • Sameer Farooqui - Product Marketing Manager @ OctoML LinkedIn
  • Ajeet Raina, Developer Advocate @ Docker LinkedIn

GitHub Sources

The code and iPython notebook sources for this workshop are contained under: https://github.com/octoml/dockercon23-octoai

Introduction

In this lab:

  • You'll learn to "containerize" a model using Python and Dockerfiles.
  • You'll learn to deploy that model container on an OctoAI compute endpoint.
  • You'll learn to build the OctoShop multi-modal, GenAI-powered image processing pipeline.
  • You'll learn to optimize consistency between input and output images using ControlNet and face preservation tools.
  • You'll learn to build an engaging and interactive Discord bot that applies unique image transformations to any image of your choice!

octoshop_pipeline

High Level Walkthrough

You'll be able to go from this logo of Moby:

lab1_input

To this hyper-realistic rendition of Moby powered by SDXL:

lab1_output

You'll learn to exploit the power of Llama 2 to bring Moby to space with some clever prompt engineering:

lab1_output_space

You'll be able to achieve better consistency through the power of ControlNet as exemplified by this hyper-realistic rendition is much more consistent with the original Docker logo:

lab2_output_whale

Finally you'll unlock the ability to leverate AI-powered face swap in order to take the Mona Lisa to the space age!

lab2_output_monalisa

Lab Overview

Pre-Requisites

  • Discord setup
  • DockerHub setup
  • OctoAI setup
  • AWS instance setup

Part 1 - Baseline OctoShop Pipeline

  • How to build a SXDL model container using Docker
  • How to launch an OctoAI SDXL inference endpoint
  • Testing a model endpoint in an iPython notebook
  • Test an "OctoShop" multi-endpoint pipeline (CLIP - Llama 2 - SDXL) in an iPython notebook
  • Deploy your own "Baseline OctoShop" discord bot
  • Share your creation with your workshop co-participants!

Part 2 - Advanced OctoShop Pipeline (ControlNet, Face Preservation)

  • Extending the SDXL model container with a ControlNet
  • Updating the model container of your OctoAI inference endpoint
  • Testing your new SDXL+ControlNet endpoint in an iPython notebook
  • Test a multi-endpoint pipeline (CLIP - Llama 2 - SDXL+ControlNet - FaceSwap) in an iPython notebook
  • Deploy your own "Advanced OctoShop" discord bot
  • Share your creation with your workshop co-participants!