Deploying Chainlit on GCP

Chainlit is an open-source Python package that “lets you create ChatGPT-like UIs on top of any Python code in minutes”. I’ve tried a lot of different solutions for building AI-powered tools lately while building and experimenting for Apollo AI, and Chainlit is by far the easiest set up experience I’ve had to get a functioning chat UI in minutes. It’s still brand new (just launched in mid-May 2023) so it’s a work in progress, but the Chainlit team has been highly active and receptive of feedback in their Discord and Github issues.

Chainlit came along exactly when I needed it. I’m a self-taught Python-er, so anything that has to do with frontend has always been a little bit challenging. Chainlit wraps around your existing Python code with simple decorators and integrated module functions to quickly create a simple and clean interface for assistants. Think of it like Streamlit, but specifically for assistants/chatbots.

After spending time experimenting with deploying a Chainlit app to Google Cloud Platform (GCP), I wanted to share this walkthrough to help the growing Chainlit community with rapid self-deployment. 🙂

There are a few requirements for the walkthrough, which I won’t cover. They’re all straightforward, so it’s self-explanatory.

What you’ll need:

- GCP account: New users get a $300 dollar credit to use the various services. I’ve been using GCP for months now hosting other products and not come close to using all of this up.

- Docker: We’re going to containerize our Chainlit app to host it on GCP Cloud Run.

- gcloud cli: For interacting with your gcloud environment.

- Chainlit: Obviously, you’ll need the Chainlit package.

- Your Python code: I’m going to deploy the tutorial code found in the documentation, so replace that with your own Python code. For the walkthrough, I’ve already gone through the Pure Python tutorial. Be sure to have run chainlit run app.py before this walkthrough, as this creates the configuration and readme files that are required for the app to function.

Be sure to set up your Artifact Registry repository in GCP ahead of time as well, as you’ll need to know where to send your built Docker image. GCP provides a lot of documentation and setup instructions on each service page, so be sure to check there for any specific things you might need to do, like client auth, etc.

Okay! Now that those requirements are out of the way, let’s get deploying.

Local Prep

After you’ve run your application for the first time, you will probably have a file structure like this:

When we package everything in a container, we’ll need all these files. So, create a new folder in your base directory, which is chainlit-gcp for this, and move the .chainlit folder, app.py, and chainlit.md into it. I’ve named mine docker-package.

In your base directory, we’ll need to create two new files: a dockerfile and a requirements.txt file. These are required for building a Docker image. You can look up how to create these two files if you need to, but they are standard and easy to get going.

One note here: my biggest challenge when deploying my own app didn’t have anything to do with Chainlit, but rather Docker. I have an M2 Macbook Air that I use as my primary dev machine (primary everything machine these days), which introduces all the ARM-based architecture …challenges. One I didn’t realize was that Docker will default to the host’s architecture when building container images. So, ARM host == ARM container image, unless you specify otherwise.

Here’s the rub: GCP doesn’t support ARM image deployments on Cloud Run (one of those challenges I was referring to), which resulted in a Docker image that would run perfectly fine locally, only to fail deployment on Cloud Run for no apparent reason. Frustrating to say the least.

Luckily, there’s an easy solution, which is to specify the base image’s architecture in your FROM statement in the dockerfile. I used `FROM — platform=linux/amd64 python:3.9` and problem gone. So, if you’re also developing on an ARM system, just remember that!

Assuming you’ve got your dockerfile and requirements.txt ready to go, we can do the following to build the image and send it up to our GCP Artifact Registry:

From within the base directory, build the image:

docker build -t chainlit-gcp-image .

Test that the image built correctly:

docker run -p 8000:8000 chainlit-gcp-image

• you may get a warning if the built image doesn’t match the host architecture if you’re building on ARM-based systems — ignore it

Tag the image with where to send it (your Artifact Regsitry):

docker tag chainlit-gcp-image:latest <region>-docker.pkg.dev/<your GCP project ID>/<your repository name>/chainlit-gcp-image:latest

Push the image up to the GCP Artifact Registry:

docker push <region>-docker.pkg.dev/<your GCP project ID>/<your repository name>/chainlit-gcp-image:latest

Deploy on GCP

Now, navigate to Cloud Run, which we’ll use to host our container. The nice thing about Cloud Run is you don’t have to deal with Kubernetes or anything like that.

Create a service (instead of a job) and select the container image that we just pushed up to our Artifact Registry.

There are a couple options you can tweak here depending on your use case, but for this I’ve mostly kept the default options and selected “Allow unauthenticated invocations” which will make this publicly accessible.

The one thing you should change is the Container port, which is found by expanding the “Container, Networking, Security” section. Change it to 8000, as this is the default port that Chainlit is hosted on (though this can be changed by setting the — port option on startup).

Then click “Create” at the bottom. If everything deploys right, you should see all greens like the screenshot below.

That’s it! Your app is accessible via the URL listed there on the page.

There are a couple points to note on this page as well. With the “Edit & Deploy New Revision” button, you can go through this whole walkthrough again, building and uploading a new image, and easily select the new image and deploy it within this same Service. Also, with the “Set Up Continuous Deployment” option, you can create a workflow for connecting to a Github repository to continuously deploy images as you make updates to them.

I hope this helps the growing Chainlit community! Hit me up in the Chainlit Discord (pseudohvr) if you have any questions about the walkthrough. I’ll do my best to answer Chainlit specific questions, but those are probably better directed towards the creators!