AI DevOps: Continuous Deployment

In Module 5, we learned how to create Modelfiles. In a professional setting, you don't want to manually run ollama create on 10 different servers. You want a CD (Continuous Deployment) pipeline that handles it for you.

1. The GitHub/GitLab Workflow

1. Commit: You change the SYSTEM prompt in your ProductionModelfile.
2. Push: You push the change to GitHub.
3. Action: A GitHub Action triggers.
4. Deployment: The action SSHs into your servers and runs ollama create my-app-bot -f ProductionModelfile.

---

2. Automating Registry Pulls

If you want to ensure your servers always have the absolute latest weights of Llama 3 (which Meta updates occasionally), you can set a Cron Job (scheduled task) on your server.

Example script (/etc/cron.daily/ollama-update):

#!/bin/bash
ollama pull llama3
ollama pull codellama

---

3. Testing in the Pipeline

Before you push a new Modelfile to production, your CD pipeline should "Test" it.

• The Script: A Python script calls the newly created model.
• The Test: It asks a question like "What is our company name?".
• The Logic: If the model answers incorrectly or returns a 500 error, the deployment is ABORTED.

This prevents you from accidentally deploying a "broken" AI to your users.

---

4. Blue/Green Deployments

If you have a high-traffic app:

1. Deploy the new model as my-bot:v2.
2. Slowly move 10% of users to v2.
3. If everything is stable, move 100% of users and delete v1.

Ollama’s tagging system (model:tag) makes this incredibly easy compared to traditional software.

---

5. Summary CD Checklist

• My Modelfiles are version-controlled in Git.
• My servers are managed by Ansible, Chef, or Puppet to ensure Ollama is installed.
• I have an automated "Sanity Check" script for new models.
• I use a specific tag (:v1.2) in production, never :latest.

---

Key Takeaways

• CD removes human error from the AI deployment process.
• Git is the single source of truth for your model's personality and settings.
• Automated testing is critical to ensure a new prompt doesn't break your app.
• Tagging allows for safe "Blue/Green" rollouts of new model versions.