Module 12 Lesson 4: Running in Air-Gapped Environments
The ultimate privacy. How to install Ollama and your models on a machine with zero internet connection.
Air-Gapped AI: Total Isolation
An "Air-Gap" means the computer has no physical or wireless connection to the internet. No Wi-Fi, no Ethernet, no Bluetooth. This is where the most sensitive work in the world happens (Nuclear, Defense, Intelligence).
Ollama is uniquely suited for this because it doesn't need to "Call Home" to verify a license or download fonts.
1. The Challenge of Installation
How do you install a 5GB model on a computer that can't download anything?
The Sneakernet Workflow:
- Online Machine: Run
ollama pull llama3. - Export: Locate the manifest and blob files in
~/.ollama/models. - Physical Media: Copy these files to an encrypted USB drive.
- Air-Gapped Machine: Install the Ollama binary (via USB) and paste the model files into the correct directory.
2. Directory Mapping
You must manually recreate the folder structure on the offline machine:
~/.ollama/models/manifests/registry.ollama.ai/library/llama3/8b~/.ollama/models/blobs/sha256-...
Once the files are pasted, run ollama list. The model will appear as if it was downloaded locally.
3. Dependencies and Drivers
The air-gapped machine must already have NVIDIA Drivers and CUDA (if using Linux) installed. If you forget these, you cannot "just download them" later. You must package the .deb or .exe driver files onto your USB stick along with the model.
4. Updates and Patches
In an air-gapped world, you don't get security patches automatically. Best Practice: Once a quarter, create a "Maintenance Window" where you bring a new USB stick with the latest Ollama update and the newest model versions (e.g., Llama 3 -> Llama 3.1).
5. Why do this?
- Government/Military: Protecting state secrets.
- Law Firms: Ensuring client privilege is 1,000% safe.
- Researchers: Working with highly valuable proprietary algorithms.
Key Takeaways
- Ollama can run 100% offline with no loss in functionality.
- Sneakernet (USB drives) is the method for moving models to isolated machines.
- You must maintain the exact folder structure for Ollama to recognize the files.
- Pre-installing GPU drivers is the most common point of failure in offline setups.