Cloud-Init: The Bootstrap Secret

When you launch an Ubuntu server on AWS, it starts in seconds. When you log in, your SSH key is already there, the hostname is correct, and maybe some packages are already installed.

How did the server know who you were?

The answer is Cloud-Init.

Cloud-Init is a "First-Boot" automation tool that runs inside the Linux kernel during the initial startup. It reaches out to a "Metadata Service" provided by the cloud (e.g., AWS at 169.254.169.254) and pulls down your configuration. In this lesson, we will learn to write User-Data scripts to customize your servers before you even log in.

---

1. What is User-Data?

"User-Data" is a string of text (usually in YAML format) that you pass to the cloud provider when you launch a server. Cloud-Init reads this text and executes the commands.

---

2. Practical: A Basic Cloud-Config

A cloud-init file must start with #cloud-config.

#cloud-config
# Create a new user with sudo access and an SSH key
users:
  - name: sudeep
    ssh-authorized-keys:
      - ssh-rsa AAAAB3Nza... your-public-key-here
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    groups: sudo
    shell: /bin/bash

# Update packages and install Nginx
package_update: true
packages:
  - nginx
  - curl
  - htop

# Run a one-time setup command
runcmd:
  - [ systemctl, start, nginx ]
  - echo "Web server initialized at $(date)" > /var/www/html/status.txt

---

3. The Three Stages of Cloud-Init

1. Local: Mounts the internal "User-Data" source.
2. Network: Once the network is up, it downloads metadata (like public IPs).
3. Config: Changes users, group, and files.

---

4. Troubleshooting: Why did it fail?

If you log into your server and your user wasn't created, or Nginx isn't installed, you need to check the Cloud-Init logs.

# View the raw output of the cloud-init scripts
sudo cat /var/log/cloud-init-output.log

# See a summary of the boot stages
cloud-init status --long

---

5. Idempotency (Once per Instance)

By default, Cloud-Init commands only run once per instance lifecycle. If you reboot the server, it won't reinstall Nginx. If you want a script to run on every boot, you need to use a specific module like bootcmd (not recommended for complex tasks).

---

6. Example: A Cloud-Init Validator (Python)

Cloud-Init is very sensitive to indentation (YAML). If you have a single extra space, the server will fail to initialize. Here is a Python script that validates your cloud-config file for standard syntax.

import yaml

def validate_cloud_init(file_path):
    """
    Checks if a file starts with #cloud-config and is valid YAML.
    """
    try:
        with open(file_path, 'r') as f:
            lines = f.readlines()
            if not lines[0].startswith("#cloud-config"):
                print("[!!!] ERROR: File must start with #cloud-config")
                return False
            
            # Join the rest of the lines and parse as YAML
            content = "".join(lines)
            yaml.safe_load(content)
            print("[OK] Cloud-Init file is syntactically correct.")
            return True
            
    except Exception as e:
        print(f"[!!!] YAML ERROR: {e}")
        return False

if __name__ == "__main__":
    validate_cloud_init("myserver-config.yml")

---

7. Professional Tip: Use 'No-Cloud' for VMs

If you are building your own private laboratory at home (using Proxmox or VirtualBox), you don't have a "Metadata Service" from a cloud provider. You can still use Cloud-Init by creating a tiny ISO file named config-drive and attaching it to the VM. This is how pros automate their local homelabs.

---

8. Summary

Cloud-Init is the bridge between a generic image and your specific server.

• User-Data provides the instructions.
• #cloud-config is the mandatory header.
• Metadata Services provide the context (IPs, keys).
• Automation happens before you ever see a login prompt.
• Logs in /var/log/ are the key to debugging initialization failures.

In the next lesson, we move from the boot process to the life cycle: Linux in CI/CD Pipelines.

Quiz Questions

1. Where does a cloud-init script get its information if it isn't hardcoded in the OS image?
2. What is the difference between runcmd and bootcmd?
3. Which file contains the record of any errors that happened during the cloud-init boot process?

---

Continue to Lesson 4: Pipeline Power—Linux in CI/CD.