Testing Ansible Roles with Molecule on Digital Ocean Droplets

For years, I’ve been using Ansible to configure my homelab, blog, and workstations. After testing out Chef and Salt, I’m glad I happened upon Ansible. The tool has grown more and I’ve gotten tons of value out of putting in the effort to learn it.

Early on, I would manually test my code to make sure whatever I added worked. It was tougher since my work used RHEL-based distributions so I would always check my work against centos as well as debian distros. This grew tiresome and I looked for ways to automate the testing.

Soon, I found Vagrant, which let me spin up local virtual machines quickly then test my Ansible roles against them to see if they had the desired impact.

Then, I was able to stand on top of Vagrant with Molecule for testing roles. It allowed you to choose a driver to set up the machine, apply your role, apply it again to see if it was idempotent, then would tear down the VM and report the results. I even got it all running in my own GitLab instance as part of a merge request. Adding the latest Ubuntu release was a two line change and BOOM, it was now part of your tests.

It was glorious…..for a short while. Then, a big breaking change came from the molecule developers after I set up testing on all of my roles. My productivity with the tool was never the same.

I would hard-code the molecule version I needed but the writing was on the wall. The vagrant driver I’d been using was no longer supported and everything would be moving to docker. My issue with that was that lots of my roles configured systemd services and emulating that in a docker container made for a testing nightmare. I could never confirm I was fixing test infrastructure bugs or bugs from my roles.

Frankly, I was spending more time keeping my test framework up to date than I was developing roles to solve real problems. I left my Molecule tests abandoned in my repository for years.

Then enters Claude Code. I’ve had lots of experience building useful web apps for myself. Even in languages and frameworks I wasn’t familiar with.

It got me thinking, is it time to revamp my role testing strategy with Molecule? With the power of the Digital Ocean API, the answer is yes.

For prerequisites, you’ll need Ansible and Molecule installed via your python package manage of choice. You’ll also need doctl to control the creation of the DO droplets.

Below is the Molecule configuration I use to test my ssh_hardening role.

# file: create.yml
# harness: Claude Code
# model: Opus 5.5

---
- name: Create DigitalOcean droplets
  hosts: localhost
  gather_facts: false
  vars:
    do_region: "{{ lookup('ansible.builtin.env', 'DO_REGION') | default('nyc3', true) }}"
    do_size: "{{ lookup('ansible.builtin.env', 'DO_SIZE') | default('s-1vcpu-1gb', true) }}"
    do_project: "{{ lookup('ansible.builtin.env', 'DO_PROJECT') | default('Molecule', true) }}"
    do_state_file: "{{ molecule_ephemeral_directory }}/do_state.yml"
    do_key_file: "{{ molecule_ephemeral_directory }}/id_ed25519"
    do_user: molecule
  tasks:
    # Accepts DIGITALOCEAN_ACCESS_TOKEN or a token saved with `doctl auth init`.
    - name: Check DigitalOcean authentication
      ansible.builtin.command:
        argv: [doctl, account, get]
      register: do_account
      changed_when: false
      failed_when: false

    - name: Assert DigitalOcean authentication works
      ansible.builtin.assert:
        that: do_account.rc == 0
        fail_msg: >-
          doctl cannot authenticate. Run `doctl auth init` or set DIGITALOCEAN_ACCESS_TOKEN.
          {{ do_account.stderr | default(do_account.msg | default('')) }}

    - name: List DigitalOcean projects
      ansible.builtin.command:
        argv: [doctl, projects, list, --output=json]
      register: do_projects
      changed_when: false

    - name: Set project ID
      ansible.builtin.set_fact:
        do_project_id: "{{ do_projects.stdout | from_json | selectattr('name', 'eq', do_project) | map(attribute='id') | first | default('') }}"

    - name: Assert project exists
      ansible.builtin.assert:
        that: do_project_id | length > 0
        fail_msg: DigitalOcean project '{{ do_project }}' not found. Create it or set DO_PROJECT.

    # Unique per run so parallel runs (macOS, Linux, CI) do not collide.
    - name: Generate run ID
      ansible.builtin.set_fact:
        do_run_id: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_lowercase', 'digits'], length=8) }}"

    - name: Generate ephemeral SSH key
      ansible.builtin.command:
        argv: [ssh-keygen, -q, -t, ed25519, -N, "", -C, "molecule-{{ do_run_id }}", -f, "{{ do_key_file }}"]
        creates: "{{ do_key_file }}"

    # Registering a key stops DigitalOcean from setting and emailing a root password.
    - name: Register SSH key with DigitalOcean
      ansible.builtin.command:
        argv:
          - doctl
          - compute
          - ssh-key
          - import
          - molecule-{{ do_run_id }}
          - --public-key-file={{ do_key_file }}.pub
          - --output=json
      register: do_ssh_key
      changed_when: true

    # Write state before droplets exist, so destroy can always clean up.
    - name: Write DigitalOcean state
      ansible.builtin.copy:
        dest: "{{ do_state_file }}"
        mode: "0600"
        content: |
          run_id: {{ do_run_id }}
          ssh_key_id: {{ (do_ssh_key.stdout | from_json | first).id }}

    # ssh_pwauth makes cloud-init write 50-cloud-init.conf with
    # PasswordAuthentication yes, which the role must override.
    - name: Write cloud-init user data
      ansible.builtin.copy:
        dest: "{{ molecule_ephemeral_directory }}/user-data.yml"
        mode: "0600"
        content: |
          #cloud-config
          ssh_pwauth: true
          users:
            - name: {{ do_user }}
              shell: /bin/bash
              sudo: ALL=(ALL) NOPASSWD:ALL
              ssh_authorized_keys:
                - {{ lookup('ansible.builtin.file', do_key_file ~ '.pub') }}

    - name: Create droplets
      ansible.builtin.command:
        argv:
          - doctl
          - compute
          - droplet
          - create
          - "molecule-{{ item.name }}-{{ do_run_id }}"
          - --image={{ item.image }}
          - --size={{ do_size }}
          - --region={{ do_region }}
          - --project-id={{ do_project_id }}
          - --ssh-keys={{ (do_ssh_key.stdout | from_json | first).id }}
          - --tag-names=molecule,molecule-{{ do_run_id }}
          - --user-data-file={{ molecule_ephemeral_directory }}/user-data.yml
          - --wait
          - --output=json
      loop: "{{ molecule_yml.platforms }}"
      loop_control:
        label: "{{ item.name }}"
      register: do_droplets
      changed_when: true

    - name: Write Molecule instance config
      ansible.builtin.copy:
        dest: "{{ molecule_instance_config }}"
        mode: "0600"
        content: "{{ do_instances | to_nice_yaml }}"
      vars:
        do_instances: >-
          {%- set out = [] -%}
          {%- for r in do_droplets.results -%}
          {%- set d = r.stdout | from_json | first -%}
          {%- set _ = out.append({
                'instance': r.item.name,
                'address': d.networks.v4 | selectattr('type', 'eq', 'public') | map(attribute='ip_address') | first,
                'user': do_user,
                'port': 22,
                'identity_file': do_key_file,
              }) -%}
          {%- endfor -%}
          {{ out }}

    - name: Wait for SSH
      ansible.builtin.wait_for:
        host: "{{ item.address }}"
        port: 22
        search_regex: OpenSSH
        timeout: 300
      loop: "{{ lookup('ansible.builtin.file', molecule_instance_config) | from_yaml }}"
      loop_control:
        label: "{{ item.instance }}"
...

# file: converge.yml

---
- name: Converge
  hosts: all
  become: true
  roles:
    - ssh_hardening
...

# file: destroy.yml
# harness: Claude Code
# model: Opus 5.5

---
- name: Destroy DigitalOcean droplets
  hosts: localhost
  gather_facts: false
  vars:
    do_state_file: "{{ molecule_ephemeral_directory }}/do_state.yml"
  tasks:
    - name: Read DigitalOcean state
      ansible.builtin.stat:
        path: "{{ do_state_file }}"
      register: do_state_stat

    - name: Destroy run resources
      when: do_state_stat.stat.exists
      vars:
        do_state: "{{ lookup('ansible.builtin.file', do_state_file) | from_yaml }}"
      block:
        - name: Delete droplets
          ansible.builtin.command:
            argv:
              - doctl
              - compute
              - droplet
              - delete
              - --tag-name=molecule-{{ do_state.run_id }}
              - --force
          register: do_delete
          changed_when: do_delete.rc == 0
          failed_when: do_delete.rc != 0 and 'not found' not in do_delete.stderr

        - name: Delete SSH key
          ansible.builtin.command:
            argv: [doctl, compute, ssh-key, delete, "{{ do_state.ssh_key_id }}", --force]
          register: do_key_delete
          changed_when: do_key_delete.rc == 0
          failed_when: do_key_delete.rc != 0 and 'not found' not in do_key_delete.stderr

        - name: Remove DigitalOcean state
          ansible.builtin.file:
            path: "{{ item }}"
            state: absent
          loop:
            - "{{ do_state_file }}"
            - "{{ molecule_ephemeral_directory }}/id_ed25519"
            - "{{ molecule_ephemeral_directory }}/id_ed25519.pub"
            - "{{ molecule_ephemeral_directory }}/user-data.yml"

    - name: Write empty instance config
      ansible.builtin.copy:
        dest: "{{ molecule_instance_config }}"
        content: "[]\n"
        mode: "0600"
...

# file: molecule.yml
# harness: Claude Code
# model: Opus 5.5

---
# Droplets are created and destroyed with doctl.
# Requires doctl, authenticated with `doctl auth init` or DIGITALOCEAN_ACCESS_TOKEN.
# Optional: DO_REGION (default nyc3), DO_SIZE (default s-1vcpu-1gb),
# DO_PROJECT (default Molecule; the project must already exist).
driver:
  name: default
  options:
    managed: true
# Slugs: doctl compute image list-distribution --public
platforms:
  - name: ubuntu2604
    image: ubuntu-26-04-x64
  - name: ubuntu2404
    image: ubuntu-24-04-x64
  - name: ubuntu2204
    image: ubuntu-22-04-x64
  - name: debian13
    image: debian-13-x64
provisioner:
  name: ansible
  env:
    ANSIBLE_ROLES_PATH: ${MOLECULE_PROJECT_DIRECTORY}/roles
scenario:
  test_sequence:
    - destroy
    - syntax
    - create
    - prepare
    - converge
    - idempotence
    - verify
    - destroy
...

# file: prepare.yml
# harness: Claude Code
# model: Opus 5.5

---
- name: Wait for droplets to finish booting
  hosts: all
  gather_facts: false
  tasks:
    - name: Wait for SSH login
      ansible.builtin.wait_for_connection:
        timeout: 300

    # Exit code 2 means done with recoverable errors, which DO images often report.
    - name: Wait for cloud-init
      ansible.builtin.command: cloud-init status --wait
      register: cloud_init_status
      changed_when: false
      failed_when: cloud_init_status.rc not in [0, 2]

    # Proves the test starts from an insecure baseline.
    - name: Assert cloud-init enabled password logins
      ansible.builtin.command: grep -Eiqr '^PasswordAuthentication yes' /etc/ssh/sshd_config /etc/ssh/sshd_config.d
      become: true
      changed_when: false
...

# file: verify.yml
# harness: Claude Code
# model: Opus 5.5

---
- name: Verify effective sshd configuration
  hosts: all
  become: true
  gather_facts: false
  vars:
    expected:
      permitrootlogin: "no"
      passwordauthentication: "no"
      kbdinteractiveauthentication: "no"
      authenticationmethods: publickey
      permitemptypasswords: "no"
      maxauthtries: "3"
      x11forwarding: "no"
      allowtcpforwarding: "no"
      allowagentforwarding: "no"
      permittunnel: "no"
      loglevel: VERBOSE
  tasks:
    # sshd -T prints the config after includes and defaults are applied.
    - name: Read effective sshd configuration
      ansible.builtin.command: sshd -T
      register: sshd_t
      changed_when: false

    - name: Assert hardened settings are in effect
      ansible.builtin.assert:
        that: (item.key ~ ' ' ~ item.value) in sshd_t.stdout_lines
        fail_msg: "Expected '{{ item.key }} {{ item.value }}' in sshd -T output."
        quiet: true
      loop: "{{ expected | dict2items }}"
      loop_control:
        label: "{{ item.key }}"

# file: cleanup.yml
# harness: Claude Code
# model: Opus 5.5

---
# Molecule runs this on failure, before destroy. Show sshd state for debugging.
- name: Collect sshd diagnostics
  hosts: all
  become: true
  gather_facts: false
  ignore_unreachable: true
  tasks:
    - name: Read recent sshd logs
      ansible.builtin.command: journalctl --unit=ssh --lines=50 --no-pager
      register: sshd_journal
      changed_when: false
      failed_when: false

    - name: Show recent sshd logs
      ansible.builtin.debug:
        var: sshd_journal.stdout_lines
      when: sshd_journal.stdout_lines is defined
...

The destroy file cleans up before and after your tests so you aren’t billed. Create and prepare set up the droplets. Converge actually calls your ansible role then verifier tests your role via assertions.

AI is exceptionally good at this stuff. I plan on slowing backfilling my roles with proper tests across Ubuntu and Debian droplets to confirm I’m not adding regressions. I’ll also set this up to run in a Forgejo Action when I open up pull requests.