Morphit Team
586271574d
Some checks failed
morphit-release / Build + publish release tarball (push) Has been cancelled
439 lines
20 KiB
YAML
439 lines
20 KiB
YAML
# Morphit relay/indexer + BunkerWeb deployment playbook.
|
|
#
|
|
# Reference: docs/OPERATIONS.md in the morphit repo.
|
|
# See README.md for usage, caveats, and pre-flight checklist.
|
|
#
|
|
# Apply roles in order — each depends on the previous:
|
|
# base → users, base packages, NTP, hostname, timezone
|
|
# hardening → SSH + sysctl + auditd + AIDE + UFW + fail2ban
|
|
# + secrets perms (§37, §34)
|
|
# tls → certbot + renew timer (§35)
|
|
# postgres → install + harden + DB provisioning (§30, §37.8)
|
|
# morphit → clone, build, env files, systemd, backups
|
|
# (§18, §23, §31, §38.7)
|
|
# bunkerweb → Docker + BunkerWeb container + WAF + trusted-
|
|
# proxy IPs (§32)
|
|
#
|
|
# Optional sidecar roles (controlled by enable_* group_vars):
|
|
# matrix_bot → operator-alerts to Matrix (cp9, §16)
|
|
# host_monitor → disk/mem/swap/cpu monitor (cp10, §16)
|
|
# smartctl_monitor → disk SMART health (cp11, §16)
|
|
# fail2ban_monitor → fail2ban jail observability (cp11, §16)
|
|
# mdadm_monitor → Linux software RAID health (cp11, §16)
|
|
# dmesg_monitor → kernel-log scan (OOM, oops, MCE) (cp12, §16)
|
|
# trivy_monitor → Docker image CVE rescan (cp12, §16)
|
|
# postfix_monitor → mail queue depth (alerting-self-check) (cp12, §16)
|
|
# certbot_monitor → TLS cert expiry + renewal-stall (cp13, §16)
|
|
# apt_monitor → pending security update count (cp13, §16)
|
|
# compose_monitor → Docker Compose service health (cp13, §16)
|
|
# systemd_monitor → systemd unit health (cp14, §16)
|
|
# journald_monitor → journal disk usage + rotation (cp14, §16)
|
|
#
|
|
# Run with:
|
|
# ansible-playbook -i inventory/hosts.yml playbook.yml --check
|
|
# ansible-playbook -i inventory/hosts.yml playbook.yml
|
|
#
|
|
# Or one tag at a time for incremental rollout:
|
|
# ansible-playbook -i inventory/hosts.yml playbook.yml --tags hardening
|
|
# ansible-playbook -i inventory/hosts.yml playbook.yml --tags monitors
|
|
|
|
---
|
|
- name: Morphit relay + indexer + BunkerWeb deployment
|
|
hosts: "{{ morphit_target_hosts | default('morphit_servers') }}"
|
|
become: true
|
|
gather_facts: true
|
|
|
|
pre_tasks:
|
|
- name: Read /etc/os-release (to find the Ubuntu base codename)
|
|
ansible.builtin.slurp:
|
|
src: /etc/os-release
|
|
register: morphit_os_release
|
|
check_mode: false
|
|
|
|
- name: Determine the Ubuntu base codename
|
|
# Ubuntu AND its derivatives (Linux Mint, Pop!_OS, Zorin, elementary,
|
|
# …) carry UBUNTU_CODENAME in /etc/os-release naming their Ubuntu
|
|
# base — e.g. Linux Mint 22 (any edition: Cinnamon/MATE/Xfce) →
|
|
# "noble". We key the codename-pinned apt repos (Docker, Trivy) off
|
|
# THIS, not ansible_distribution_release, which on a derivative is the
|
|
# derivative's OWN codename (e.g. Mint's "wilma") that the Ubuntu
|
|
# repos don't publish. Appending [''] keeps the list non-empty so
|
|
# `first` is safe when UBUNTU_CODENAME is absent (Debian/LMDE).
|
|
ansible.builtin.set_fact:
|
|
morphit_ubuntu_codename: >-
|
|
{{ (morphit_os_release.content | b64decode
|
|
| regex_findall('^UBUNTU_CODENAME=(.*)$', multiline=True)
|
|
+ ['']) | first | trim }}
|
|
|
|
- name: Verify target is Ubuntu 24.04 LTS or an Ubuntu-24.04-based derivative
|
|
ansible.builtin.assert:
|
|
that:
|
|
- morphit_ubuntu_codename == "noble"
|
|
fail_msg: >
|
|
This playbook targets Ubuntu 24.04 LTS (codename "noble") OR an
|
|
Ubuntu-24.04-based derivative such as Linux Mint 22 (any edition —
|
|
Cinnamon/MATE/Xfce), Pop!_OS 24.04, or Zorin OS 17. Detected
|
|
{{ ansible_distribution }} {{ ansible_distribution_version }} with
|
|
Ubuntu base codename "{{ morphit_ubuntu_codename }}". Earlier
|
|
Ubuntu LTS (22.04 "jammy" / Linux Mint 21), Debian, LMDE, and
|
|
non-Ubuntu distros are not supported — package names and config
|
|
paths differ. See RUN-A-MORPHIT-NODE.md §4.
|
|
success_msg: >
|
|
Target is {{ ansible_distribution }} {{ ansible_distribution_version }}
|
|
on the Ubuntu 24.04 "noble" base — supported.
|
|
|
|
- name: Verify the connection is safe (non-root over SSH, OR a local install)
|
|
ansible.builtin.assert:
|
|
that:
|
|
# Order matters: check the local-install flag FIRST so a local install
|
|
# short-circuits before `ansible_user` is ever evaluated. On a local
|
|
# connection `ansible_user` is undefined, and `undefined != "root"`
|
|
# RAISES (Ansible undefined-var error) rather than returning true — so
|
|
# it must never be the left operand, and it is `default("")`-guarded as
|
|
# belt-and-suspenders for any remote path that leaves it unset.
|
|
- (morphit_local_install | default(false) | bool) or (ansible_user | default("") != "root")
|
|
fail_msg: >
|
|
For a REMOTE install, connect with a non-root sudo-capable user — the
|
|
hardening role disables root SSH, so connected as root you'd lock
|
|
yourself out partway through. For a LOCAL install on the machine
|
|
itself (set morphit_local_install=true, which `morphit-ops install`
|
|
does automatically) running as root is fine: there is no SSH session
|
|
to lock out.
|
|
|
|
roles:
|
|
- role: vendor
|
|
tags: [vendor, always]
|
|
|
|
- role: base
|
|
tags: [base]
|
|
|
|
- role: hardening
|
|
tags: [hardening]
|
|
|
|
# HOME nodes only (enable_ddns): keep the domain pointed at a changing
|
|
# public IP. Runs BEFORE tls so certbot can validate the domain once DNS
|
|
# is current. A VPS has a static IP and leaves enable_ddns off.
|
|
- role: ddns
|
|
tags: [ddns]
|
|
when: enable_ddns | default(false)
|
|
|
|
- role: tls
|
|
tags: [tls]
|
|
when: enable_tls | default(true)
|
|
|
|
- role: postgres
|
|
tags: [postgres]
|
|
|
|
- role: morphit
|
|
tags: [morphit]
|
|
|
|
- role: bunkerweb
|
|
tags: [bunkerweb]
|
|
when: enable_bunkerweb | default(true)
|
|
|
|
# Basic Tor v3 onion by default (privacy is the first priority).
|
|
# Serves the address the wizard generated + the site advertises;
|
|
# set morphit_tor_key_src to the wizard's tor-hidden-service/ dir.
|
|
- role: tor
|
|
tags: [tor]
|
|
when: enable_tor | default(true)
|
|
|
|
# Every instance gets a basic .b32.i2p by default (the wizard generates
|
|
# one and the site advertises it via MORPHIT_INSTANCE_I2P_B32_ADDRESS →
|
|
# footer pill). This role makes i2pd actually SERVE it; set
|
|
# morphit_i2pd_key_src to the wizard's i2p-tunnel/ dir. An operable
|
|
# keyfile already on the server is preserved.
|
|
- role: i2pd
|
|
tags: [i2pd]
|
|
when: enable_i2pd | default(true)
|
|
|
|
# Every instance runs a small Kubo node that pins this instance's current
|
|
# signed release, so releases survive even if every commercial pinning
|
|
# service drops them (decentralization priority #2). ON by default; the
|
|
# CID comes from this instance's own /v1/release. Low-footprint (lowpower
|
|
# profile, capped connections, ~12 MB pinned).
|
|
- role: ipfs
|
|
tags: [ipfs]
|
|
when: enable_ipfs | default(true)
|
|
|
|
# ─── Optional sidecar monitors ────────────────────────────
|
|
# All disabled by default — set enable_<name>: true in
|
|
# group_vars/all.yml (or per-host in inventory) to opt in.
|
|
# Each emits structured JSON via systemd-cat that the
|
|
# matrix-bot picks up automatically.
|
|
|
|
- role: matrix_bot
|
|
tags: [matrix_bot, monitors]
|
|
when: enable_matrix_bot | default(false)
|
|
|
|
- role: host_monitor
|
|
tags: [host_monitor, monitors]
|
|
when: enable_host_monitor | default(false)
|
|
|
|
- role: smartctl_monitor
|
|
tags: [smartctl_monitor, monitors]
|
|
when: enable_smartctl_monitor | default(false)
|
|
|
|
- role: fail2ban_monitor
|
|
tags: [fail2ban_monitor, monitors]
|
|
when: enable_fail2ban_monitor | default(false)
|
|
|
|
- role: mdadm_monitor
|
|
tags: [mdadm_monitor, monitors]
|
|
when: enable_mdadm_monitor | default(false)
|
|
|
|
- role: dmesg_monitor
|
|
tags: [dmesg_monitor, monitors]
|
|
when: enable_dmesg_monitor | default(false)
|
|
|
|
- role: trivy_monitor
|
|
tags: [trivy_monitor, monitors]
|
|
when: enable_trivy_monitor | default(false)
|
|
|
|
- role: postfix_monitor
|
|
tags: [postfix_monitor, monitors]
|
|
when: enable_postfix_monitor | default(false)
|
|
|
|
- role: certbot_monitor
|
|
tags: [certbot_monitor, monitors]
|
|
when: enable_certbot_monitor | default(false)
|
|
|
|
- role: apt_monitor
|
|
tags: [apt_monitor, monitors]
|
|
when: enable_apt_monitor | default(false)
|
|
|
|
- role: compose_monitor
|
|
tags: [compose_monitor, monitors]
|
|
when: enable_compose_monitor | default(false)
|
|
|
|
- role: systemd_monitor
|
|
tags: [systemd_monitor, monitors]
|
|
when: enable_systemd_monitor | default(false)
|
|
|
|
- role: journald_monitor
|
|
tags: [journald_monitor, monitors]
|
|
when: enable_journald_monitor | default(false)
|
|
|
|
post_tasks:
|
|
# cp681 — start the deferred AIDE baseline build now, in post_tasks, AFTER
|
|
# every role has written its files (the morphit code under
|
|
# {{ morphit_repo_path }}, /etc/morphit/*.env, and the morphit-*.service/
|
|
# .timer units are all in place). Building the baseline from this settled
|
|
# state avoids hashing files mid-write and the spurious "file changed" alerts
|
|
# that would follow. The build itself runs in the background at idle priority
|
|
# (fire-and-forget, --no-block), so this does not slow the wizard; the
|
|
# one-shot service self-removes when the baseline is complete. Gated on the
|
|
# unit still being present, so it's a no-op when intrusion detection is off or
|
|
# the baseline already built and the service removed itself.
|
|
- name: Check for a pending AIDE baseline build
|
|
ansible.builtin.stat:
|
|
path: /etc/systemd/system/morphit-aide-init.service
|
|
register: post_aide_service
|
|
|
|
- name: Start the deferred AIDE baseline build in the background (after all install writes)
|
|
ansible.builtin.command: systemctl start --no-block morphit-aide-init.service
|
|
when: post_aide_service.stat.exists
|
|
changed_when: true
|
|
|
|
|
|
# .onion + .b32.i2p addresses only now. Capture them into
|
|
# morphit.config.env (which `morphit-ops register` + the indexer read) and
|
|
# restart the indexer so the footer pills + on-chain identity carry them.
|
|
# Best-effort: a node with tor/i2pd disabled, or a slow onion publish, just
|
|
# leaves the pills off — it never fails the run. On re-converge the morphit
|
|
# role slurp-preserves these, so lineinfile is a no-op (no restart churn).
|
|
- name: "cp663 #14 — capture Tor/i2p addresses into morphit.config.env"
|
|
when: (enable_tor | default(true)) or (enable_i2pd | default(true))
|
|
block:
|
|
- name: Wait for Tor to publish its onion hostname
|
|
ansible.builtin.wait_for:
|
|
path: "{{ morphit_tor_hs_dir | default('/var/lib/tor/morphit') }}/hostname"
|
|
timeout: 120
|
|
register: morphit_onion_wait
|
|
failed_when: false
|
|
when: enable_tor | default(true)
|
|
|
|
- name: Read the generated .onion address
|
|
ansible.builtin.slurp:
|
|
src: "{{ morphit_tor_hs_dir | default('/var/lib/tor/morphit') }}/hostname"
|
|
register: morphit_onion_slurp
|
|
failed_when: false
|
|
changed_when: false
|
|
when: enable_tor | default(true)
|
|
|
|
- name: Derive the .b32.i2p address from the i2pd keyfile
|
|
ansible.builtin.shell:
|
|
cmd: >-
|
|
set -o pipefail;
|
|
head -c 391 {{ ((morphit_i2pd_datadir | default('/var/lib/i2pd')) ~ '/' ~ (morphit_i2pd_keyfile | default('morphit-web.dat'))) | quote }}
|
|
| sha256sum | cut -d' ' -f1 | xxd -r -p | base32 | tr 'A-Z' 'a-z' | tr -d '=';
|
|
echo .b32.i2p
|
|
args:
|
|
executable: /bin/bash
|
|
register: morphit_i2p_b32
|
|
changed_when: false
|
|
failed_when: false
|
|
when: enable_i2pd | default(true)
|
|
|
|
- name: Write the .onion address into morphit.config.env
|
|
ansible.builtin.lineinfile:
|
|
path: "{{ morphit_repo_path }}/morphit.config.env"
|
|
regexp: '^MORPHIT_INSTANCE_TOR_ADDRESS='
|
|
line: "MORPHIT_INSTANCE_TOR_ADDRESS={{ (morphit_onion_slurp.content | b64decode).strip() }}"
|
|
owner: root
|
|
group: "{{ morphit_service_group }}"
|
|
mode: '0640'
|
|
when:
|
|
- enable_tor | default(true)
|
|
- morphit_onion_slurp.content is defined
|
|
- (morphit_onion_slurp.content | b64decode).strip() is search('\.onion$')
|
|
notify: Restart morphit-indexer
|
|
|
|
# Tor-only nodes have no clearnet domain, so their instance + relay
|
|
# origins were left empty at template time (the onion wasn't generated
|
|
# yet). Now that it exists, point all three at http://<onion> so the
|
|
# /v1/instance advertisement, invite links, web-push, and CORS all use
|
|
# the address the node is actually reachable at.
|
|
- name: "Tor-only — point instance + relay origins at the onion"
|
|
when:
|
|
- morphit_tor_only | default(false)
|
|
- morphit_onion_slurp.content is defined
|
|
- (morphit_onion_slurp.content | b64decode).strip() is search('\.onion$')
|
|
vars:
|
|
morphit_onion_origin: "http://{{ (morphit_onion_slurp.content | b64decode).strip() }}"
|
|
block:
|
|
- name: "Tor-only — MORPHIT_INSTANCE_ORIGIN = onion"
|
|
ansible.builtin.lineinfile:
|
|
path: "{{ morphit_repo_path }}/morphit.config.env"
|
|
regexp: '^MORPHIT_INSTANCE_ORIGIN='
|
|
line: "MORPHIT_INSTANCE_ORIGIN={{ morphit_onion_origin }}"
|
|
owner: root
|
|
group: "{{ morphit_service_group }}"
|
|
mode: '0640'
|
|
notify: Restart morphit-indexer
|
|
|
|
- name: "Tor-only — relay public origin + CORS allowlist = onion"
|
|
ansible.builtin.lineinfile:
|
|
path: /etc/morphit/relay.env
|
|
regexp: "{{ item.re }}"
|
|
line: "{{ item.line }}"
|
|
owner: root
|
|
group: "{{ morphit_service_group }}"
|
|
mode: '0640'
|
|
loop:
|
|
- re: '^MORPHIT_RELAY_PUBLIC_ORIGIN='
|
|
line: "MORPHIT_RELAY_PUBLIC_ORIGIN={{ morphit_onion_origin }}"
|
|
- re: '^MORPHIT_RELAY_ALLOWED_ORIGINS='
|
|
line: "MORPHIT_RELAY_ALLOWED_ORIGINS={{ morphit_onion_origin }}"
|
|
notify: Restart morphit-relay
|
|
|
|
- name: Write the .b32.i2p address into morphit.config.env
|
|
ansible.builtin.lineinfile:
|
|
path: "{{ morphit_repo_path }}/morphit.config.env"
|
|
regexp: '^MORPHIT_INSTANCE_I2P_B32_ADDRESS='
|
|
line: "MORPHIT_INSTANCE_I2P_B32_ADDRESS={{ morphit_i2p_b32.stdout | trim }}"
|
|
owner: root
|
|
group: "{{ morphit_service_group }}"
|
|
mode: '0640'
|
|
when:
|
|
- enable_i2pd | default(true)
|
|
- morphit_i2p_b32.stdout is defined
|
|
- (morphit_i2p_b32.stdout | trim) is search('\.b32\.i2p$')
|
|
- (morphit_i2p_b32.stdout | trim | length) >= 60
|
|
notify: Restart morphit-indexer
|
|
|
|
|
|
# believing they are monitored when they are not.
|
|
#
|
|
# `enable_matrix_bot` CANNOT default to true: the role asserts on
|
|
# `matrix_bot_access_token`, a secret only the operator can mint, so
|
|
# flipping the default would fail every fresh install at the assert
|
|
# instead of producing alerts. The monitors default off with it, since
|
|
# a monitor with nowhere to report is just a timer burning CPU.
|
|
#
|
|
# So the default is right and the DOCS were the gap: nothing in the run
|
|
# told a new operator that their node is silent. Ansible printing green
|
|
# is exactly when someone stops reading. This block is the last thing
|
|
# they see.
|
|
- name: Warn when the node has no alerting
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
─────────────────────────────────────────────────────────
|
|
⚠ THIS NODE HAS NO ALERTING.
|
|
|
|
enable_matrix_bot is false, so nothing will tell you when
|
|
this node runs out of disk, fails a backup, gets its TLS
|
|
cert expired, or falls over at 3am. You will find out
|
|
from your users.
|
|
|
|
It is off by default because it needs a secret we cannot
|
|
generate for you — a Matrix access token. Ten minutes of
|
|
setup, once:
|
|
|
|
1. Make a Matrix account for the bot (any homeserver).
|
|
2. Get its access token.
|
|
3. Put it in group_vars/vault.yml (encrypted):
|
|
vault_matrix_bot_access_token: "syt_..."
|
|
4. In group_vars/all.yml set:
|
|
enable_matrix_bot: true
|
|
matrix_bot_homeserver: "https://matrix.org"
|
|
matrix_bot_alert_mxid: "@you:matrix.org"
|
|
5. Turn the monitors on — they report THROUGH the bot,
|
|
so they stay quiet until it exists:
|
|
enable_host_monitor: true
|
|
enable_smartctl_monitor: true
|
|
enable_certbot_monitor: true
|
|
enable_apt_monitor: true
|
|
enable_systemd_monitor: true
|
|
6. Re-run this playbook with --tags monitors.
|
|
|
|
Running a Morphit node unmonitored is a choice. Make it
|
|
on purpose, not by not reading.
|
|
─────────────────────────────────────────────────────────
|
|
when: not (enable_matrix_bot | default(false))
|
|
|
|
- name: Print verification reminder
|
|
ansible.builtin.debug:
|
|
msg: |
|
|
─────────────────────────────────────────────────────────
|
|
Ansible reported success. This does NOT mean Morphit is
|
|
working or that the host is secure.
|
|
|
|
FIRST, the one-command check — it actively verifies the
|
|
indexer is syncing (in parallel across your RPC nodes), the
|
|
relay is up, the matrix-bot is alerting the address you
|
|
entered, backups are scheduled, HTTPS/TLS is valid, and the
|
|
AIDE baseline built:
|
|
|
|
sudo morphit-ops → option 13 (Node health)
|
|
|
|
Everything green there means the live services came up.
|
|
THEN run the security/host verification checklist in
|
|
morphit-sysadmin-handoff.txt:
|
|
|
|
- ssh root@host fails
|
|
- ssh user@host with password fails
|
|
- nmap shows only 22, 80, 443 open externally
|
|
- psql -h <public-ip> -U morphit times out
|
|
- X-Forwarded-For spoof test from an untrusted IP fails
|
|
- systemctl status morphit-relay morphit-indexer
|
|
- curl https://{{ morphit_domain }}/v1/instance returns
|
|
expected JSON
|
|
- Diamond-hardened squatter env vars all present in
|
|
/etc/morphit/relay.env
|
|
- Daily backup timer is scheduled (systemctl list-
|
|
timers | grep morphit-backup)
|
|
- AIDE database building in the background, then self-removes
|
|
(systemctl status morphit-aide-init — absent once the baseline
|
|
at /var/lib/aide/aide.db is built)
|
|
- auditd is logging
|
|
|
|
If you enabled the matrix-bot / monitor sidecars:
|
|
- systemctl status morphit-matrix-bot (if enable_matrix_bot)
|
|
- systemctl list-timers | grep morphit- (host/smartctl/
|
|
fail2ban/mdadm timers should fire on schedule)
|
|
- sudo journalctl -t morphit-host-monitor --since '10
|
|
minutes ago' should show structured JSON
|
|
- sudo journalctl -u morphit-matrix-bot should show
|
|
the bot ready + DM-on-alert log line
|
|
─────────────────────────────────────────────────────────
|