morphit/docs/AUTOMATION-AUDIT.md

15 KiB
Raw Permalink Blame History

Morphit — automation audit of manual interventions (Q6)

Last reviewed: 2026-05-01

This catalog enumerates every place in Morphit where a human operator, admin, or user has to do something manually that could potentially be automated. For each item: the current state, the automation potential (HIGH / MEDIUM / LOW), the cost, and the recommendation.

The goal isn't to automate everything — some manual gates exist deliberately to preserve operator control over expensive, irreversible, or high-trust actions. The goal is to surface where automation would meaningfully reduce operator load without increasing risk.

This audit complements docs/OPERATIONS.md (the operator runbook) and docs/RUN-A-MORPHIT-NODE.md (the new-operator walkthrough). Both must stay aligned with this catalog.


1. Operator-side manual tasks

1.1 Weekly ACT minting ceremony — REMOVED (beta.28 / cp329)

There is no ACT minting task any more, so there is nothing to automate here. Blurt disabled claim_account / create_claimed_account at hard fork 2, so Account Creation Tokens cannot be minted at all. As of beta.28 (ADR-0010, as amended 2026-06) the relay creates each account with a direct account_create op, paying the ~100 BLURT account_creation_fee inline from its liquid BLURT at signup time. The mint-acts.ts script, the morphit-relay-mint-acts.{service,timer} units, and the MORPHIT_RELAY_WEEKLY_ACT_COUNT / MORPHIT_RELAY_AUTOMINT_* config knobs have all been removed.

Operator-side, the only standing task is funding — keep @morphit-relay topped up with enough liquid BLURT to cover signups (see OPERATIONS §0a for the sizing math and §16 for the relay_low_balance_for_signups Matrix alert). That funding is itself "automatable" via a recurrent on-chain transfer (§1.2), and the low-balance alerting is already automated (§1.6), so no new automation is owed here.

STATUS: Implemented 2026-05-01. See OPERATIONS.md §2 "Unattended mode" for the operator-facing setup steps.


1.1-archive: original framing (preserved for ADR-0010 traceability)

Why manual originally: ADR-0010 §4 framed this as a deliberate operator-in-the-loop step so the relay's working BLURT balance stays small and a compromised relay can't drain unbounded BLURT into ACTs. The 2026-05 review concluded the caps still hold under timer-driven minting (§1.1 above), so the in-the-loop confirmation became a cron job.


1.2 Recurrent BLURT top-up to relay — ALREADY AUTOMATED

Today: ADR-0010 §3 ships a recurrent_transfer op the operator broadcasts ONCE at setup time. Blurt natively recurses the transfer weekly for executions=52 (one year).

Why this is good: Native chain primitive. No off-chain cron, no hot key on a server. The operator re-broadcasts once a year (or sooner if their fee-account address changes) — that's a quarterly-or-better cadence, not weekly.

STATUS: Already automated; no work to do.


1.3 TLS certificate renewal — ALREADY AUTOMATED

Today: certbot installs a systemd timer that runs twice daily and auto-renews when the cert is within 30 days of expiry.

Recommended addition: a weekly cron.weekly job that checks expiry and pages the operator if certs ARE near expiry but renewals haven't fired (catches certbot configuration drift). This is in OPERATIONS.md §16 (the morphit-certbot-monitor renewal-stall detector) already.

STATUS: Already automated; the optional alert wrapper is documented but not shipped as a default unit.


1.4 Witness fee change response — MEDIUM automation potential

Today: When Blurt witnesses change account_creation_fee, the operator sees an alert and manually updates MORPHIT_INDEXER_ACCOUNT_CREATION_FEE_BLURT and MORPHIT_RELAY_ACCOUNT_CREATION_FEE_BLURT env vars, restarts services.

Why manual today: Setting a wrong fallback value silently breaks signup. Operator-in-the-loop catches misreads.

Why automation is constrained: The fallback is exactly that — a fallback for when chain RPCs are unreachable. Auto-updating it from the chain would create a self-referential bootstrap problem if the auto-update path itself relies on chain RPCs.

Recommended partial automation:

  • Indexer + relay should both observe account_creation_fee changes via their normal block-walk and EMIT a structured log line at WARN level when the observed fee differs from the configured fallback by more than 10%. This is already partly there for the indexer (chain-fee tracker); not yet for the relay.
  • Operator gets an alert via their journald-watching pipeline when the warning fires; manual restart is fast.

Cost: ~20 lines (relay-side warn-log of fee divergence).

STATUS: Both shipped. Indexer-side tracker existed at audit time; relay-side warn-log shipped in Part 117 — apps/relay/src/blurt/client.ts:157 defines FEE_DIVERGENCE_WARN_THRESHOLD = 0.1 and analyzeFeeDivergence() at line 168 emits a structured chain_props_account_creation_fee_diverges_from_config warn-log when the observed fee differs from the configured fallback by more than 10%, throttled once-per-process-startup. Smoke regression at apps/relay/scripts/fee-divergence-smoke.ts registered in scripts/run-smokes.sh.


1.5 Stale price feed alert — ALREADY AUTOMATED

Today: Indexer's price source emits stale=true after N minutes without a successful upstream fetch. The frontend hides the USD echo when stale. Operator sees journald price.fetch.failed lines.

STATUS: Already automated; alerting is via the operator's normal journald pipeline.


1.6 Operator-account balance monitoring — ALREADY AUTOMATED

Today: Indexer scans operator-fees balances and emits structured alerts when below operator-configured thresholds. Documented at OPERATIONS.md §16.

STATUS: Already automated.


1.7 Relay queue stuck — MEDIUM automation potential

Today: A signup queue entry can get stuck (network errors, RC depletion, etc.). The operator manually nulls the broadcast_at to retry.

Why manual today: Stuck entries can mask deeper relay problems (RC depletion, key issues, broken endpoints). An automated retry loop could mask these.

Recommended partial automation:

  • Auto-retry up to 3 times with exponential backoff (1m, 5m, 15m). After 3 failures, the entry stays stuck and waits for operator review.
  • Today the relay does retry on transient network errors but not on the stuck-entry flagged-for-manual case.

Cost: ~80 lines (queue-entry retry counter + scheduler).

STATUS: Documented as a future hardening pass. Not urgent — operator manual flow is rare and well-documented.


1.8 Schema migrations — ALREADY AUTOMATED

Today: Indexer auto-runs migrations on boot via apps/indexer/src/db/migrations.ts. v17 has a brief sequential-scan window during the upgrade documented at OPERATIONS.md §21; otherwise transparent to the operator.

STATUS: Already automated.


Today: OPERATIONS.md §37.2 recommends unattended-upgrades for security patches. Operator chooses to enable.

STATUS: Documented; not enforced by Morphit's installer because the installer doesn't exist as a single thing — the operator runs through RUN-A-MORPHIT-NODE.md. The walkthrough SHOULD include enabling unattended-upgrades as a checklist item. (See §3.3 below.)


2. User-side manual tasks (Q9 setup-checklist territory)

These are the user/operator's checklist responsibilities during account setup or daily use. Q9's "operator setup checklist + nudges" answer expands on this — see that section.

2.1 Backup the 12-word seed — AUTOMATED 2026-05-01

Today: Signup flow displays the seed once with a "I've saved it" gate. Plus, after 7 days of active use on this device, a soft amber banner appears at the top of every page asking "have you backed up your seed somewhere durable?" with a "Show me how →" link to the recovery FAQ and a one-tap permanent dismissal.

Implementation:

  • localStorage anchor morphit.keystore.first_persist_at stamped on first successful writeEnvelope (in apps/web/src/lib/crypto/persistentKeystore.ts).
  • Banner component SeedBackupNudge.svelte calls shouldShowSeedBackupNudge() on mount; renders only when ≥7 days since first persist AND not dismissed.
  • "Got it" sets morphit.keystore.backup_nudge_dismissed = '1', hiding the banner permanently on this device.
  • Full sign-out (clearKeystore) wipes both anchors so re-onboarding restarts the prompt schedule.

STATUS: Shipped.


2.2 Verify counterparty's chat-pub on first contact — SHIPPED

Today: apps/web/src/lib/components/VerifyPeerPanel.svelte renders an explicit Verify-peer panel inside ConversationView showing the deterministic (my_chat_pub, peer_chat_pub) fingerprint pair (computed in apps/web/src/lib/chat/fingerprint.ts). Users compare fingerprints out-of-band (voice, in-person, side-channel) and confirm. That's a manual user action by design — the machine can't out-of-band compare.

STATUS: Shipped (panel + fingerprint helper + locale strings in all 10 locales).


2.3 Mark trade complete + leave feedback — MANUAL BY DESIGN

Today: After receiving funds, the buyer/seller manually clicks "Trade complete" and "Leave feedback". This is deliberately manual — Morphit doesn't run any wallet, so "funds received" is something only the user can attest to.

STATUS: Cannot be automated without breaking the non-custodial design. Already as automated as it can be.


3. Operator setup tasks (Q9 territory — feeds into the checklist)

3.1 Enable unattended-upgrades — should be in setup checklist

Cost: 1 command in the walkthrough.

STATUS: Documented in OPERATIONS.md §37.2. Should be flagged in ops-cli init as a checkbox + ufw / fail2ban / SSH key-only mode.

3.2 Configure systemd timers for auto-tasks — should be in setup checklist

The weekly ACT-mint timer referenced here historically was removed at beta.28 (see §1.1). The remaining auto-task timers the setup wizard should install + enable alongside the relay are the monitoring/backup units (TLS renewal, balance alerts, backups) — none of them touch account creation, which now pays its fee inline per signup.

3.3 ops-cli should verify everything's in place — Q9 deliverable

The ops-cli init wizard already does: chain RPC reachability, account existence check, key-file perms check, fee-address viewkey verification, etc. It should ALSO run a final checklist verification step that pings each of these and prints " done" or "⚠ missing — run X to fix":

Check Today Recommended
morphit-fees account exists done Keep
Active-key file mode 0400 done Keep
Indexer DB connection works done Keep
RPC endpoint reachability done Keep
Relay BLURT balance > floor done Keep
TLS cert valid + auto-renew configured done Keep
unattended-upgrades installed + enabled not yet Add
ufw enabled + correct ruleset not yet Add
SSH PasswordAuthentication no not yet Add
fail2ban installed + sshd jail enabled not yet Add
journald disk cap configured not yet Add
weekly-mint timer installed (§1.1 shipped 2026-05-01) not yet Add
witness-fee-divergence warn-log configured not yet Add
frontend CSP headers verified done (§15) Keep
Tor onion mirror reachability not yet Add

This expanded checklist becomes Q9's deliverable — see the Q9 plan in this session's notes.


4. What we deliberately do NOT automate

A list of "could automate, but shouldn't":

  • Trade matching. Buyers and sellers pick each other based on reputation, terms, location. Auto-matching breaks the trust model.
  • Dispute resolution. Morphit has no arbitrator role (FAQ no_escrow_arbitration). A bot that "decides who's right" would be a centralization vector.
  • Fund release. Non-custodial means the seller releases funds when the buyer pays. No bot can decide when payment has arrived in a non-custodial flow — only the human actually receiving the funds knows.
  • Reputation moderation. Feedback is permanent and signed. A bot that "removes spam reviews" is a centralization vector. (Operators can suppress display on their own instance; the chain record is permanent.)
  • Owner-key rotation. OPERATIONS.md §8 — must be human with cold-storage backups verified.
  • Initial ACT-key passphrase entry on relay first boot. Documented in OPERATIONS.md §3. Subsequent reboots can use systemd LoadCredentialEncrypted= if the operator chooses; first boot is human.

5. Cross-cutting automation ideas

5.1 Health endpoints for external monitoring

The indexer already exposes /v1/health. The relay should too (today it has a /v1/health but coverage is thin). Operators can wire these into their existing monitoring (Uptime Kuma, Statping, etc.).

5.2 Structured log shipping

Both indexer and relay log to journald. Operators wanting remote log aggregation can ship via journald → Vector → any sink. Documented in OPERATIONS.md.

5.3 Automated backup of operator config + key envelopes

Encrypted active-key envelopes are at /etc/morphit/*.key.enc. The operator should back these up off-host (not the relay's own filesystem). OPERATIONS.md should add a recommendation for an automated nightly rsync to a backup destination, encrypted at rest. Currently silent on this.


Summary table

# Task Auto today? Recommend automating? Effort
1.1 Weekly ACT minting Removed (beta.28) n/a — task no longer exists done
1.2 Recurrent BLURT top-up Native chain n/a done
1.3 TLS renewal certbot n/a done
1.4 Witness fee change response Manual 🟡 Partial — log warn ~20 lines
1.5 Stale price feed stale flag + log n/a done
1.6 Operator-account balance scanner + alerts n/a done
1.7 Relay queue stuck Manual 🟡 Partial — auto-retry 3× ~80 lines
1.8 Schema migrations on-boot n/a done
1.9 OS package updates 🟡 Documented Add to setup checklist doc only
2.1 Backup-seed nudge Manual Add a 7-day prompt ~30 lines
2.2 Counterparty pub verify Manual UX gap 🟡 Tracked (Option 6) ~150 lines
2.3 Mark trade complete Manual by design Cannot — non-custodial n/a
3.x Setup checklist 🟡 Partial Q9 deliverable ~200 lines

Next steps

  1. Implement §3.3 (expanded setup checklist) as part of Q9.
  2. Implement §2.1 (7-day backup-seed nudge). Small.
  3. Document §1.9 + §5.3 in OPERATIONS.md and RUN-A-MORPHIT-NODE.md.

These can be picked up in any order; they don't depend on each other.