Infrastructure Overview

Specter's off-chain infrastructure consists of 11 services running on a single relayer droplet, managed by PM2 and exposed through a Caddy reverse proxy at relayer.specterchain.com. These services handle everything from Merkle tree synchronization and ZK proof generation to cross-chain bridging and gasless transaction submission.

Architecture

                        ┌─────────────────────────┐
                        │    relayer.specterchain  │
                        │         .com             │
                        │      (Caddy HTTPS)       │
                        └────────────┬────────────┘
                                     │
                    Path-based routing to internal ports
                                     │
        ┌────────────────────────────┼────────────────────────────┐
        │            ┌───────────────┼───────────────┐            │
        ▼            ▼               ▼               ▼            ▼
   ┌────────┐  ┌────────┐     ┌────────┐      ┌────────┐   ┌────────┐
   │ :3001  │  │ :3002  │     │ :3003  │      │ :3005  │   │ :3010  │
   │ Root   │  │Commit  │     │ Proof  │      │Faucet  │   │ Ember  │
   │Updater │  │Relayer │     │Relayer │      │        │   │ Proxy  │
   └────────┘  └────────┘     └────────┘      └────────┘   └────────┘
        │                          │
        │  ┌────────┐              │         ┌────────┐
        │  │ :3009  │              │         │  No    │
        │  │ Token  │              │         │ Port   │
        │  │Registry│              │         │        │
        │  └────────┘              │    ┌────┴────────┴────┐
        │                          │    │ Offline Relayer   │
        │                          │    │ Base Conversion   │
   ┌────┴─────────┐                │    │ Hyperlane Bridge  │
   │ Open Ghost   │                │    │ Multi-Chain       │
   │ Root Updater │                │    └───────────────────┘
   └──────────────┘                │
                                   │
                              Specter Chain
                             (Chain ID 5446)

Service Directory

Service	Port	Status	Description
ghost-root-updater	3001	Active — 131 commitments synced to block ~883495	Listens for `Committed` events on `CommitmentTree`, maintains an incremental Poseidon Merkle tree, and submits updated roots on-chain via `updateRoot()`
ghost-commitment-relayer	3002	Healthy	HTTP API for server-side Poseidon hash computation. Provides Poseidon2, Poseidon4, and Poseidon7 endpoints for mobile clients that cannot run circomlibjs efficiently
ghost-proof-relayer	3003	Active — ~13.5s avg proof time, 4 proofs generated on testnet	Loads Groth16 circuit files and generates ZK proofs from private inputs. Returns Groth16 proof + public signals for on-chain verification
ghost-faucet	3005	Active — 56.9M GHOST balance	Testnet faucet dispensing 100 GHOST per drip with a 24-hour cooldown per address
ghost-ember-proxy	3010	Active	Persistent phantom key access proxy with server-side Access Proof generation. Also proxies AI chat requests to Anthropic's API
ghost-token-registry	3009	Active	Tracks deployed GhostERC20 tokens and serves metadata (name, symbol, decimals, icon) to the webapp
ghost-offline-relayer	—	Active	Gasless/sponsored transaction relayer. Submits reveal transactions on behalf of users who lack GHOST for gas
base-conversion-relayer	—	Active	Watches for `TokensConverted` events on Base chain and mints corresponding g-tokens on Specter via `GhostMinter`
hyperlane-bridge-relayer	—	Active	Monitors `Dispatch` events on source chains and delivers Hyperlane messages to Specter. Bidirectional Base-Specter bridging
hyperlane-multi-chain-relayer	—	Active	Extends Hyperlane bridging to Ethereum, Base, and Arbitrum simultaneously
open-ghost-root-updater	—	Active	Separate Merkle tree instance for OpenGhost/Revels data privacy commitments. Targets `OpenCommitmentTree` contract

Process Management

All services are managed by PM2, a production process manager for Node.js applications. PM2 provides:

Automatic restarts: If a service crashes, PM2 restarts it immediately with configurable retry backoff.
Log management: Stdout and stderr are captured per-service under ~/.pm2/logs/. Logs are rotated to prevent disk exhaustion.
Startup persistence: The PM2 process list is saved and restored on system reboot via pm2 startup and pm2 save.
Monitoring: pm2 monit provides real-time CPU/memory usage per service. pm2 status shows uptime and restart counts.

Common PM2 commands for operating the relayer:

# View all services
pm2 status

# View logs for a specific service
pm2 logs ghost-proof-relayer

# Restart a service
pm2 restart ghost-root-updater

# Reload all services (zero-downtime)
pm2 reload all

Reverse Proxy

Caddy serves as the HTTPS reverse proxy at relayer.specterchain.com. It handles:

TLS termination with automatic certificate provisioning and renewal via Let's Encrypt.
Path-based routing to internal service ports (e.g., /root-updater/* routes to localhost:3001).
CORS headers for cross-origin requests from the Specter webapp.
Request logging for debugging and audit purposes.

See Caddy Routing for the full routing table.

Network Topology

All services run on a single DigitalOcean droplet. This simplifies deployment and inter-service communication (all traffic is localhost) but creates a single point of failure. The services communicate with the Specter chain via an RPC endpoint and with external chains (Base, Ethereum, Arbitrum) via their respective RPC providers.

┌─────────────────────────────────────────────────┐
│               Relayer Droplet                    │
│                                                  │
│  PM2 ──▶ [11 services on localhost ports]        │
│  Caddy ──▶ HTTPS termination + routing           │
│                                                  │
│  Outbound connections:                           │
│    → Specter RPC (chain ID 5446)                 │
│    → Base RPC                                    │
│    → Ethereum RPC                                │
│    → Arbitrum RPC                                │
│    → Anthropic API (for Ember chat proxy)        │
└─────────────────────────────────────────────────┘

Health Monitoring

Each HTTP-accessible service exposes a health endpoint (typically GET /health or GET /) that returns service status, uptime, and relevant metrics. These endpoints are used for:

Liveness checks: Caddy and PM2 can detect unresponsive services.
Dashboard metrics: The webapp queries health endpoints to display system status.
Alerting: External monitoring can poll these endpoints to detect outages.

Example health response from the root updater:

{
  "status": "healthy",
  "commitmentCount": 131,
  "lastSyncedBlock": 883495,
  "merkleRoot": "0x1a2b3c...",
  "uptime": 864000
}

Operator Keys

Several services require operator wallet keys to submit on-chain transactions:

Service	On-Chain Action
ghost-root-updater	`updateRoot()` on `CommitmentTree`
open-ghost-root-updater	`updateRoot()` on `OpenCommitmentTree`
ghost-offline-relayer	`reveal()` on `CommitRevealVault`
ghost-faucet	Native GHOST transfers
base-conversion-relayer	`mint()` on `GhostMinter`
hyperlane-bridge-relayer	`process()` on Hyperlane `Mailbox`
hyperlane-multi-chain-relayer	`process()` on Hyperlane `Mailbox` (multiple chains)

Operator keys are stored in environment variables or encrypted keystores on the droplet. Each service uses a dedicated key to isolate risk — a compromised faucet key cannot submit root updates.

Architecture​

Service Directory​

Process Management​

Reverse Proxy​

Network Topology​

Health Monitoring​

Operator Keys​