Skip to main content

​
Build (CI on every push)

The Containerfile is a stagex multi-stage build, chosen specifically so the result is reproducible — this is the prerequisite for the Nautilus claim that anyone can rebuild the source and get the same PCRs: 
1. Pinned stagex base images (musl toolchain, rust, busybox, socat)
2. node:22-alpine build stage → musl-linked node binary + libstdc++/libgcc_s
3. npm ci --omit=dev against scripts/package.json (pinned sidecar deps)
4. cargo build --release --target x86_64-unknown-linux-musl
   for init, coordinator, and the support crates
5. Initramfs assembly: kernel module, busybox, socat, Rust binaries,
   node binary + libs + scripts dir
6. eif_build → coordinator.eif + coordinator.pcrs
Rebuilding the same source tree produces the same coordinator.pcrs byte-for-byte. Those PCRs are what EnclaveConfig<ENCLAVE> is updated to via update_pcrs before the first register_enclave succeeds — see On-chain contracts.

​
EC2 deployment

Triggered by every push to main. Workflow: .github/workflows/deploy.yml. 
GitHub Actions
  ├─ scp source to EC2
  ├─ write ~/.env.runtime from PINAIVU_ENV_FILE secret
  ├─ install docker, nitro-cli, socat, jq, node, npm on EC2 (idempotent)
  ├─ docker build → coordinator.eif + coordinator.pcrs in ./out
  ├─ stop previous bridges, pkill any leftover manual socats
  ├─ terminate previous enclave, compact memory, restart allocator
  │                        (NOT just `start` — restart so memory_mib changes apply)
  ├─ nitro-cli run-enclave --cpu-count 2 --memory 4096 --eif-path …
  ├─ register systemd units for the host-side socat bridges:
  │       pinaivu-bridge-http        TCP:4000 → VSOCK:CID:4000
  │       pinaivu-bridge-libp2p      TCP:4001 → VSOCK:CID:4001
  │       pinaivu-logs               VSOCK:5000 → /tmp/coordinator.log
  │       pinaivu-outbound-postgres  VSOCK:8101 → TCP:supabase
  │       pinaivu-outbound-redis     VSOCK:8102 → TCP:upstash
  ├─ awk 1 ~/.env.runtime ~/.env.runtime.dynamic | socat - VSOCK:CID:7000
  │                       # concatenation must be newline-terminated — see
  │                       # the .env.runtime vs .env.runtime.dynamic note below
  ├─ wait for /health, then re-check uptime after 60s
  │                       (catches coordinators that pass /health then die)
  └─ register coordinator on Sui via scripts/register-coordinator.sh
     (full steps in On-chain contracts → Coordinator registration)

​
Inside the enclave at boot

From src/init/src/main.rs: 
init (PID 1) runs:
  1. Mount /proc, /sys, /dev, /tmp, cgroups
  2. NSM heartbeat (signals nitro-cli the enclave is alive)
  3. Insert nsm.ko, seed kernel entropy
  4. Bring up loopback
  5. vsock_accept(7000) — blocks until parent pushes the env file
  6. Apply env defaults (PINAIVU_BIND, PINAIVU_LIBP2P_LISTEN, ...)
  7. Generate SIDECAR_SECRET if not set (random 32 bytes from NSM RNG)
  8. Write /etc/hosts entries mapping the real Postgres/Redis/Sui RPC
     hostnames to 127.0.0.1 (TLS SNI must see the real name; bridge
     forwards bytes)
  9. socat bridges:
       TCP-LISTEN:5432   → VSOCK:3:8101   Postgres outbound
       TCP-LISTEN:6379   → VSOCK:3:8102   Redis outbound
       TCP-LISTEN:443    → VSOCK:3:8103   Sui RPC outbound (HTTPS)
       VSOCK-LISTEN:4000 → TCP:127.0.0.1:4000   HTTP inbound
       VSOCK-LISTEN:4001 → TCP:127.0.0.1:4001   libp2p inbound
 10. Truncate /tmp/coordinator.log; open append-only for both
     sidecar and coordinator (O_APPEND on both writers avoids
     race-corrupting each other's line-sized appends)
 11. Spawn TS sidecar on 127.0.0.1:8200 (loopback only) — reads
     PINAIVU_ENCLAVE_OBJECT_ID at startup so it can settle without an
     HTTP push if the dynamic env file already holds an id
 12. Spawn coordinator binary; stdout+stderr → /tmp/coordinator.log
 13. log_forwarder thread re-opens the file every ~100ms, tracks its own
     position, writes new bytes to VSOCK:5000 via libc::send(MSG_NOSIGNAL)
 14. wait(coordinator)
 15. reboot()   # exits the enclave on any process exit

​
~/.env.runtime vs ~/.env.runtime.dynamic

  • ~/.env.runtime is overwritten on every deploy from the PINAIVU_ENV_FILE GitHub Actions secret. Static config.
  • ~/.env.runtime.dynamic is written by post-boot host scripts (e.g. register-coordinator.sh). Holds values discovered after enclave start — currently just PINAIVU_ENCLAVE_OBJECT_ID. Survives across deploys.
  • The VSOCK:7000 push concatenates both with awk 1 so every line ends in \n — a plain cat A B can glue the last line of A onto the first line of B, silently corrupting whichever variable sits at that boundary.

​
Local dev

Running the coordinator outside an enclave still works (mock NSM attestation): 
cd ~/projects/pinaivu/coordinator
DATABASE_URL=postgres://... REDIS_URL=redis://localhost:6379 \
SIDECAR_SECRET=local-dev \
cargo run -p coordinator
Skip the sidecar in dev — registration just logs a warning and the HTTP server keeps serving. The inference flow doesn’t depend on it.

​
Smoke testing prod

EC2_IP=<your IP>
curl -s http://$EC2_IP:4000/health           # → "ok"
curl -s http://$EC2_IP:4000/enclave_health   # → { pubkey, peer_id, uptime_ms }

Environment variables

Required env vars for the coordinator, gateway, chat-relayer, and explorer-indexer

Full E2E smoke test

Reproduce a real settled inference, with troubleshooting for common failures