End-to-End Smoke Test

The first fully working E2E run landed at Sui testnet tx 5ePLsmVqcCAzFmpLz9XAQaKpsDFWbjWHJtSfkNoLUwMe. This walks through reproducing it.

Prerequisites

Piece	Where	Required
Coordinator running in a Nitro Enclave on EC2	e.g. `13.206.80.190:4000`	`curl /health` returns `ok`
`/enclave_health` returns a non-null `enclave_object_id`	same	means `register_enclave` already landed on-chain
Vault on Sui testnet has a positive `treasury_balance<SUI>`	`pinaivu::vault`	`top_up` was called once after publish
Local Ollama running	`127.0.0.1:11434`	`curl /api/tags` lists a model like `llama3.2:1b`
Local `pinaivu-node` built	`~/projects/pinaivu/node/target/release/pinaivu-node`	`cargo build --release`

Step 1 — get the live coordinator’s peer_id

curl -s http://$EC2_IP:4000/enclave_health | jq -r .peer_id
# e.g. 12D3KooWMSsNbsLWeLjdNRdFAhyGd488HGTqQpkQGREvUN7GN5zG

Every deploy generates a fresh enclave key — and therefore a fresh peer_id — so a stale value from an earlier deploy won’t dial through.

Step 2 — start the node

PEER=$(curl -s http://$EC2_IP:4000/enclave_health | jq -r .peer_id)
pkill -f pinaivu-node 2>/dev/null

nohup ~/projects/pinaivu/node/target/release/pinaivu-node \
  --coordinator-addr  /ip4/$EC2_IP/tcp/4001/p2p/$PEER \
  --coordinator-http  http://$EC2_IP:4000 \
  --listen            127.0.0.1:5000 \
  --advertise-url     http://127.0.0.1:5000 \
  --model             llama3.2:1b \
  --payout-address    <your Sui address> \
  > /tmp/pinaivu-node.log 2>&1 &
disown

tail -5 /tmp/pinaivu-node.log   # wait for "connection established peer=<peer_id>"

--advertise-url 127.0.0.1:5000 only works when the client runs on the same machine. For a remote client, expose port 5000 (ngrok or a public IP) and pass that URL so the coordinator’s response gives the client a reachable node_url.

Step 3 — client submits the request

RESP=$(curl -s -X POST http://$EC2_IP:4000/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model": "llama3.2:1b",
    "messages": [{"role":"user","content":"say hi"}],
    "client_pubkey_hex": "'$(printf '01%.0s' {1..32})'"
  }')
echo "$RESP" | jq '{request_id, node_url}'

REQ=$(echo "$RESP" | jq -r .request_id)
TOKEN=$(echo "$RESP" | jq -c .dispatch_token)
NODE_URL=$(echo "$RESP" | jq -r .node_url)

What this exercises: the coordinator publishes InferenceRequest on /pinaivu/inference/any/1.0.0, the dialed node bids on /pinaivu/bids/1.0.0, the coordinator runs its 200ms auction window, picks the node, and signs the DispatchToken with its enclave key.

Step 4 — client posts the prompt to the node

curl -s -X POST "$NODE_URL/v1/inference" \
  -H 'content-type: application/json' \
  -d "{\"prompt\":\"say hi\",\"dispatch_token\":$TOKEN}" | jq .

Expected:

{
  "request_id": "<uuid>",
  "content": "Hi. How can I assist you today?",
  "output_tokens": 8,
  "latency_ms": 1093
}

The node verified the dispatch token’s signature against the coordinator_pubkey it cached from /enclave_health at startup, called Ollama, built a ProofOfInference, wrapped it in a signed CompletionAck, and sent it to the coordinator over /pinaivu/completion/1.0.0. The coordinator verified every signature, signed a RoutingReceipt (BCS IntentMessage), stored it, inserted a pending row into payments, and enqueued a settlement job.

Step 5 — wait for settlement

sleep 12   # apalis poll + sidecar PTB + Sui finality

SECRET=$(grep -E '^SIDECAR_SECRET=' ~/projects/pinaivu/coordinator/.env | cut -d= -f2)
curl -s "http://$EC2_IP:4000/v1/admin/settlements/$REQ" \
  -H "x-sidecar-secret: $SECRET" | jq .

Expected (status submitted or confirmed):

{
  "request_id": "<uuid>",
  "payments": [{
    "status": "submitted",
    "tx_digest": "5ePLsmVqcCAzFmpLz9XAQaKpsDFWbjWHJtSfkNoLUwMe",
    "submitted_at": "2026-05-21T18:58:23.134765+00:00",
    "amount_nanox": 1000000,
    "payee_sui_address": "0x..."
  }]
}

Step 6 — verify on Sui

PKG=<package id>

curl -s -X POST https://fullnode.testnet.sui.io \
  -H 'content-type: application/json' \
  -d "{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"suix_queryEvents\",
       \"params\":[{\"MoveEventType\":\"$PKG::vault::Settled\"},null,3,true]}" \
  | jq '.result.data[] | {tx: .id.txDigest, parsedJson}'

The latest event should show request_id matching your UUID’s 16 raw bytes, payee matching your payout address, amount matching what you expected, and tx matching the digest from Step 5.

Troubleshooting

Symptom	Diagnose with	Likely cause
Coordinator boot stalls at `CHK 05.1`	`tail /tmp/coordinator.log` on EC2	Postgres connect: IPv6-only host, missing `sslmode=require`, unescaped `@` in password (`%40`)
`Bridges active … socat … Network is unreachable`	`journalctl -u pinaivu-outbound-postgres`	socat resolving AAAA on an IPv4-only EC2 host — use `TCP4:`, not `TCP:`
Admin endpoint returns 401 with matching secrets	grep `admin secret mismatch` in coordinator log	env file concatenated by `cat` without a trailing newline — use `awk 1`
`CommandArgumentError { kind: InvalidUsageOfPureArg }`	sidecar log	passing a typed `vector<Payout>` as `tx.pure.vector("u8", …)`; build it via `receipts::new_payout` + `makeMoveVec`
`MoveAbort code 1 in vault::settle`	sidecar log	`EInvalidReceipt`: signed bytes diverge from on-chain BCS — most often a UUID encoded as a 36-byte string instead of 16 raw bytes
`MoveAbort code 2 in vault::settle`	sidecar log	`EPayeeNotInReceipt`: settle’s `(payee, amount)` isn’t present in `payouts`
`MoveAbort code 3 in vault::settle`	sidecar log	`EInsufficientTreasury`: top up the vault

​Prerequisites

​Step 1 — get the live coordinator’s peer_id

​Step 2 — start the node

​Step 3 — client submits the request

​Step 4 — client posts the prompt to the node

​Step 5 — wait for settlement

​Step 6 — verify on Sui

​Troubleshooting