---
description: How to deploy the AI Hub application to the production server (192.168.68.113)
---

## Environment Context

| Feature | Local Development (Mac Mini) | Production Hub (Alpine Linux) | Windows Node (Swarm) |
| :--- | :--- | :--- | :--- |
| **Hostname** | `jerxie-macbookpro` | `ai.jerxie.com` (192.168.68.113) | `192.168.68.7` |
| **Project Path** | `/Users/axieyangb/Project/CortexAI` | `/home/coder/project/cortex-hub` | `C:\CortexAgent` |
| **Execution** | Native Shell (Zsh) | Docker Compose | Scheduled Task (`CortexAgent`) |
| **Python** | `source cortex-ai/bin/activate` | Internal Docker Venv | `C:\CortexAgent\venv\Scripts\python.exe` |

---

## 🚀 Hub Deployment (192.168.68.113)

The Hub deployment is automated via a wrapper script that handles GitBucket secrets and remote rebuilding.

### 1. Automated Command
```bash
# This script handles authentication, syncing, and remote rebuilding.
./scripts/deploy_to_prod.exp
```

---

## 🖥️ Swarm Node Deployment (Windows - 192.168.68.7)

Windows nodes are managed natively via SSH and Scheduled Tasks. 

### 1. Synchronization
Sync the latest `agent-node` code to the production folder. We use `scp` as `rsync` is typically not available on Windows.

```bash
# Sync agent-node source AND mesh-sdk (both must be synced together)
sshpass -p 'Y@ngy@ngX1e2019' scp -o StrictHostKeyChecking=no -r agent-node/src/ axiey@192.168.68.7:C:/CortexAgent/src/
sshpass -p 'Y@ngy@ngX1e2019' scp -o StrictHostKeyChecking=no -r mesh-sdk/mesh_core/ axiey@192.168.68.7:C:/CortexAgent/src/mesh_core/
```

> **Important**: `mesh_core` is a separate package from `agent_node`. Always sync both together — syncing only `agent-node/src/` leaves the old `mesh_core` on disk and will silently run stale SDK code.

### 2. Restarting the Agent
The agent runs as a Scheduled Task named `CortexAgent`. To activate changes, you MUST restart the task. **Use PowerShell** for reliable command chaining over SSH.

```bash
ssh axiey@192.168.68.7 "powershell -Command \"Stop-ScheduledTask -TaskName CortexAgent -ErrorAction SilentlyContinue; Start-ScheduledTask -TaskName CortexAgent\""
```

### 3. Verification
Verify the agent is running and connected:
1. **Process Check**: `ssh axiey@192.168.68.7 "tasklist | findstr python"`
2. **Log Check**: `ssh axiey@192.168.68.7 "powershell -Command \"Get-Content C:\CortexAgent\agent.log -Tail 20\""`
3. **UI Check**: Verify the node shows as green/online at `ai.jerxie.com/nodes`.

---

---

## ⚠️ MANDATORY: Run verification BEFORE committing / deploying

### 1. Sanity Check Backend Startup
Before committing or deploying, ensure the backend can start without immediate crashes.

**Local Development (Native):**
```bash
source cortex-ai/bin/activate
cd ai-hub && export DEVELOPER_MODE=true && export DATABASE_URL="sqlite:///./test_cortex.db"
uvicorn app.main:app --host 0.0.0.0 --port 8010 --log-level debug
```

**Remote Production Environment Check:**
```bash
# This command runs inside the container on the remote server
sshpass -p "$REMOTE_PASSWORD" ssh ${REMOTE_USER}@${REMOTE_HOST} \
  "docker exec ai_hub_service uvicorn app.main:app --host 0.0.0.0 --port 8000 --log-level debug"
```
Ensure the logs show `Started server process [...]` and `Waiting for application startup.` without errors.

### 2. Run file sync tests
Any change to the following files **requires running the file sync integration tests first** and all must pass before pushing:

- `agent-node/src/agent_node/core/watcher.py`
- `agent-node/src/agent_node/core/sync.py`
- `agent-node/src/agent_node/node.py`
- `ai-hub/app/core/grpc/services/assistant.py`
- `ai-hub/app/core/grpc/services/grpc_server.py`
- `ai-hub/app/core/grpc/core/mirror.py`
- `ai-hub/app/protos/agent.proto`

**Run small file tests (Cases 1–4 and 9, ~5s) — the minimum bar before any commit:**

// turbo
```bash
source /app/.agent/utils/env_loader.sh
sshpass -p "$REMOTE_PASSWORD" ssh -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} \
  "echo '$REMOTE_PASSWORD' | sudo -S docker exec \
    -e SYNC_TEST_BASE_URL=http://127.0.0.1:8000 \
    -e SYNC_TEST_USER_ID=9a333ccd-9c3f-432f-a030-7b1e1284a436 \
    ai_hub_service \
    python3 -m pytest /tmp/test_file_sync.py::TestSmallFileSync -v -s -p no:warnings 2>&1"
```

> The test file is deployed automatically by `remote_deploy.sh`. If you need to refresh it without a full deploy, see `/file_sync_tests` workflow.

All 5 small-file cases must show `PASSED`. If any fail, **do not deploy** — diagnose the regression first (see `/file_sync_tests` for troubleshooting).

**Run macOS File Persistence tests (~30s) — ensures Mac Mini symlink bugs didn't regress:**

// turbo
```bash
awk '/```bash/{flag=1; next} /```/{flag=0} flag' .agent/workflows/mac_mini_file_sync_persistence.md > /tmp/mac_test.sh && bash /tmp/mac_test.sh
```
Or view the `.agent/workflows/mac_mini_file_sync_persistence.md` workflow for the verbose script. If the file output is deleted or missing after 20 seconds, **do not deploy**.

---


---

## ⚠️ MANDATORY: Post-Deployment Verification

After deploy completes, run **both** checks:

### 1. Basic API health check
```bash
source /app/.agent/utils/env_loader.sh
sshpass -p "$REMOTE_PASSWORD" ssh -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} \
  "echo '$REMOTE_PASSWORD' | sudo -S docker exec ai_hub_service python3 -c \"
import requests
headers = {'X-User-ID': '9a333ccd-9c3f-432f-a030-7b1e1284a436'}
r = requests.get('http://localhost:8000/api/v1/nodes/test-node-1/status', headers=headers)
print(f'Status: {r.status_code}')
print(r.text[:200])
\""
```
**Expected Outcome**: `200 OK` and `{"status": "online", ...}`.

### 2. File sync regression test (re-run after deploy)

// turbo
```bash
source /app/.agent/utils/env_loader.sh
sshpass -p "$REMOTE_PASSWORD" ssh -o StrictHostKeyChecking=no ${REMOTE_USER}@${REMOTE_HOST} \
  "echo '$REMOTE_PASSWORD' | sudo -S docker exec \
    -e SYNC_TEST_BASE_URL=http://127.0.0.1:8000 \
    -e SYNC_TEST_USER_ID=9a333ccd-9c3f-432f-a030-7b1e1284a436 \
    ai_hub_service \
    python3 -m pytest /tmp/test_file_sync.py::TestSmallFileSync -v -p no:warnings 2>&1"
```
All 5 must pass. If any fail post-deploy, **rollback immediately** using `git revert` and re-deploy.

### Visual Verification (Worst Case)
If the backend check passes but UI behaviors are suspect or need deeper frontend validation, use the `/frontend_tester` workflow:
```bash
/frontend_tester
```
