# Harness Engineering: Execution Plan

Based on the [Harness Engineering Design Document](./harness_engineering_design.md), this execution plan translates the theoretical architecture into concrete, step-by-step implementation tasks for the Cortex codebase. 

We are adopting an evolutionary approach, starting with a Monolithic MVP (Phase 1) using FastAPI `BackgroundTasks` to validate the core loop before scaling to a Celery Worker Fleet.

---

## Area 0: Swarm Baseline Architecture Upgrades (Prerequisites)
*Before writing a single line of Harness Engineering code, the underlying generic Swarm Platform must be refactored to support Agent-level security sandboxing and dynamic personalities.*

### Task 0.1: Skill-to-Session Binding (Principle of Least Privilege)
Currently, all available skills are likely loaded into a global pool. Giving an autonomous Agent access to a "Global Pool" is a catastrophic security risk if it suffers prompt injection.
- **Action:** Refactor the Database Models to support a Many-to-Many mapping: `Session <-> Skill`.
- **Action:** Modify the LLM prompt constructor in `profiles.py`. Before requesting the LLM to execute, it must query the DB for the specific `session_id`, retrieve the bounded active skills (e.g., *only* `pytest` and `cat`), and pass those restricted Tool definitions to the OpenAI API.

### Task 0.2: Dynamic System Prompts per Session
Currently, the system likely defaults to `DEFAULT_PROMPT_TEMPLATE` for all chat sessions. Autonomous Agents cannot run without highly specialized System Prompts (e.g., `github_reviewer.md` vs `deploy_bot.md`).
- **Action:** Add a `system_prompt_override` text column or file-reference to the underlying `Session` model.
- **Action:** Update the Chat Engine to natively conditionally load this localized prompt instead of the global default.

### Task 0.3: Session Locking (Purge Protection)
In the existing Swarm Control UI, users have a "Clean up all sessions" global purge button. If an `AgentInstance` natively maps to a `Session` ID to store its memory loop, a human pressing that purge button would instantly lobotomize all active background Agents by wiping their chat histories.
- **Action (Backend):** Expand the `Session` DB model with an `is_locked` (Boolean) property. Update the `DELETE /api/v1/sessions/purge` endpoint to strictly run a condition (`WHERE is_locked = False`), making active Agents immune to global deletion sweeps.
- **Action (Frontend):** In the normal Swarm Control sidebar, visually render a literal 🔒 Lock Icon next to any session where `is_locked` is true. Disable the manual delete button for that specific row, enforcing that an Agent's memory can only be purged by destroying the Agent itself from the Orchestrator Dashboard.

### Task 0.4: Context Truncation (Head & Tail Preservation)
If `history=session.messages` is passed natively to the AI orchestrator, an autonomous loop will rapidly exceed the 128k token API limit and crash with an `HTTP 400 ContextWindowExceededError`. However, blindly slicing the last 20 messages destroys the Agent's foundational mission prompt at the start of the chat.
- **Action:** Refactor `chat_with_rag` to aggressively chunk the message array. Provide the central AI with the **Head** (The initial 3 messages containing its core directive) and the **Tail** (The most recent 10-15 messages containing its immediate working memory/errors).
- **Action (The Summarizer):** Instead of blindly deleting the middle section and causing amnesia, dispatch the "middle" messages to a fast, cheap sub-agent (e.g., `gpt-4o-mini` or `claude-haiku`) with a strict prompt to compress it into a dense *"Timeline of Past Actions"* paragraph. Replace the middle array with this single string. This guarantees the API token limit is never breached, while providing seamless chronological tracking to the main Agent.

---

## Area 1: Core Database & Context Scaffolding
*The foundational building blocks required to store and track Agents in the `ai-hub` system.*

### Task 1.1: Define SQLAlchemy Models
We need to extend the backend relational database to track defining constraints, runtime loops, and external hooks.
- **Action:** Create models inside `/app/ai-hub/app/models/agent.py`.
- **`AgentTemplate`:** The blueprint.
  - `id` (UUID)
  - `name` (String)
  - `description` (String)
  - `system_prompt_path` (String) - Path to the `.md` persona file.
  - `max_loop_iterations` (Integer) - Circuit breaker cap (default: 20).
- **`AgentInstance`:** The living, breathing run-state.
  - `id` (UUID)
  - `template_id` (ForeignKey)
  - `session_id` (ForeignKey) - Links to the existing Swarm Control `Session` table (where chat history is stored).
  - `mesh_node_id` (ForeignKey) - The physical server this agent is locked to.
  - `status` (Enum: `active`, `idle`, `listening`, `error_suspended`)
  - `current_workspace_jail` (String) - E.g., `/tmp/cortex/agent_abc/`.
  - `last_heartbeat` (Timestamp) - Used by the Zombie Sweeper algorithm.
- **`AgentTrigger`:** The Wakeup hooks.
  - `id` (UUID)
  - `instance_id` (ForeignKey)
  - `trigger_type` (Enum: `webhook`, `cron`, `manual`)
  - `cron_expression` (String, nullable)
  - `webhook_secret` (String, nullable)
  - `webhook_mapping_schema` (JSON, nullable) - How to map the incoming JSON to a string prompt.

### Task 1.2: Define Pydantic Schemas & CRUD Endpoints
- **Action:** Create `/app/ai-hub/app/schemas/agent.py` mirroring the SQLAlchemy models.
- **Action:** Create routers in `/app/ai-hub/app/api/v1/endpoints/agents.py` to allow the Frontend dashboard to Query, Create, and Manage Agents.
  - `GET /api/v1/agents`
  - `POST /api/v1/agents`
  - `PATCH /api/v1/agents/{id}/status`

### Task 1.3: The Acknowledge-First API Controller
- **Action:** Build the Webhook receiver endpoint.
  - `POST /api/v1/agents/{id}/webhook`
  - Validate the `?token=` parameter.
  - Parse the JSON payload according to `webhook_mapping_schema`.
  - Push the mapped string as a `User` message into the associated `Session` database table.
  - Dispatch a `FastAPI.BackgroundTask` to wake up the Async Agent Loop.
  - Immediately return `HTTP 202 Accepted`.

---

## Area 2: The Agent Execution Loop & Circuit Breakers
*The core autonomous engine that actually "thinks" and "does" inside the backend, heavily fortified with error-handling.*

### Task 2.1: The Lifecycle Manager & Leases
When the Hub throws an agent into the `BackgroundTasks` queue, the `AgentExecutor` python class takes over.
- **Action:** Build `AgentExecutor.run(agent_id)` in `/app/ai-hub/app/core/orchestration/agent_loop.py`.
- **Acquire Lease:** Immediately update the DB `AgentInstance.last_heartbeat = NOW()` and `status = active`. Spawn an `asyncio` background thread to update this `last_heartbeat` every 60 seconds while the loop runs.
- **Node Lock:** Attempt to claim the `Global Node Lock` (if a deployment agent) or verify access to the `/tmp/cortex/agent_x/` Workspace Jail (if a concurrent worker).
- **Idempotency Check:** Instruct the initial LLM prompt to explicitly read its `.txt` local Scratchpad to verify if it is recovering from a previous crash state.

### Task 2.2: The `while True` Orchestration Loop
- **Action:** Implement the core AI loop integrating with the existing `profile.py` logic.
- **Circuit Breaker:** Initialize `iteration_count = 0`. At the top of the loop, check `if iteration_count >= template.max_loop_iterations`. If true, break the loop, update DB status to `error_suspended` and exit.
- **Exponential Backoff:** Wrap the OpenAI/LLM network call with the `@retry(wait=wait_exponential(multiplier=1, min=2, max=10))` decorator from the `tenacity` library to gracefully absorb `HTTP 502` or `HTTP 429` errors without crashing.

### Task 2.3: Context Saturation (Rolling memory)
- **Action:** Inside the loop, measure `len(chat_history)`.
- If the token payload exceeds 80k tokens, call a secondary, fast LLM model (`gpt-4o-mini` or `haiku`) with a specific prompt: *"Summarize these past 50 interactions into a single condensed scratchpad paragraph."*
- Replace the raw 50 messages in the DB with this single Summary message.

### Task 2.4: The JSON Handoff Protocol
- **Action:** Define a native `Tool/Skill` called `handoff_to_agent()`.
- Force the LLM to output parameter arguments matching the strict JSON Manifest Schema:
  - `target_agent_id`
  - `working_dir`
  - `files_changed`
  - `summary_for_target`
- **Execution:** When this tool is triggered, the `AgentExecutor` explicitly terminates the current Agent's loop, spins up the Target Agent's DB instance, injects the JSON Manifest as its *only* initial message, and pushes the new Agent into the `BackgroundTasks` queue.
---

## Area 3: Dashboard React UI (Dual-Track Views)
*The frontend evolution transforming Cortex from an interactive chatbot UI into a high-level "System Orchestrator" console.*

### Task 3.1: The Agent Harness Dashboard (Grid & Cards)
- **Action:** Create `AgentHarnessPage.jsx` at the router level. Map over `AgentInstance` records to render an `AgentCard`.
- **Telemetry Sparklines:** Integrate `recharts` to render a mini-graph on the card. Query `GET /api/v1/agents/{id}/telemetry` (pulling CPU/Memory metrics isolated by the Agent's specific Linux Sandbox/cgroup).
- **The Interceptor Mode:** Add "Global Kill-Switch" and "Pause on Tool-Call" buttons directly to the card. Clicking Pause triggers a `PATCH` request setting `status = paused_mid_loop`, telling the background `AgentExecutor` to halt execution *before* the next LLM tool execution fires, giving the human time to inspect the Jail state.

### Task 3.2: The Dual-Track Session Drill-Down
- **Action:** Build the `AgentDrillDown.jsx` view utilizing CSS Grid to split the viewport 50/50.
- **Left Pane (Chat Tracker):** Mount the existing `ChatWindow` component. This streams the live AI "thought process" and allows the human to inject custom prompt overrides to steer the loop.
- **Right Pane (Live Jail Filesystem):** Mount the existing `FileSystemNavigator.jsx` component.
  - *Crucial UI Filter:* Do not build a new generic filesystem tracker. Pass `rootPath={AgentInstance.current_workspace_jail}` (e.g., `/tmp/cortex/agent_abc/`) as a prop directly to `FileSystemNavigator`.
  - The Mirror sync system continues to operate natively, but the UI component mathematically locks the human's visual file-tree into the Agent's sandbox, providing immediate "proof of work" verification.

### Task 3.3: Context-Aware Terminal & Time-Travel Logs
- **Action:** Update the docked `MultiNodeConsole.jsx`.
- **UI Prompt Regex:** Based on the `AgentInstance.Global_Node_Lock` boolean, dynamically rewrite the PS1 prompt text natively in xterm.js. (e.g., green `[root@app-server] $` vs purple `[Agent_X@Jail] $`).
- **Scrubbing Slider:** Implement a `range` slider input. The terminal reads from `GET /nodes/{id}/persistent_agent_log` rather than ephemeral WebSockets, allowing React to "seek" back through thousands of output lines to pinpoint exactly where an obscure error happened 4 hours prior.

### Task 3.4: The Dependency Graph (Link View)
- **Action:** Integrate `react-flow-renderer` (or similar node-graph library).
- Query the database to find links where `target_agent_id` maps inside the JSON Manifest of a completed Agent. Visually draw edges connecting "Planner Agent" > "Coder Agent" > "QA Agent" with live token flow rates.
---

## Area 4: The Zombie Sweeper & Edge Artifacts
*The asynchronous safety nets that guarantee 100% uptime and completely prevent Agent Dementia during long loops.*

### Task 4.1: The Zombie Sweeper Service
- **Action:** Create `zombie_sweeper.py` in the Hub workers directory.
- **Scheduler:** Use `apscheduler` (or Celery Beat in Phase 3) to execute this job strictly every 5 minutes.
- **Logic:** Execute `UPDATE agent_instances SET status='idle' WHERE status='active' AND last_heartbeat < (NOW() - INTERVAL '3 minutes')`.
- **Requeue:** For every row updated, automatically dispatch a new `BackgroundTasks` execute call, forcing the Hub to instantly pick back up the crashed Agents exactly where they left off.

### Task 4.2: The "Hippocampus" Persistent Scratchpad
- **Action:** Add a native, mandatory `manage_scratchpad(text)` tool/skill to the Agent's baseline Sandbox.
- **The File Constraint:** Force this tool to strictly parse and append text directly to `{AgentInstance.workspace_jail}/.cortex_memory_scratchpad.txt`.
- **The System Prompt Override:** Inject a hardcoded string at the bottom of the user's `personas.md` file: *"CRITICAL: As your history is summarized, you will lose exact variables. You must continuously write critical facts to your `.cortex_memory_scratchpad.txt` file. Every time you wake up from a sleep or webhook, you MUST `cat` this file first."*

### Task 4.3: Stateful Headless Logging
- **Action:** Modify the gRPC `agent-node` shell execution handler.
- **File Sink:** Instead of merely streaming the `stdout/stderr` buffer into the live WebSocket, bind a `FileOutputStream` to dynamically write every line of PTY output to `~/.cortex/agent_logs/{session_id}.log` on the physical machine.
- **API Endpoint:** Create the HTTP endpoint `GET /api/v1/nodes/{id}/persistent_agent_log` so the Frontend's Time-Travel UI slider can request specific byte-ranges of this log file indefinitely, guaranteeing zero dropped terminal lines even if the frontend disconnects for hours.
