INFO:app.core.tools.registry:Registered dynamic tool plugin: 'browser_automation_agent'
INFO:app.core.tools.registry:Registered dynamic tool plugin: 'mesh_file_explorer'
INFO:app.core.tools.registry:Registered dynamic tool plugin: 'mesh_inspect_drift'
INFO:app.core.tools.registry:Registered dynamic tool plugin: 'mesh_sync_control'
INFO:app.core.tools.registry:Registered dynamic tool plugin: 'mesh_terminal_control'
INFO:app.core.tools.registry:Registered dynamic tool plugin: 'mesh_wait_tasks'
INFO:app.core.tools.registry:Registered dynamic tool plugin: 'read_skill_artifact'
INFO: Started server process [78674]
INFO: Waiting for application startup.
INFO:app.db.migrate:Starting database migrations...
INFO:app.db.migrate:Column 'audio_path' already exists in 'messages'.
INFO:app.db.migrate:Column 'model_response_time' already exists in 'messages'.
INFO:app.db.migrate:Column 'token_count' already exists in 'messages'.
INFO:app.db.migrate:Column 'reasoning_content' already exists in 'messages'.
INFO:app.db.migrate:Column 'stt_provider_name' already exists in 'sessions'.
INFO:app.db.migrate:Column 'tts_provider_name' already exists in 'sessions'.
INFO:app.db.migrate:Column 'sync_workspace_id' already exists in 'sessions'.
INFO:app.db.migrate:Column 'attached_node_ids' already exists in 'sessions'.
INFO:app.db.migrate:Column 'node_sync_status' already exists in 'sessions'.
INFO:app.db.migrate:Column 'sync_config' already exists in 'sessions'.
INFO:app.db.migrate:Column 'is_cancelled' already exists in 'sessions'.
INFO:app.db.migrate:Column 'restrict_skills' already exists in 'sessions'.
INFO:app.db.migrate:Column 'allowed_skill_names' already exists in 'sessions'.
INFO:app.db.migrate:Column 'system_prompt_override' already exists in 'sessions'.
INFO:app.db.migrate:Column 'is_locked' already exists in 'sessions'.
INFO:app.db.migrate:Database migrations complete.
INFO:app.core.services.node_registry:[NodeRegistry] Reset all DB node statuses to 'offline'.
INFO:app.core.grpc.services.grpc_server:🚀 CORTEX gRPC Orchestrator starting on [::]:50051
INFO:app.app:[M6] Agent Orchestrator gRPC server started on port 50051.
INFO:app.core.orchestration.scheduler:[Scheduler] Agent background services (Zombie Sweeper & CRON) started.
INFO:app.core.skills.bootstrap:Checking for system skills bootstrapping...
INFO:app.core.skills.bootstrap:System skills bootstrap completed.
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 04:57:40.128841')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
✅ Loading configuration from app/config.yaml
Application startup...
--- ⚙️ Application Configuration ---
- ACTIVE_LLM_PROVIDER: gemini
- ALLOW_OIDC_LOGIN: False
- ALLOW_PASSWORD_LOGIN: ***
- DATABASE_URL: sqlite:///./test.db
- DATA_DIR: ./data
- DB_MODE: sqlite
- DEEPSEEK_API_KEY: sk-a...6bf2
- DEEPSEEK_MODEL_NAME: deepseek-chat
- EMBEDDING_API_KEY: AIza...sKuI
- EMBEDDING_DIMENSION: 768
- EMBEDDING_MODEL_NAME: models/text-embedding-004
- EMBEDDING_PROVIDER: google_gemini
- FAISS_INDEX_PATH: data/faiss_index.bin
- GEMINI_API_KEY: AIza...sKuI
- GEMINI_MODEL_NAME: gemini/gemini-3-flash-preview
- GRPC_CERT_PATH: None
- GRPC_EXTERNAL_ENDPOINT: None
- GRPC_KEY_PATH: Not Set
- GRPC_TLS_ENABLED: False
- LLM_PROVIDERS: {'gemini': {'api_key': 'AIzaSyBn5HYiZ8yKmNL0ambyz4Aspr5lKw1sKuI', 'model': 'gemini/gemini-3-flash-preview'}, 'deepseek': {'api_key': 'sk-a1b3b85a32a942c3b80e06566ef46bf2'}, 'openai': {'api_key': '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'}}
- LOG_LEVEL: DEBUG
- OIDC_CLIENT_ID: cortex-server
- OIDC_CLIENT_SECRET: aYc2...leZI
- OIDC_ENABLED: False
- OIDC_REDIRECT_URI: http://localhost:8001/users/login/callback
- OIDC_SERVER_URL: https://auth.jerxie.com
- OPENAI_API_KEY: sk-p...h34A
- PROJECT_NAME: Cortex Hub
- SECRET_KEY: inte...-123
- SKILLS_DIR: ./data/skills
- STT_API_KEY: AIza...sKuI
- STT_MODEL_NAME: gemini-2.5-flash
- STT_PROVIDER: google_gemini
- STT_PROVIDERS: {}
- SUPER_ADMINS: ['axieyangb@gmail.com']
- TTS_API_KEY: AIza...sKuI
- TTS_MODEL_NAME: gemini-2.5-flash-preview-tts
- TTS_PROVIDER: google_gemini
- TTS_PROVIDERS: {}
- TTS_VOICE_NAME: Kore
- VERSION: 1.0.0
------------------------------------
Creating database tables...
INFO: 127.0.0.1:40396 - "HEAD /api/v1/users/login/local HTTP/1.1" 405 Method Not Allowed
INFO: 127.0.0.1:40412 - "POST /api/v1/users/login/local HTTP/1.1" 200 OK
INFO:app.core.services.preference:Saving updated global preferences via admin 915a44b3-7ab9-4670-bb86-cb5ae31304bc
🏠 Configuration synchronized to app/config.yaml
INFO: 127.0.0.1:40412 - "PUT /api/v1/users/me/config HTTP/1.1" 200 OK
INFO: 127.0.0.1:40412 - "POST /api/v1/users/admin/groups HTTP/1.1" 409 Conflict
INFO: 127.0.0.1:40412 - "GET /api/v1/users/admin/groups HTTP/1.1" 200 OK
INFO: 127.0.0.1:40412 - "PUT /api/v1/users/admin/groups/75fb001c-25a2-4f40-97e8-6d9ce38f1c2c HTTP/1.1" 200 OK
INFO: 127.0.0.1:40412 - "PUT /api/v1/users/admin/users/915a44b3-7ab9-4670-bb86-cb5ae31304bc/group HTTP/1.1" 200 OK
INFO: 127.0.0.1:40412 - "POST /api/v1/nodes/admin?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 409 Conflict
INFO:app.core.services.node_registry:[📋] NodeRegistry: Deregistered test-node-1
INFO: 127.0.0.1:40412 - "DELETE /api/v1/nodes/admin/test-node-1?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 200 OK
INFO:app.api.routes.nodes:[admin] Created node 'test-node-1' by admin 915a44b3-7ab9-4670-bb86-cb5ae31304bc
INFO: 127.0.0.1:40412 - "POST /api/v1/nodes/admin?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 200 OK
INFO: 127.0.0.1:40412 - "POST /api/v1/nodes/admin?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 409 Conflict
INFO:app.core.services.node_registry:[📋] NodeRegistry: Deregistered test-node-2
INFO: 127.0.0.1:40412 - "DELETE /api/v1/nodes/admin/test-node-2?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 200 OK
INFO:app.api.routes.nodes:[admin] Created node 'test-node-2' by admin 915a44b3-7ab9-4670-bb86-cb5ae31304bc
[NodeRegistry] DB mark-offline failed for test-node-2: UPDATE statement on table 'agent_nodes' expected to update 1 row(s); 0 were matched.
INFO: 127.0.0.1:40412 - "POST /api/v1/nodes/admin?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 200 OK
INFO: 127.0.0.1:40412 - "POST /api/v1/users/admin/groups HTTP/1.1" 409 Conflict
INFO:app.api.routes.agent_update:[AgentUpdate] Version check → 1.0.77
INFO:app.api.routes.agent_update:[AgentUpdate] Version check → 1.0.77
INFO: 127.0.0.1:40420 - "GET /api/v1/agent/version HTTP/1.1" 200 OK
INFO: 127.0.0.1:40416 - "GET /api/v1/agent/version HTTP/1.1" 200 OK
INFO:app.core.grpc.services.grpc_server:[gRPC] Incoming RPC Call: /agent.AgentOrchestrator/SyncConfiguration
INFO:app.core.grpc.services.grpc_server:[gRPC] Incoming RPC Call: /agent.AgentOrchestrator/SyncConfiguration
INFO:app.core.grpc.services.grpc_server:[🔑] SyncConfiguration REQUEST from test-node-1 (token prefix: nw-b...)
INFO:app.core.grpc.services.grpc_server:[🔑] SyncConfiguration REQUEST from test-node-2 (token prefix: 3GoS...)
INFO:app.core.grpc.services.grpc_server:[🔑] Token validated for test-node-2 (owner: 915a44b3-7ab9-4670-bb86-cb5ae31304bc)
INFO:app.core.grpc.services.grpc_server:[🔑] Handshake successful for test-node-2 (owner: 915a44b3-7ab9-4670-bb86-cb5ae31304bc)
INFO:app.core.grpc.services.grpc_server:[🔑] Token validated for test-node-1 (owner: 915a44b3-7ab9-4670-bb86-cb5ae31304bc)
INFO:app.core.services.node_registry:[📋] NodeRegistry: Registered test-node-2 (owner: 915a44b3-7ab9-4670-bb86-cb5ae31304bc) | Stats enabled
INFO:app.core.grpc.services.grpc_server:[🔑] Handshake successful for test-node-1 (owner: 915a44b3-7ab9-4670-bb86-cb5ae31304bc)
INFO:app.core.grpc.services.grpc_server:[gRPC] Incoming RPC Call: /agent.AgentOrchestrator/ReportHealth
INFO:app.core.services.node_registry:[📋] NodeRegistry: Registered test-node-1 (owner: 915a44b3-7ab9-4670-bb86-cb5ae31304bc) | Stats enabled
INFO:app.core.grpc.services.grpc_server:[gRPC] Incoming RPC Call: /agent.AgentOrchestrator/TaskStream
INFO:app.core.grpc.services.grpc_server:[gRPC] Incoming RPC Call: /agent.AgentOrchestrator/ReportHealth
INFO:app.core.grpc.services.grpc_server:[gRPC] Incoming RPC Call: /agent.AgentOrchestrator/TaskStream
INFO:app.core.grpc.services.grpc_server:[*] Node test-node-2 Attempting to establish TaskStream...
INFO:app.core.grpc.services.grpc_server:[*] Node test-node-2 Online (TaskStream established)
INFO:app.core.grpc.services.grpc_server:[*] Node test-node-1 Attempting to establish TaskStream...
INFO:app.core.grpc.services.grpc_server:[*] Node test-node-1 Online (TaskStream established)
[📁🔄] Triggering Resync Check for test-node-2...
[📁🔄] Triggering Resync Check for test-node-1...
[📁⚠️] Failed to fetch active sessions for node reconciliation: (sqlite3.OperationalError) no such column: sessions.auto_clear_history
[SQL: SELECT sessions.id AS sessions_id, sessions.user_id AS sessions_user_id, sessions.title AS sessions_title, sessions.provider_name AS sessions_provider_name, sessions.stt_provider_name AS sessions_stt_provider_name, sessions.tts_provider_name AS sessions_tts_provider_name, sessions.feature_name AS sessions_feature_name, sessions.created_at AS sessions_created_at, sessions.is_archived AS sessions_is_archived, sessions.is_cancelled AS sessions_is_cancelled, sessions.sync_workspace_id AS sessions_sync_workspace_id, sessions.attached_node_ids AS sessions_attached_node_ids, sessions.node_sync_status AS sessions_node_sync_status, sessions.sync_config AS sessions_sync_config, sessions.restrict_skills AS sessions_restrict_skills, sessions.allowed_skill_names AS sessions_allowed_skill_names, sessions.system_prompt_override AS sessions_system_prompt_override, sessions.is_locked AS sessions_is_locked, sessions.auto_clear_history AS sessions_auto_clear_history
FROM sessions
WHERE sessions.is_archived = 0 AND sessions.sync_workspace_id IS NOT NULL]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
[📁⚠️] Failed to fetch active sessions for node reconciliation: (sqlite3.OperationalError) no such column: sessions.auto_clear_history
[SQL: SELECT sessions.id AS sessions_id, sessions.user_id AS sessions_user_id, sessions.title AS sessions_title, sessions.provider_name AS sessions_provider_name, sessions.stt_provider_name AS sessions_stt_provider_name, sessions.tts_provider_name AS sessions_tts_provider_name, sessions.feature_name AS sessions_feature_name, sessions.created_at AS sessions_created_at, sessions.is_archived AS sessions_is_archived, sessions.is_cancelled AS sessions_is_cancelled, sessions.sync_workspace_id AS sessions_sync_workspace_id, sessions.attached_node_ids AS sessions_attached_node_ids, sessions.node_sync_status AS sessions_node_sync_status, sessions.sync_config AS sessions_sync_config, sessions.restrict_skills AS sessions_restrict_skills, sessions.allowed_skill_names AS sessions_allowed_skill_names, sessions.system_prompt_override AS sessions_system_prompt_override, sessions.is_locked AS sessions_is_locked, sessions.auto_clear_history AS sessions_auto_clear_history
FROM sessions
WHERE sessions.is_archived = 0 AND sessions.sync_workspace_id IS NOT NULL]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 2 Nodes Online
--------------------------------------------------
🟢 test-node-2 | Workers: 0 | Running: 0 tasks
Capabilities: {'has_sudo': 'true', 'has_display': 'false', 'os_release': '6.10.11-linuxkit', 'arch': 'aarch64', 'os': 'linux', 'gpu': 'none', 'local_ip': '172.27.0.2', 'is_root': 'false', 'shell': 'v1'}
🟢 test-node-1 | Workers: 0 | Running: 0 tasks
Capabilities: {'has_sudo': 'true', 'has_display': 'false', 'os_release': '6.10.11-linuxkit', 'arch': 'aarch64', 'os': 'linux', 'gpu': 'none', 'local_ip': '172.27.0.2', 'is_root': 'false', 'shell': 'v1'}
==================================================
ERROR:app.app:[📁🧹] Ghost Mirror periodic cleanup fail: (sqlite3.OperationalError) no such column: sessions.auto_clear_history
[SQL: SELECT sessions.id AS sessions_id, sessions.user_id AS sessions_user_id, sessions.title AS sessions_title, sessions.provider_name AS sessions_provider_name, sessions.stt_provider_name AS sessions_stt_provider_name, sessions.tts_provider_name AS sessions_tts_provider_name, sessions.feature_name AS sessions_feature_name, sessions.created_at AS sessions_created_at, sessions.is_archived AS sessions_is_archived, sessions.is_cancelled AS sessions_is_cancelled, sessions.sync_workspace_id AS sessions_sync_workspace_id, sessions.attached_node_ids AS sessions_attached_node_ids, sessions.node_sync_status AS sessions_node_sync_status, sessions.sync_config AS sessions_sync_config, sessions.restrict_skills AS sessions_restrict_skills, sessions.allowed_skill_names AS sessions_allowed_skill_names, sessions.system_prompt_override AS sessions_system_prompt_override, sessions.is_locked AS sessions_is_locked, sessions.auto_clear_history AS sessions_auto_clear_history
FROM sessions
WHERE sessions.sync_workspace_id IS NOT NULL]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
INFO:app.api.routes.nodes:[admin] Created node 'test-agent-node-3f60acad' by admin 915a44b3-7ab9-4670-bb86-cb5ae31304bc
[📁⚠️] Mirror Cleanup Thread Error: (sqlite3.OperationalError) no such column: sessions.auto_clear_history
[SQL: SELECT sessions.id AS sessions_id, sessions.user_id AS sessions_user_id, sessions.title AS sessions_title, sessions.provider_name AS sessions_provider_name, sessions.stt_provider_name AS sessions_stt_provider_name, sessions.tts_provider_name AS sessions_tts_provider_name, sessions.feature_name AS sessions_feature_name, sessions.created_at AS sessions_created_at, sessions.is_archived AS sessions_is_archived, sessions.is_cancelled AS sessions_is_cancelled, sessions.sync_workspace_id AS sessions_sync_workspace_id, sessions.attached_node_ids AS sessions_attached_node_ids, sessions.node_sync_status AS sessions_node_sync_status, sessions.sync_config AS sessions_sync_config, sessions.restrict_skills AS sessions_restrict_skills, sessions.allowed_skill_names AS sessions_allowed_skill_names, sessions.system_prompt_override AS sessions_system_prompt_override, sessions.is_locked AS sessions_is_locked, sessions.auto_clear_history AS sessions_auto_clear_history
FROM sessions
WHERE sessions.is_archived = 0 AND sessions.sync_workspace_id IS NOT NULL]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
INFO: 127.0.0.1:44190 - "POST /api/v1/nodes/admin?admin_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc HTTP/1.1" 200 OK
INFO: 127.0.0.1:44190 - "POST /api/v1/agents/deploy HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table sessions has no column named auto_clear_history
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/vscode/.local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 416, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1160, in __call__
await super().__call__(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/applications.py", line 107, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 87, in __call__
await self.app(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py", line 716, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py", line 736, in app
await route.handle(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/routing.py", line 290, in handle
await self.app(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py", line 119, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py", line 105, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py", line 431, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/fastapi/routing.py", line 315, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
return await anyio.to_thread.run_sync(func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/anyio/to_thread.py", line 63, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2502, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 986, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/ai-hub/app/api/routes/agents.py", line 298, in deploy_agent
db.flush()
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4331, in flush
self._flush(objects)
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4466, in _flush
with util.safe_reraise():
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 121, in __exit__
raise exc_value.with_traceback(exc_tb)
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4427, in _flush
flush_context.execute()
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
rec.execute(self)
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 642, in execute
util.preloaded.orm_persistence.save_obj(
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/persistence.py", line 93, in save_obj
_emit_insert_statements(
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/orm/persistence.py", line 1233, in _emit_insert_statements
result = connection.execute(
^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
self._handle_dbapi_exception(
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/vscode/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table sessions has no column named auto_clear_history
[SQL: INSERT INTO sessions (user_id, title, provider_name, stt_provider_name, tts_provider_name, feature_name, created_at, is_archived, is_cancelled, sync_workspace_id, attached_node_ids, node_sync_status, sync_config, restrict_skills, allowed_skill_names, system_prompt_override, is_locked, auto_clear_history) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: ('agent-system', None, 'gemini', None, None, 'agent_harness', '2026-03-24 05:00:51.756379', 0, 0, None, '["test-agent-node-3f60acad"]', '{}', '{}', 0, '[]', 'You are a cron agent. Run shell tasks periodically.', 1, 0)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
INFO: 127.0.0.1:44206 - "GET /api/v1/speech/voices HTTP/1.1" 200 OK
INFO:app.api.routes.tts:Using TTS provider: GeminiTTSProvider for user=915a44b3-7ab9-4670-bb86-cb5ae31304bc
INFO:app.core.providers.tts.gemini:TTS request [model=gemini-2.5-flash-preview-tts, vertex=False]: 'Hello from integration test audio pipeline.'
INFO:app.core.services.tts:Successfully gathered audio data for all 1 chunks.
INFO:app.core.services.tts:Concatenated 1 chunks into a single PCM stream.
INFO: 127.0.0.1:44218 - "POST /api/v1/speech?stream=false HTTP/1.1" 200 OK
INFO:app.api.routes.stt:Received transcription request for file: test_audio_pipeline.wav
INFO:app.api.routes.stt:Resolving STT. user_id=915a44b3-7ab9-4670-bb86-cb5ae31304bc, provider=google_gemini
INFO:app.api.routes.stt:Using STT provider: GoogleSTTProvider
INFO:app.core.services.stt:Starting transcription for audio data (165690 bytes).
INFO:app.core.services.stt:Transcribed audio. Length: 43 characters.
INFO: 127.0.0.1:44218 - "POST /api/v1/stt/transcribe HTTP/1.1" 200 OK
INFO: 127.0.0.1:44222 - "GET /api/v1/nodes/test-node-1/status HTTP/1.1" 200 OK
INFO: 127.0.0.1:44222 - "POST /api/v1/sessions/ HTTP/1.1" 500 Internal Server Error
INFO: 127.0.0.1:44222 - "POST /api/v1/sessions/ HTTP/1.1" 500 Internal Server Error
INFO: 127.0.0.1:38894 - "POST /api/v1/sessions/ HTTP/1.1" 500 Internal Server Error
INFO: 127.0.0.1:38898 - "POST /api/v1/users/login/local HTTP/1.1" 200 OK
INFO: 127.0.0.1:38900 - "POST /api/v1/users/login/local HTTP/1.1" 401 Unauthorized
INFO: 127.0.0.1:38910 - "POST /api/v1/users/login/local HTTP/1.1" 401 Unauthorized
[92m05:00:57 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO: 127.0.0.1:38920 - "POST /api/v1/users/me/config/verify_llm HTTP/1.1" 200 OK
[92m05:00:57 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
ERROR:app.api.routes.user:LLM Verification failed for gemini (None): Authentication failed for gemini/gemini-3-flash-preview. Check your API key.
[1;31mGive Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new[0m
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.
INFO: 127.0.0.1:38932 - "POST /api/v1/users/me/config/verify_llm HTTP/1.1" 200 OK
INFO:app.core.services.preference:Saving updated global preferences via admin 915a44b3-7ab9-4670-bb86-cb5ae31304bc
🏠 Configuration synchronized to app/config.yaml
INFO: 127.0.0.1:38940 - "PUT /api/v1/users/me/config HTTP/1.1" 200 OK
INFO: 127.0.0.1:38940 - "GET /api/v1/users/me/config HTTP/1.1" 200 OK
[92m05:00:58 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO: 127.0.0.1:38954 - "POST /api/v1/users/me/config/verify_llm HTTP/1.1" 200 OK
ERROR:app.api.routes.user:LLM Verification failed for non_existent_provider_xyz (None): LiteLLM Error (non_existent_provider_xyz/unknown_model): litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=non_existent_provider_xyz/unknown_model
Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
INFO: 127.0.0.1:38970 - "POST /api/v1/users/me/config/verify_llm HTTP/1.1" 200 OK
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
[1;31mProvider List: https://docs.litellm.ai/docs/providers[0m
INFO: 127.0.0.1:38978 - "GET /api/v1/users/me/config/models?provider_name=gemini HTTP/1.1" 200 OK
WARNING:app.core.grpc.services.grpc_server:Results listener closed for test-node-1:
WARNING:app.core.grpc.services.grpc_server:Results listener closed for test-node-2:
WARNING:app.core.grpc.services.grpc_server:[📶] gRPC Stream TERMINATED for test-node-1. Cleaning up.
WARNING:app.core.grpc.services.grpc_server:[📶] gRPC Stream TERMINATED for test-node-2. Cleaning up.
INFO:app.core.services.node_registry:[📋] NodeRegistry: Deregistered test-node-1
INFO:app.core.services.node_registry:[📋] NodeRegistry: Deregistered test-node-2
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 04:58:40.135427')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
[92m05:01:40 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
[92m05:01:40 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= deepseek-chat; provider = deepseek
INFO:LiteLLM:
LiteLLM completion() model= deepseek-chat; provider = deepseek
INFO:app.app:[Health Check] System LLM statuses updated.
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 04:59:40.149991')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:00:40.159972')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:01:40.168100')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:02:40.181130')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:03:40.211583')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
[92m05:06:43 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
[92m05:06:44 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= deepseek-chat; provider = deepseek
INFO:LiteLLM:
LiteLLM completion() model= deepseek-chat; provider = deepseek
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:04:40.229064')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:05:40.247844')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:06:40.264966')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:07:40.273954')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
[📁⚠️] Mirror Cleanup Thread Error: (sqlite3.OperationalError) no such column: sessions.auto_clear_history
[SQL: SELECT sessions.id AS sessions_id, sessions.user_id AS sessions_user_id, sessions.title AS sessions_title, sessions.provider_name AS sessions_provider_name, sessions.stt_provider_name AS sessions_stt_provider_name, sessions.tts_provider_name AS sessions_tts_provider_name, sessions.feature_name AS sessions_feature_name, sessions.created_at AS sessions_created_at, sessions.is_archived AS sessions_is_archived, sessions.is_cancelled AS sessions_is_cancelled, sessions.sync_workspace_id AS sessions_sync_workspace_id, sessions.attached_node_ids AS sessions_attached_node_ids, sessions.node_sync_status AS sessions_node_sync_status, sessions.sync_config AS sessions_sync_config, sessions.restrict_skills AS sessions_restrict_skills, sessions.allowed_skill_names AS sessions_allowed_skill_names, sessions.system_prompt_override AS sessions_system_prompt_override, sessions.is_locked AS sessions_is_locked, sessions.auto_clear_history AS sessions_auto_clear_history
FROM sessions
WHERE sessions.is_archived = 0 AND sessions.sync_workspace_id IS NOT NULL]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:08:40.283905')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
[92m05:11:46 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
[92m05:11:47 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= deepseek-chat; provider = deepseek
INFO:LiteLLM:
LiteLLM completion() model= deepseek-chat; provider = deepseek
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:09:40.295121')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:10:40.308718')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:11:40.320828')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:12:40.335084')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
ERROR:app.core.orchestration.scheduler:[Scheduler] Zombie Sweeper iteration failed: (sqlite3.OperationalError) no such column: agent_instances.total_runs
[SQL: SELECT agent_instances.id AS agent_instances_id, agent_instances.template_id AS agent_instances_template_id, agent_instances.session_id AS agent_instances_session_id, agent_instances.mesh_node_id AS agent_instances_mesh_node_id, agent_instances.status AS agent_instances_status, agent_instances.current_workspace_jail AS agent_instances_current_workspace_jail, agent_instances.last_heartbeat AS agent_instances_last_heartbeat, agent_instances.total_runs AS agent_instances_total_runs, agent_instances.successful_runs AS agent_instances_successful_runs, agent_instances.total_tokens_accumulated AS agent_instances_total_tokens_accumulated, agent_instances.total_input_tokens AS agent_instances_total_input_tokens, agent_instances.total_output_tokens AS agent_instances_total_output_tokens, agent_instances.total_running_time_seconds AS agent_instances_total_running_time_seconds, agent_instances.tool_call_counts AS agent_instances_tool_call_counts
FROM agent_instances
WHERE agent_instances.status = ? AND agent_instances.last_heartbeat < ?]
[parameters: ('active', '2026-03-24 05:13:40.347270')]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
[92m05:16:50 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
INFO:LiteLLM:
LiteLLM completion() model= gemini-3-flash-preview; provider = gemini
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
[92m05:16:50 - LiteLLM:INFO[0m: utils.py:3895 -
LiteLLM completion() model= deepseek-chat; provider = deepseek
INFO:LiteLLM:
LiteLLM completion() model= deepseek-chat; provider = deepseek
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
==================================================
📡 CORTEX MESH DASHBOARD | 0 Nodes Online
--------------------------------------------------
No nodes currently connected.
==================================================
