# Plan: Remove broken Coolify cron-worker deployment; add reliable 5-min cron

## Audit findings

The "Coolify cron worker flow" is a **dual-deployment mechanism** that:
- Tries to auto-deploy a **separate Coolify "cron" application** every time the API is deployed
- That separate app runs `php index.php run cron-worker` as a long-running process
- Tracks worker heartbeats in a `cron_worker_state` table

The "auto-deploy" part is implemented in `release_manager.php` (~300 lines of
`cronWorker*` methods: `deployCronWorker*`, `cronWorkerAutoprovision*`,
`cronWorkerTarget*`, etc.) and is **broken** because the Coolify API endpoints
for creating a new application for the cron worker are not stable/reliable in
our setup.

Meanwhile, the **actual cron mechanism** (`cron_worker.php`, `cron_scheduler.php`,
`cron_task_registry.php`, and the 20+ scheduled tasks in `modules/*/cron/tasks.php`)
is sound. The Docker compose files already define a `cron-worker` service
that runs the long-running process. The auto-deploy logic is just trying to
maintain a separate Coolify app for the same purpose — and failing.

## The plan

### 1. Remove the broken auto-deploy logic

Delete or no-op the following from `release_manager.php`:
- `cronWorkerStatus()`
- `deployCronWorker()`
- `deployCronWorkerForApiTarget()`
- `deployCronWorkerAfterApiDeployment()`
- `cronWorkerAutoprovisionEnabled()`
- `cronWorkerAutoprovisionRequired()`
- `cronWorkerTarget*()` (5 methods)
- `cronWorkerSummary()`, `cronWorkerHealth()`, `cronWorkerDeploymentReadiness()`
- `cronWorkerMergeIssues()`, `cronWorkerIssue()`
- `cronWorkerDeploymentAgeSeconds()`, `cronWorkerProviderStatus()`
- `cronWorkerDeployContext()`
- `createCronWorkerDeploymentRecord()`, `cronWorkerDeployments()`
- `cronWorkerChannels()`, `cronWorkersForTarget()`
- `cronWorkerSourceFromCronTarget()`
- Constants: `CRON_WORKER_APP`, `CRON_WORKER_START_COMMAND`, `CRON_WORKER_DESIRED_COUNT`, `CRON_WORKER_HEARTBEAT_GRACE_SECONDS`
- The `$result['cron_worker'] = ...` call after API deployment

Keep:
- `cron_worker.php` class (the actual worker)
- `cron_scheduler.php`, `cron_schedule.php`, `cron_task_registry.php`
- `cron_schema_bootstrap.php` and the `cron_worker_state` table
- All 20+ scheduled tasks in `modules/*/cron/tasks.php`
- The `cron-worker` service in `docker-compose*.yml`
- The `cron-worker` case in `cli.php`

### 2. Remove the corresponding tests

- `tests/Unit/Cron/CronWorkerWiringTest.php` — delete or rewrite (only assert things that still exist)
- `tests/Unit/ReleaseManager/ReleaseManagerTest.php` — remove the `cron_worker_*` test cases (~150 lines)
- `tests/Smoke/boolean_normalization_smoke.php` — remove cron_worker reference

### 3. Add a reliable 5-min cron mechanism

Two-layer approach:
1. **Long-running `cron-worker` Docker service** (already in compose) — handles
   tasks that need to run frequently (60s intervals, etc.). Started automatically
   with the rest of the stack.
2. **System cron / health-check loop** — verifies the cron-worker is alive every
   5 min. If no fresh heartbeat in 10 min, alert.

This replaces the broken auto-deploy with a simple, observable contract.

### 4. Add a verification harness

`/workspace/scripts/verify-api-cron.py`:
- Hits the API's `cronWorkerStatus` endpoint
- Reads `cron_worker_state` rows via the public route (or a new `/api/admin/cron-status` endpoint)
- If no fresh heartbeat in 10 min, post to #ai-daily
- Run every 5 min via a new cron job

### 5. Update documentation

- `inventory/self-serve-inventory.md` — remove coolify-cron-worker references
- `openapi.yaml` — remove `cron_worker_status` route documentation
- `routes/cronRoute.php` — remove the coolify-cron-worker endpoints

## Acceptance criteria

- [ ] `release_manager.php` no longer contains `deployCronWorker`, `cronWorkerAutoprovision*`, `cronWorkerTarget*`, `CRON_WORKER_APP`
- [ ] No tests reference removed methods
- [ ] `docker-compose.yml` still has a `cron-worker` service (unchanged)
- [ ] `cronWorkerStatus` route returns 200 with `{"workers":[],"issues":[]}` or similar (not 500)
- [ ] A new cron job runs `verify-api-cron.py` every 5 min
- [ ] Verify script posts to #ai-daily if no heartbeat in 10 min
- [ ] PR created, tests pass, merge

## Risk

- **Removing `deployCronWorker*` could break live deployments** if someone is
  actively using the API endpoint to deploy a cron worker. Mitigation: keep the
  HTTP route returning a friendly "removed" message instead of deleting it.
- **Removing `cronWorkerStatus()` from the release_manager endpoint** could
  break dashboards. Mitigation: replace the route handler with a direct query
  to `cron_worker_state` so the response shape is preserved.

## Steps

1. Create a feature branch `fix/remove-coolify-cron-worker`
2. Edit `release_manager.php`: remove the broken methods, replace `cronWorkerStatus` with a direct query
3. Edit `tests/Unit/ReleaseManager/ReleaseManagerTest.php`: remove cron_worker tests
4. Edit `tests/Unit/Cron/CronWorkerWiringTest.php`: drop assertions on removed wiring
5. Edit `routes/cronRoute.php`: keep the status endpoint but call the new direct query
6. Edit `cli.php`: no change needed (cron-worker case still works)
7. Edit `docker-compose*.yml`: no change needed (cron-worker service unchanged)
8. Create `/workspace/scripts/verify-api-cron.py` for the verification harness
9. Add a new cron job `5 * * * *` Europe/Copenhagen that runs `verify-api-cron.py`
10. Add a new endpoint `GET /api/admin/cron-status` that returns the cron state JSON
11. Run the test suite locally
12. Push branch, create PR, get user review