## Summary The Coolify-based auto-deployment of a separate `cron` worker app after every API deploy was never reliable. This PR removes the ~800 lines of dead auto-deploy logic from `release_manager.php` while keeping the underlying cron mechanism (`cron_worker.php`, `cron_scheduler.php`, the docker-compose `cron-worker` service) intact. ## Changes - **`release_manager.php`** (-818 lines) - Removed 19 private methods: `deployCronWorker*`, `cronWorker*`, `cronWorkerAutoprovision*`, etc. - Removed 3 constants: `CRON_WORKER_APP`, `CRON_WORKER_START_COMMAND`, `CRON_WORKER_DESIRED_COUNT` - Kept `cronWorkerStatus()` but rewrote as a direct DB query (no Coolify dependency) - **`tests/Unit/ReleaseManager/ReleaseManagerTest.php`** (-208 lines, removed 9 cron-worker tests) - **`tests/Unit/Cron/CronWorkerWiringTest.php`** (rewritten — now asserts removed wiring is GONE) - **`docs/CRON_PLAN.md`** (new — comprehensive plan) ## What replaced the broken auto-deploy - The cron worker runs as part of the main API docker-compose stack (the `cron-worker` service is unchanged) - New verification cron `1bb56ba8-2f3e-4bea-baa2-39801ea88ea8` runs `/workspace/scripts/verify-api-cron.py` every 5 min - Alerts to Slack #ai-daily (`C0AM3E43249`) if no fresh heartbeat in 10+ min ## Test results - 4/4 cron tests pass - 54/54 ReleaseManager tests pass - Full Unit suite: **1279 passed** (same 7 pre-existing failures on master, unchanged) - `php -l` passes on all modified files ## Plan See `docs/CRON_PLAN.md` for the full audit, plan, and acceptance criteria. 🤖 Generated with [OpenClaw](https://docs.openclaw.ai) --------- Co-authored-by: bugfix <bugfix@truckwash.local> Co-authored-by: openhands <openhands@all-hands.dev>
5.5 KiB
Plan: Remove broken Coolify cron-worker deployment; add reliable 5-min cron
Audit findings
The "Coolify cron worker flow" is a dual-deployment mechanism that:
- Tries to auto-deploy a separate Coolify "cron" application every time the API is deployed
- That separate app runs
php index.php run cron-workeras a long-running process - Tracks worker heartbeats in a
cron_worker_statetable
The "auto-deploy" part is implemented in release_manager.php (~300 lines of
cronWorker* methods: deployCronWorker*, cronWorkerAutoprovision*,
cronWorkerTarget*, etc.) and is broken because the Coolify API endpoints
for creating a new application for the cron worker are not stable/reliable in
our setup.
Meanwhile, the actual cron mechanism (cron_worker.php, cron_scheduler.php,
cron_task_registry.php, and the 20+ scheduled tasks in modules/*/cron/tasks.php)
is sound. The Docker compose files already define a cron-worker service
that runs the long-running process. The auto-deploy logic is just trying to
maintain a separate Coolify app for the same purpose — and failing.
The plan
1. Remove the broken auto-deploy logic
Delete or no-op the following from release_manager.php:
cronWorkerStatus()deployCronWorker()deployCronWorkerForApiTarget()deployCronWorkerAfterApiDeployment()cronWorkerAutoprovisionEnabled()cronWorkerAutoprovisionRequired()cronWorkerTarget*()(5 methods)cronWorkerSummary(),cronWorkerHealth(),cronWorkerDeploymentReadiness()cronWorkerMergeIssues(),cronWorkerIssue()cronWorkerDeploymentAgeSeconds(),cronWorkerProviderStatus()cronWorkerDeployContext()createCronWorkerDeploymentRecord(),cronWorkerDeployments()cronWorkerChannels(),cronWorkersForTarget()cronWorkerSourceFromCronTarget()- Constants:
CRON_WORKER_APP,CRON_WORKER_START_COMMAND,CRON_WORKER_DESIRED_COUNT,CRON_WORKER_HEARTBEAT_GRACE_SECONDS - The
$result['cron_worker'] = ...call after API deployment
Keep:
cron_worker.phpclass (the actual worker)cron_scheduler.php,cron_schedule.php,cron_task_registry.phpcron_schema_bootstrap.phpand thecron_worker_statetable- All 20+ scheduled tasks in
modules/*/cron/tasks.php - The
cron-workerservice indocker-compose*.yml - The
cron-workercase incli.php
2. Remove the corresponding tests
tests/Unit/Cron/CronWorkerWiringTest.php— delete or rewrite (only assert things that still exist)tests/Unit/ReleaseManager/ReleaseManagerTest.php— remove thecron_worker_*test cases (~150 lines)tests/Smoke/boolean_normalization_smoke.php— remove cron_worker reference
3. Add a reliable 5-min cron mechanism
Two-layer approach:
- Long-running
cron-workerDocker service (already in compose) — handles tasks that need to run frequently (60s intervals, etc.). Started automatically with the rest of the stack. - System cron / health-check loop — verifies the cron-worker is alive every 5 min. If no fresh heartbeat in 10 min, alert.
This replaces the broken auto-deploy with a simple, observable contract.
4. Add a verification harness
/workspace/scripts/verify-api-cron.py:
- Hits the API's
cronWorkerStatusendpoint - Reads
cron_worker_staterows via the public route (or a new/api/admin/cron-statusendpoint) - If no fresh heartbeat in 10 min, post to #ai-daily
- Run every 5 min via a new cron job
5. Update documentation
inventory/self-serve-inventory.md— remove coolify-cron-worker referencesopenapi.yaml— removecron_worker_statusroute documentationroutes/cronRoute.php— remove the coolify-cron-worker endpoints
Acceptance criteria
release_manager.phpno longer containsdeployCronWorker,cronWorkerAutoprovision*,cronWorkerTarget*,CRON_WORKER_APP- No tests reference removed methods
docker-compose.ymlstill has acron-workerservice (unchanged)cronWorkerStatusroute returns 200 with{"workers":[],"issues":[]}or similar (not 500)- A new cron job runs
verify-api-cron.pyevery 5 min - Verify script posts to #ai-daily if no heartbeat in 10 min
- PR created, tests pass, merge
Risk
- Removing
deployCronWorker*could break live deployments if someone is actively using the API endpoint to deploy a cron worker. Mitigation: keep the HTTP route returning a friendly "removed" message instead of deleting it. - Removing
cronWorkerStatus()from the release_manager endpoint could break dashboards. Mitigation: replace the route handler with a direct query tocron_worker_stateso the response shape is preserved.
Steps
- Create a feature branch
fix/remove-coolify-cron-worker - Edit
release_manager.php: remove the broken methods, replacecronWorkerStatuswith a direct query - Edit
tests/Unit/ReleaseManager/ReleaseManagerTest.php: remove cron_worker tests - Edit
tests/Unit/Cron/CronWorkerWiringTest.php: drop assertions on removed wiring - Edit
routes/cronRoute.php: keep the status endpoint but call the new direct query - Edit
cli.php: no change needed (cron-worker case still works) - Edit
docker-compose*.yml: no change needed (cron-worker service unchanged) - Create
/workspace/scripts/verify-api-cron.pyfor the verification harness - Add a new cron job
5 * * * *Europe/Copenhagen that runsverify-api-cron.py - Add a new endpoint
GET /api/admin/cron-statusthat returns the cron state JSON - Run the test suite locally
- Push branch, create PR, get user review