## Why Customer `k.sand@ksand.dk` reported never receiving wash certificates for completed bookings. Two methods contained silent early-return guards so the actual reason was unobservable from container logs: - `order_bookings_o::sendWashCertificateToCustomer()` — 5 silent returns - `email::sendWashCertificateEmailToCustomer()` — 1 silent return The most likely root cause: `email_notifications_enabled` defaults to `0` in the schema and `users.add()` does not set it on insert, so newly imported customers have notifications off until toggled. `wantsEmailNotifications()` then returns false and the email silently skips. ## What changed ### Original commit (`0ead5de5`) - `objects/order_bookings_o.php` — all 5 silent early-returns now log via new `logWashCertificateSkip()` helper (Redis stream `module=email / action=WASH_CERT_SKIP` + `error_log('[wash-cert-skip] …')`). - `classes/email.php` — silent `hasTransaction()` return in `sendWashCertificateEmailToCustomer()` now logs too. - `objects/bookings_o.php` — emits `WASH_CERT_SKIP` (legacy_no_wash_certificate_email) when `washCertificateEmail` is empty; no behavioural change. - **New** `routes/washCertificateDebugRoute.php` — `GET /debug/wash-certificates/diagnose?customer_number=&from=&to=` (404 in prod via $DEBUG; superuser-auth otherwise) replays the decision tree and reports `blocking_reason` per booking. ### Follow-up commit (`46a59e4e`) — silent-failure sweep **PART A — silent returns / silent errors (10 fixes):** - `email::sendEmailMailerSend()` — blacklisted-recipient skip now logs with context. - `email::sendNewCustomerRegistrationNotifications()` — empty-email skip + per-recipient try/catch with error_log (was unprotected; a single MailerSend error broke the loop). - `bookings_new_o::generateWashCertificate()` — wrapped `sendWashCertificateEmail()` in try/catch with error_log and re-throw (same pattern as the k.sand fix). - `users_o::getCustomerName()` — replaced catch-and-swallow with structured error_log. - `users_o::getCustomerEcocomicData()` — same. - `bookingsRoute.php` — added booking-id context to 4 × `$response->error('Booking not found', 404)` calls. **PART B — cron paths (10 files):** Added error_log breadcrumb + try/catch to `CheckUnfulfilledBookings`, `ClearAllUsersEconomicCustomerDetails`, `ClearAllUsersEconomicCustomerDiscounts`, `RunXLVaskModuleCron`, `SyncBookings`, `SyncEconomicInvoiceStatus`, `SyncLogs`, `BackfillEconomicV2History`, `EnsureXLVaskAutomationSchema`, and 3 functions in `Cron.php`. Each uses a distinct `[cron-…]` prefix for grep-ability. **PART C — real bugs (2 fixed):** 1. `email::sendEmailMailerSend()` attachment `array_map` — the previous exception message emitted a binary blob because `$attachment[0]` was already overwritten by `file_get_contents()`. Now captures $path first. 2. `bookings_new_o::generateWashCertificate()` — booking persisted as `completed` before email was sent, with no try/catch. Fixed (see PART A). ## How to verify 1. Deploy to staging. 2. Hit `/debug/wash-certificates/diagnose?customer_number=<k.sand's customer_number>` as a superuser — the response lists every booking's `blocking_reason`. 3. Tail container logs for `[wash-cert-skip]`, `[email-skip]`, `[cron-…]`, and Redis stream `module=email` action `WASH_CERT_SKIP` to see real-world skips going forward. ## Follow-ups (out of scope) - Schema migration to default `email_notifications_enabled` to `1` and backfill non-empty-email customers. - Move `error_log` to a proper PSR-3 logger. ## Risk - Logging only + new debug endpoint (404-gated in prod). No behavioural change for any path that previously sent mail successfully. `php -l` could not be run in the original sandbox; please verify on your CI box before deploying. 🤖 Generated with [OpenClaw](https://openclaw.ai)
Copenhagen Truck Wash API
Backend API for Copenhagen Truck Wash services.
Changes are published from a scoped feature branch through a pull request to
master; direct default-branch pushes are not part of the release workflow.
See default branch protection for the CI gate
and emergency procedure.
Architecture & Stack
- Edge Proxy: Traefik 2.11 (Handles TLS termination, routing, and rate limiting).
- Web Server: Caddy 2.7 (Serves the PHP application via FastCGI).
- PHP Runtime: PHP 8.2-FPM (Scale out with workers
php1tophp5). - Cache/Queue: Redis 7.
- Observability: Jaeger (Tracing), Prometheus (Metrics).
Getting Started
Prerequisites
- Docker Desktop 4.x+
- PHP 8.2 CLI (optional, for host-side testing)
Local Development
To bring up the minimal development stack (Traefik, Redis, MySQL debug DB, Caddy, and one PHP worker):
docker compose up -d traefik redis mysql-debug php1 caddy
The API is accessible at:
http://localhosthttps://localhost(using Traefik default cert)http(s)://localhost/api/(proxied with/apiprefix stripped)
Test Gateway Container
To run a real PHP edge agent as a disposable Dockerized test gateway against the local stack, first create an install token from the edge gateway admin UI, then start the helper:
.\scripts\test-gateway.ps1 start --install-token <token>
./scripts/test-gateway.sh start --install-token <token>
The helper will:
- start the local compose dependencies if needed
- claim a gateway through
http://localhost/api/edge-agent/claim - write the generated config to
.tmp/test-gateway/test-gateway.json - build
services/edge-agent/Dockerfile.test-gateway - run the PHP agent container on the local compose network
Useful follow-up commands:
.\scripts\test-gateway.ps1 logs
.\scripts\test-gateway.ps1 status
.\scripts\test-gateway.ps1 stop
To start all services including multiple PHP workers and development tools (Jaeger, Portainer):
docker compose --profile dev up -d
Configuration
Configuration is primarily managed via environment variables.
- The application root is located at
services/nginx/app. services/nginx/app/config.phploads configuration from the environment (requiresUSE_ENV=true).php1performs an automaticcomposer installon startup ifAUTO_COMPOSER_INSTALL=true.
Environment Change Runbook
When updating .env values used by PHP containers (for example e-conomic tokens), recreate affected services so Docker applies the new env:
docker compose up -d --force-recreate php1 php2 php3 php4 php5 php-cron
Edge Broker Public URL
Set EDGE_PUBLIC_BROKER_URL to the public route that serves the edge broker, including the path prefix handled by the proxy. Local Traefik uses http://localhost/api/edge-broker; production routes use the public broker prefix, for example https://api.truckwash.dk/edge-broker.
The browser terminal connects to the exact advertised EDGE_PUBLIC_BROKER_URL plus /ws/browser-shell. That URL must be routable through the proxy to the edge-broker service. Do not rely on derived /api/edge-broker fallback paths outside the local Traefik setup.
Testing
The project now uses Pest as the primary test runner in services/nginx/app.
Running Tests
All commands are run from services/nginx/app:
composer test
composer test:unit
composer test:integration
composer test:api
composer test:coverage
For local Docker development, run the PHP suites inside php1:
docker exec php1 sh -lc "cd /var/www/html && composer test:unit"
docker exec php1 sh -lc "cd /var/www/html && composer test:integration"
docker exec php1 sh -lc "cd /var/www/html && composer test:api"
docker exec php1 sh -lc "cd /var/www/html && composer run-script test:api:edge"
docker exec php1 sh -lc "cd /var/www/html && composer run-script test:integration:edge"
Integration tests are opt-in and should be run with required services available:
$env:RUN_INTEGRATION_TESTS='1'
composer test:integration
Edge Gateway Regression Coverage
The dedicated backend regression lane for the PHP edge gateway stack is split into:
- API contract tests for operator, agent, and broker-facing routes
- DB-backed integration tests for install sessions, heartbeats, tasks, logs, statistics, and shell persistence
- a local dockerized smoke that runs the real PHP edge agent against the local backend and broker
Run the targeted PHP suites inside php1:
docker exec php1 sh -lc "cd /var/www/html && composer run-script test:api:edge"
docker exec php1 sh -lc "cd /var/www/html && composer run-script test:integration:edge"
Run the full local smoke from backend-php on the host:
node .\scripts\edge-gateway-e2e.mjs
The E2E smoke expects the local compose stack and Docker daemon to be available. It boots a disposable gateway container, waits for a real heartbeat, validates live operations and telemetry, and verifies browser shell transcript persistence.
Public Staging Edge-Gateway Smoke
api.truckwash.io:4433 is the public staging ingress. For Edge Gateways v2, the router must serve the canonical artifacts from services/nginx/app/resources/edge-gateway-agent, not from the legacy dist/agent.mjs output or a separate runtime mount.
To verify the public staging stack after a deploy, use a real installer token and run:
node .\scripts\staging-edge-gateway-smoke.mjs --install-token <token>
node ./scripts/staging-edge-gateway-smoke.mjs --install-token <token>
The smoke check fails unless all of these return 200 from the public domain:
/ping/edge-agent/artifacts/agent.php/edge-agent/artifacts/truckwash-edge-agent.service/edge-agent/install.sh?token=<real token>
API Test Suite
The Api suite exercises real HTTP endpoints instead of calling route handlers in-process.
composer test:apienablesRUN_API_TESTS=1automatically.- By default the suite starts a temporary PHP server with
php -S 127.0.0.1:18080 index.phpand hits the app over HTTP. - The suite covers the real router, request parsing, auth headers, status codes, and JSON response envelopes.
- Tests run serially and use explicit database/Redis fixtures plus reverse-order cleanup instead of transaction rollbacks.
- The default phase-1 coverage includes
/ping, auth session routes, departments, department categories, and orders CRUD including the legacyPUT /orderalias. - API server logs are written to
services/nginx/app/build/logs/api-server.out.logandservices/nginx/app/build/logs/api-server.err.log.
To point the suite at an already-running base URL instead of the self-started PHP server:
$env:API_TEST_BASE_URL='http://127.0.0.1:18080'
composer test:api
The suite requires an initialized application schema. Redis-backed flows are used when Redis is configured, but the suite can still boot without caddy or the full reverse-proxy stack.
Run Tests Against A Cloned Live DB (Docker-Isolated)
Use the helper scripts in scripts/ to:
- Clone the configured live DB into a local MySQL Docker container.
- Start an isolated Redis container.
- Run tests in a separate temporary PHP Docker container pointed at that clone.
PowerShell:
powershell -ExecutionPolicy Bypass -File .\scripts\clone-live-db-and-test.ps1 -Force
Bash:
FORCE=1 ./scripts/clone-live-db-and-test.sh
Optional overrides:
- Test command:
-TestCommand "composer test:integration"orTEST_COMMAND="composer test:integration" - Keep containers after run:
-KeepContainersorKEEP_CONTAINERS=1 - Skip image build:
-SkipBuildorSKIP_BUILD=1 - Force image rebuild:
-ForceBuildorFORCE_BUILD=1
Clone Live DB Into mysql-debug Service
To refresh the docker-compose debug DB from live credentials in .env:
PowerShell:
powershell -ExecutionPolicy Bypass -File .\scripts\clone-live-to-debug-db.ps1 -Force
Bash:
FORCE=1 ./scripts/clone-live-to-debug-db.sh
Test Layout
tests/Unit/*: isolated unit and route-level behavior tests.tests/Integration/*: Redis/DB-backed tests intended for Docker/CI environments.tests/Api/*: real HTTP endpoint tests that boot a temporary PHP server and assert full request/response behavior.tests/Api/api_coverage_manifest.php: selected phase-1 endpoint manifest used by the API meta-test to enforce happy-path and failure coverage.tests/<legacy-domain>/*: legacy procedural scripts retained during migration; keep them runnable until matching Pest coverage exists.
Logs & Monitoring
- Logs:
docker compose logs -f [service_name](e.g.,php1,caddy,traefik). - Traefik Dashboard:
http://traefik.localhost(available in dev). - Jaeger (Tracing):
http://localhost:16686(when running with--profile dev). - Portainer:
http://localhost:9000(when running with--profile dev).
API Documentation
- OpenAPI: The authoritative OpenAPI 3.0 contract is at
openapi.yaml. - Self-Serve Module Guide: Implementation and API guide at
services/nginx/app/modules/selfserve/selfserve.md. - Writerside: Documentation projects are located in
/Writersideand/Writerside2. - Generated Writerside API Reference: The active Writerside project lives in
/documentationand is generated fromopenapi.yaml.
Writerside OpenAPI Generation Workflow
Prerequisite:
- Python 3 with PyYAML (
pip install pyyaml)
Run from repository root:
python scripts/generate_writerside_openapi_docs.py generate
python scripts/generate_writerside_openapi_docs.py check
Contribution rule:
- Update
openapi.yaml. - Regenerate docs (
python scripts/generate_writerside_openapi_docs.py generate). - Verify (
python scripts/generate_writerside_openapi_docs.py check).