UpdateProof
PHP · PLAYWRIGHT · SARIF
Plugin and theme maintainers need evidence that an update is compatible before it ships, not a support ticket afterwards. UpdateProof turns a declared test plan into a deterministic, immutable matrix and records what was tested, where, and what changed.
- Maintainer
- Pixy Puala
- Vertical
- Release assurance · QA tooling
- Year
- 2026
- Role
- Open source
- Stack
- PHP 8.1+
- Status
- ● Private repo
Problem
Compatibility testing is usually manual, undocumented and unrepeatable — someone clicks through one site, on one PHP version, once. There is no artefact afterwards, so the next release starts from zero and regressions are found by customers.
Constraints
Decisions
The plan is the contract
A YAML/JSON plan is parsed, schema-validated and resolved into an immutable matrix before anything runs. If the plan is wrong, it fails at parse time rather than halfway through a browser run.
Exit codes are a public enum
A deterministic 0–8 exit-code contract with defined retry semantics means CI can branch on the result instead of grepping log text.
SARIF, so results land where reviewers already look
Alongside JSON, JUnit and Markdown, SARIF 2.1.0 output puts findings into GitHub code scanning rather than a build log nobody opens.
Build
Stack
- ▸ PHP 8.1+ · Composer
- ▸ YAML/JSON plan parser · JSON Schema validation
- ▸ Deterministic immutable matrix resolver
- ▸ Reporters · JSON · JUnit · Markdown · SARIF 2.1.0
- ▸ Playwright (TypeScript) browser runner
- ▸ WP-CLI command · GitHub Actions
Tooling
- ✓ kept Composer · Playwright · GitHub Actions
Results · Measured
The Playwright runner is proven end-to-end in CI against a bundled fixture, and runner/tests/fixture-site.spec.ts drives a full journey — plugin activation, WooCommerce admin, add-to-cart, a real checkout order, orders and webhooks screens — against WordPress 7.0.2 with WooCommerce 10.9.4, recorded in docs/RUNTIME-VERIFICATION.md.
Tradeoffs
Single-cell browser journeys run for real; orchestrating many WordPress/WooCommerce/PHP cells simultaneously is the documented boundary and is not built. The repository is not yet public, so no source link is published here — the runtime evidence above is what the claim rests on.