QA / Regression Automation — device farm + AI test generation

Combines a physical device farm (10–20 devices for hardware products like POS terminals) with AI-generated test cases and overnight automated regression. AI writes new tests alongside features; AI modifies tests when features change; AI generates test data.

For a hardware/POS product, regression eats 2–3 weeks of a 3-week release cycle. Real reported wins: release regression cut from 5 days to 2; one product release pipeline 2× faster end-to-end. At the unit level: 15–25 of 25 QA roles freed (low-millions/year). Production bug rate down 20–30%.

1. Stand up a physical device farm racking the target hardware (terminal types, phone OSes) with USB/network control and a remote driver (e.g. STF, custom Appium grid).
2. Define the regression suite as code with each test linked to a user story and a screenshot baseline.
3. Skill gen-tests reads a new feature's PR + spec and proposes test cases (positive, negative, edge). Engineer reviews; merged tests join the suite.
4. Skill update-tests runs on every change to specs/UI and proposes test-case updates.
5. Nightly: full suite runs across the farm; AI categorises failures (real regression vs. flake vs. environment) and pings the right owner.
6. Track coverage: % of new tests AI-authored, % AI-assisted, false-flag rate.

• Don't let the AI mark flakes as "fixed"; require a human to close the loop. A team measuring "AI test pass rate" without humans soon has a suite that passes everything.
• Out-of-the-box AI works poorly for legacy stacks (e.g. 1C-based, ancient COBOL adapters). Scope to where the lift is real.