Case file a1qa-1
eCommerce · Flaky tests cut from 18% to 4.6% (grade B, baseline yes)
grade B, partial Anonymous client with a metric that states its baseline.
Claim ledger
Each row quotes the vendor. A row is a claim, and it counts toward outcome verifiability only when the source states the baseline and the measurement method.
| Kind | Vendor claim (quoted) | Before | After | Baseline | Method |
|---|---|---|---|---|---|
| other | “Flaky tests cut from 18% to 4.6%” (grade B, baseline yes) | 18% flaky test rate | 4.6% | yes | yes |
| cost | “32% lower regression and triage cost after ongoing maintenance was included” (grade B, baseline no) | not stated | not stated | no | yes |
- baseline:
- The measured value before the work started, against which a later value is compared. source
Engagement facts
- Client
- not disclosed
- Industry
- eCommerce stated
- Client type
- not disclosed
- Frameworks
- Selenium WebDriver and REST Assured stated
- Languages
- Java stated
- CI tools
- Jenkins stated
- Device or browser clouds
- not disclosed
- Team size
- 5 stated
- Duration in months
- 3 stated
- Start year
- not disclosed
- Publication date
- not disclosed
What this case does not tell you
- Client
- Client type
- Device or browser clouds
- Start year
- Publication date
Source
eCommerce · Flaky tests cut from 18% to 4.6% (grade B, baseline yes) (a1qa.com)
accessed 2026-11-11, verified 2026-11-11
Related pages
- a1qa: vendor profile and score
- Selenium WebDriver: vendors with published cases
- REST Assured: vendors with published cases
- DeviQA: QA services for Shopify appointment scheduling solution
- DeviQA: Software testing services for video production software
- All case files
- Framework index
- Vendor ranking
- How cases are graded A, B, C or X
- Corrections and regrades log
Scores and grades describe published evidence as of the verifiedOn date on each item. They do not measure service quality. Verify with the vendor before contracting.