Skip to content
Test Automation Vendors / 263 cases

Note for buyers

What an automation coverage figure can mean: the denominators seen in published cases

Research desk, testautomationvendors.com · published 2026-11-12 · data as of 2026-11-12 · 76 records

88
coverage claims
4
denominators seen

The 88 coverage claims in this dataset use at least four denominators: test cases or scenarios, regression or manual effort, features or flows, and code. The same suite gives a different figure under each.

Counts come from verified records. A baseline is the before value a vendor states for a metric; quoted claims carry their grade and baseline flag.

baseline:
The measured value before the work started, against which a later value is compared. source

Denominators found in the quotes

A quote can match more than one row; quotes that match none name no denominator.

DenominatorClaims
share of test cases or scenarios34
share of regression or manual effort13
share of features, flows or requirements15
share of code0
no denominator in the quote40

Examples

Six coverage claims as quoted, with their flags.

  • “65% of a critical scope automated in 10 weeks”before no automated suite after 65% of the prioritized critical scope a1qa a1qa-3 grade B, partial baseline yes method yes
  • “Reached 66% test automation coverage”before not stated after 66% a1qa a1qa-4 grade B, partial baseline no method yes
  • “85-90% of pre-release regression automated”before not stated after 85-90% a1qa a1qa-5 grade B, partial baseline no method yes
  • “helped automate 20% of app regression tests”before not stated after 20% a1qa a1qa-14 grade C, claim only baseline no method yes
  • “helped cover 95% of tests without any tool limitations”before not stated after 95% a1qa a1qa-22 grade C, claim only baseline no method no
  • “achieved 90% automated test coverage across critical user flows”before not stated after 90% a1qa a1qa-29 grade B, partial baseline no method yes

Questions

Which denominator is most common?

share of test cases or scenarios, in this dataset.

Can two vendors' coverage figures be compared?

Only when both state the same denominator and the starting value.

What should a coverage claim include?

The denominator, the starting value and the date or release it was measured on.

Where are coverage claims listed?

In each case file's claim ledger, with kind coverage.

Records used

This post reads 76 records; the newest verifiedOn date among them sets the data-as-of date.

Show record ids

a1qa-3, a1qa-4, a1qa-5, a1qa-14, a1qa-22, a1qa-29, a1qa-62, a1qa-66, a1qa-75, abstracta-2, bugraptors-6, codoid-12, codoid-20, codoid-21, deviqa-1, deviqa-2, deviqa-3, deviqa-5, deviqa-7, deviqa-8, deviqa-9, deviqa-11, deviqa-12, deviqa-13, deviqa-15, deviqa-18, deviqa-19, deviqa-23, deviqa-24, deviqa-27, deviqa-28, deviqa-29, deviqa-30, deviqa-32, deviqa-33, deviqa-35, deviqa-37, deviqa-38, deviqa-40, deviqa-41, deviqa-42, deviqa-43, deviqa-44, deviqa-46, deviqa-47, deviqa-52, qa-madness-2, qa-wolf-1, qawerk-2, qawerk-6, qawerk-8, qualitest-4, qualitest-8, qualitest-10, qualitest-12, testfort-7, testfort-8, testfort-9, testfort-12, testfort-17, testfort-19, testfort-20, testfort-22, testfort-24, testfort-26, testfort-30, testingxperts-3, testrig-1, testrig-11, thinksys-1, thinksys-2, thinksys-5, zuci-systems-3, zuci-systems-4, zuci-systems-5, zuci-systems-8

More from the blog

Scores and grades describe published evidence as of the verifiedOn date on each item. They do not measure service quality. Verify with the vendor before contracting.