Skip to content
Test Automation Vendors / 263 cases

Note for buyers

Why a percentage without a baseline is a claim, not a result

Research desk, testautomationvendors.com · published 2026-11-12 · data as of 2026-11-12 · 126 records

246
percentage claims
193
no starting value
53
state a starting value

246 metrics in this dataset are stated as percentages. 193 give no starting value, so the change cannot be checked against a baseline; 53 state one.

What stands out

  1. 0116 vendors publish percentage claims; 193 of the 246 claims give no starting value.

Counts come from verified records. A baseline is the before value a vendor states for a metric; quoted claims carry their grade and baseline flag.

baseline:
The measured value before the work started, against which a later value is compared. source

Examples without a starting value

Each of these states where the vendor ended up, not where it started.

  • “32% lower regression and triage cost after ongoing maintenance was included”a1qa a1qa-1 grade B, partial baseline no method yes
  • “30%+ less setup time on converted tests”a1qa a1qa-2 grade B, partial baseline no method no
  • “Reached 66% test automation coverage”before not stated after 66% a1qa a1qa-4 grade B, partial baseline no method yes
  • “85-90% of pre-release regression automated”before not stated after 85-90% a1qa a1qa-5 grade B, partial baseline no method yes
  • “curtailed the time required for release verification testing by 90%”a1qa a1qa-6 grade C, claim only baseline no method yes
  • “decreasing the download time by 30%”a1qa a1qa-6 grade C, claim only baseline no method yes

Examples with a starting value

These state the value before the work, so the change can be read against it.

  • “Flaky tests cut from 18% to 4.6%”before 18% flaky test rate after 4.6% a1qa a1qa-1 grade B, partial baseline yes method yes
  • “65% of a critical scope automated in 10 weeks”before no automated suite after 65% of the prioritized critical scope a1qa a1qa-3 grade B, partial baseline yes method yes
  • “100% sync success achieved via rigorous end-to-end validation”before Intermittent connectivity with Surescripts caused silent failures and data gaps after 100% sync success BugRaptors bugraptors-4 grade B, partial baseline yes method no
  • “Automated CDS validation ensured 100% reliable alerts”before Drug–drug and drug–allergy checks were inconsistently triggered after 100% reliable alerts BugRaptors bugraptors-4 grade B, partial baseline yes method no
  • “100% regulatory adherence secured by validating dynamic templates and TFA workflows”before Hard-coded logic led to inconsistencies in state-specific labels after 100% regulatory adherence BugRaptors bugraptors-4 grade B, partial baseline yes method no
  • “Scaled automation from a fragmented 20% to a comprehensive 70%”before 20% after 70% BugRaptors bugraptors-6 grade B, partial baseline yes method no

Percentage claims by vendor

Each row counts a vendor's percentage claims and those that state a starting value.

VendorPercentage claimsWith a starting value
TestFort641
DeviQA6241
a1qa202
Zuci Systems190
Codoid160
ThinkSys122
Qualitest111
QAwerk90
TestingXperts91
BugRaptors65
ImpactQA50
QA Wolf40
QualityLogic40
Abstracta20
Testrig Technologies20
QA Madness10

What a baseline adds

A starting value fixes what the percentage is measured against. A measurement period or sample then fixes when and on what it was measured. This site grades a case A only when one metric has both.

Questions

Is a percentage without a baseline false?

Not necessarily. It cannot be checked from the page, so the site shows it as a claim with grade and baseline flag.

How many percentage claims state a baseline?

53 of 246.

What does grade A require?

A named client and one metric with both a stated baseline and a stated method or period.

Where are the claims shown?

In each case file's claim ledger.

Records used

This post reads 126 records; the newest verifiedOn date among them sets the data-as-of date.

Show record ids

a1qa-1, a1qa-2, a1qa-3, a1qa-4, a1qa-5, a1qa-6, a1qa-14, a1qa-22, a1qa-24, a1qa-29, a1qa-38, a1qa-62, a1qa-65, a1qa-66, a1qa-67, a1qa-77, abstracta-2, bugraptors-4, bugraptors-6, codoid-1, codoid-12, codoid-17, codoid-19, codoid-20, codoid-21, codoid-22, codoid-29, codoid-31, codoid-37, deviqa-1, deviqa-2, deviqa-3, deviqa-5, deviqa-6, deviqa-7, deviqa-8, deviqa-9, deviqa-11, deviqa-12, deviqa-13, deviqa-15, deviqa-16, deviqa-18, deviqa-19, deviqa-22, deviqa-23, deviqa-24, deviqa-27, deviqa-28, deviqa-29, deviqa-30, deviqa-32, deviqa-33, deviqa-35, deviqa-36, deviqa-37, deviqa-38, deviqa-40, deviqa-41, deviqa-42, deviqa-43, deviqa-44, deviqa-46, deviqa-47, deviqa-52, impactqa-1, impactqa-2, impactqa-3, qa-madness-2, qa-wolf-1, qa-wolf-2, qawerk-2, qawerk-3, qawerk-4, qawerk-6, qawerk-8, qualitest-1, qualitest-2, qualitest-3, qualitest-4, qualitest-8, qualitest-10, qualitest-12, qualitylogic-1, qualitylogic-2, testfort-1, testfort-5, testfort-7, testfort-8, testfort-9, testfort-10, testfort-11, testfort-12, testfort-13, testfort-14, testfort-15, testfort-16, testfort-17, testfort-23, testfort-24, testfort-25, testfort-26, testfort-27, testfort-28, testfort-29, testfort-30, testingxperts-1, testingxperts-2, testingxperts-3, testingxperts-5, testingxperts-10, testrig-9, testrig-11, thinksys-1, thinksys-2, thinksys-3, thinksys-5, thinksys-6, zuci-systems-1, zuci-systems-2, zuci-systems-3, zuci-systems-4, zuci-systems-5, zuci-systems-6, zuci-systems-7, zuci-systems-8

More from the blog

Scores and grades describe published evidence as of the verifiedOn date on each item. They do not measure service quality. Verify with the vendor before contracting.