Framework note
Apache JMeter: what the published cases show
Research desk, testautomationvendors.com · published 2026-11-12 · data as of 2026-11-12 · 63 records
- 63
- verified cases
- 12
- vendors
- 20 of 63
- name the client
- 61 of 160
- metrics with a baseline
63 verified cases in this dataset name Apache JMeter, published by 12 vendors. Grades: A 2, B 27, C 34. 20 of 63 name the client, and 61 of 160 published metrics state a baseline.
What stands out
- 01a1qa publishes 19 of the 63 Apache JMeter cases; the three vendors with the most publish 41, so a reader is looking at a concentrated set of sources.
- 0261 of 160 metrics in these cases state a before value; across all 639 metrics in the dataset the count is 209.
- 032 of 63 reach grade A, which needs a named client and one metric with both a before value and a stated method.
- 04The field left out most often is start year: 54 of 63 cases do not state it.
Counts come from verified records. A baseline is the before value a vendor states for a metric; quoted claims carry their grade and baseline flag.
- baseline:
- The measured value before the work started, against which a later value is compared. source
01Cases, vendors and grades
The count covers every verified case whose text names Apache JMeter, including vendors outside the ranked 12.
Vendors with cases: a1qa (19), Abstracta (2), BugRaptors (2), Codoid (11), DeviQA (11), ImpactQA (2), QAwerk (1), Qualitest (2), TestFort (9), TestingXperts (2), Testrig Technologies (1) and ThinkSys (1).
Table view
| Item | Count |
|---|---|
| Grade A | 2 |
| Grade B | 27 |
| Grade C | 34 |
| Total | 63 |
02Clients and industries
Named clients appear in 20 cases; 43 keep the client anonymous.
Industries stated: Cryptocurrency wallet (2), Architecture and construction technology (1), Automotive services B2B marketplace (1), Banking (1), Banking software (system integrator) (1), Commercial real estate (IoT access solutions) (1), Connected planning SaaS (1) and Cryptocurrency exchange (1).
Client types stated: enterprise (2), smb (2), public (1) and startup (1).
03Metric kinds and baselines
The Apache JMeter cases publish 160 metrics. 61 state a baseline and 8 state both a baseline and a method.
- “regression testing time was reduced from 5 days to 1.5 days”before 5 days after 1.5 days a1qa a1qa-4 grade B, partial baseline yes method yes
- “reducing the release cadence from over 1.5 months to just two weeks”before over 1.5 months after two weeks a1qa a1qa-5 grade B, partial baseline yes method yes
- “established complete test automation infrastructure from zero to 1,000+ tests”before zero after 1,000+ tests a1qa a1qa-5 grade B, partial baseline yes method yes
- “the number of total handled requests increased from 400 to 46,000”before 400 after 46,000 a1qa a1qa-9 grade B, partial baseline yes method yes
- “100% sync success achieved via rigorous end-to-end validation”before Intermittent connectivity with Surescripts caused silent failures and data gaps after 100% sync success BugRaptors bugraptors-4 grade B, partial baseline yes method no
- “Automated CDS validation ensured 100% reliable alerts”before Drug–drug and drug–allergy checks were inconsistently triggered after 100% reliable alerts BugRaptors bugraptors-4 grade B, partial baseline yes method no
| Metric kind | Metrics | With baseline |
|---|---|---|
| other | 56 | 21 |
| execution-time | 26 | 13 |
| suite-size | 26 | 16 |
| defects | 23 | 1 |
| coverage | 18 | 8 |
| cost | 7 | 0 |
| release-frequency | 4 | 2 |
04Languages, CI tools and clouds named alongside
Languages: Java (14), Python (8), JavaScript (6), TypeScript (6), C# (5), Ruby (2) and Groovy (1).
CI tools: Jenkins (12), Azure DevOps (5), GitHub Actions (5), Bamboo (2), CircleCI (2), AWS CodeCommit (1), AWS CodePipeline (1) and Azure DevOps Server (1).
Device or browser clouds: BrowserStack (14), AWS (4), BlazeMeter (2), LambdaTest (2), AWS Device Farm (1) and Azure (1).
05Migration flows
No verified case tags a migration into or out of Apache JMeter.
06The evidence ladder
Each row counts the Apache JMeter cases that make that disclosure; the rows are separate counts, ordered from the most basic disclosure to grade A. Of 63 Apache JMeter cases, 2 reach grade A, and the rarest disclosure below it is "reach grade a" (2 cases).
The right-hand column gives the same steps for all 263 verified cases, so a reader can set this subset against the whole dataset.
Table view
| Item | Count |
|---|---|
| Name a listed framework | 63 |
| Name the client | 20 |
| Publish at least one metric | 52 |
| State a baseline for a metric | 18 |
| State baseline and method for one metric | 6 |
| Reach grade A | 2 |
| Total | 161 |
| Step | Apache JMeter cases | All verified cases |
|---|---|---|
| Name a listed framework | 63 of 63 | 263 of 263 |
| Name the client | 20 of 63 | 91 of 263 |
| Publish at least one metric | 52 of 63 | 210 of 263 |
| State a baseline for a metric | 18 of 63 | 75 of 263 |
| State baseline and method for one metric | 6 of 63 | 35 of 263 |
| Reach grade A | 2 of 63 | 18 of 263 |
07Frameworks named alongside Apache JMeter
43 of 63 cases name at least one other listed framework. The pairing shows which stacks vendors describe together, not which combination works.
Table view
| Item | Count |
|---|---|
| Postman and Newman | 23 |
| Selenium WebDriver | 17 |
| Appium | 11 |
| Cucumber, SpecFlow, Behave | 9 |
| WebdriverIO | 7 |
| REST Assured | 6 |
| Playwright | 5 |
| Cypress | 4 |
| Total | 82 |
08Where the cases come from
a1qa (19), Codoid (11) and DeviQA (11) publish the most Apache JMeter cases. A vendor with many cases here shows breadth of published work; the grade columns show how much of it can be checked.
| Vendor | Cases | Grade A or B | With a baselined metric |
|---|---|---|---|
| a1qa | 19 | 4 | 3 |
| Codoid | 11 | 3 | 2 |
| DeviQA | 11 | 11 | 10 |
| TestFort | 9 | 3 | 1 |
| Abstracta | 2 | 2 | 0 |
| BugRaptors | 2 | 2 | 2 |
| ImpactQA | 2 | 2 | 0 |
| Qualitest | 2 | 0 | 0 |
| TestingXperts | 2 | 0 | 0 |
| QAwerk | 1 | 1 | 0 |
| Testrig Technologies | 1 | 0 | 0 |
| ThinkSys | 1 | 1 | 0 |
09What vendors leave out
25 of 63 cases do not state a team size and 35 do not state a duration.
7 of 160 metrics are cost figures and 0 mention flakiness, so maintenance cost and flaky test rates are rarely published.
10Questions for a Apache JMeter proposal
Ask for suite size and run time before and after the work: 50 of 63 cases give no before value for either.
Ask for the flaky test rate over a stated number of runs: 0 of 160 metrics mention flakiness at all.
Ask who owns maintenance after hand-over and what it costs per month: 7 of 160 metrics are cost figures.
Ask for a reference client you can call: 43 of 63 cases keep the client anonymous.
Questions
How many vendors have published Apache JMeter cases?
12 vendors in this dataset, with 63 verified cases between them, as of the dataAsOf date on this page.
How many Apache JMeter cases name the client?
20 of 63. A named client is required for grade A.
Do Apache JMeter case studies publish baselines?
61 of 160 metrics state a before value.
Does this page rank Apache JMeter against other frameworks?
No. The site counts which vendors have evidence on a framework and never compares frameworks.
Records used
This post reads 63 records; the newest verifiedOn date among them sets the data-as-of date.
Show record ids
a1qa-4, a1qa-5, a1qa-6, a1qa-7, a1qa-9, a1qa-18, a1qa-20, a1qa-21, a1qa-22, a1qa-28, a1qa-35, a1qa-36, a1qa-60, a1qa-61, a1qa-62, a1qa-65, a1qa-66, a1qa-68, a1qa-69, abstracta-4, abstracta-5, bugraptors-4, bugraptors-6, codoid-1, codoid-4, codoid-7, codoid-13, codoid-14, codoid-28, codoid-31, codoid-32, codoid-35, codoid-37, codoid-38, deviqa-5, deviqa-8, deviqa-9, deviqa-12, deviqa-19, deviqa-21, deviqa-23, deviqa-24, deviqa-25, deviqa-38, deviqa-51, impactqa-2, impactqa-3, qawerk-3, qualitest-6, qualitest-9, testfort-2, testfort-3, testfort-7, testfort-11, testfort-15, testfort-16, testfort-23, testfort-26, testfort-27, testingxperts-6, testingxperts-7, testrig-8, thinksys-9
More from the blog
Framework note
Playwright: what the published cases show
48verified cases
48 verified cases from 14 vendors name Playwright: grades, client types, metric kinds, stacks and the fields vendors leave out.
data as of 2026-11-12 · 48 records
Framework note
Cypress: what the published cases show
25verified cases
25 verified cases from 10 vendors name Cypress: grades, client types, metric kinds, stacks and the fields vendors leave out.
data as of 2026-11-12 · 25 records
Framework note
Selenium WebDriver: what the published cases show
88verified cases
88 verified cases from 13 vendors name Selenium WebDriver: grades, client types, metric kinds, stacks and the fields vendors leave out.
data as of 2026-11-12 · 88 records
Related pages
Scores and grades describe published evidence as of the verifiedOn date on each item. They do not measure service quality. Verify with the vendor before contracting.