Table of Contents

The Bottom Line for 2026:

The best AI test automation tools for 2026, by category: agentic testing with no script (Thunders.ai, QA.tech, Perfecto), autonomous test creation (mabl, testers.ai), visual AI (Applitools), all-in-one and codeless platforms (Katalon, ACCELQ, TestMu KaneAI), open source AI on the framework you already use (Alumnium), AI-authored mobile tests (Maestro), and AI failure analysis inside your pipeline (Testkube).

I’ve spent 25+ years in QA and interviewed 600+ automation experts on the TestGuild podcast, and most “AI testing tools” are still GPT wrappers. The 12 below are the ones I’ve used, demoed, or grilled the founders on, and they’re the ones I’d actually shortlist.

If you want a shortlist for your specific stack, use my free Tool Matcher. Otherwise, here are the 12 AI testing tools worth your time.

Why you can trust this list: Every tool here is one I’ve covered on TestGuild with the people who actually build it or use it. The quotes you’ll read come straight from those podcast interviews. They’re real conversations, not AI-generated summaries. Many of these companies also sponsor TestGuild, and every one of them is tagged “TestGuild sponsor” right under its name. Sponsorship buys access to my audience, not a spot on this list. Some sponsors aren’t on it, and at least one tool that has never paid me is.

AI Testing Waves

I first wrote about the AI “three waves” back in 2017, and honestly?

I thought I was just documenting a trend.

Turns out, I was watching a revolution.

(Quick note: I first wrote this in 2017, updated it in 2023, and I’m updating it again now in 2026. Two tools that were on this list are gone, which tells you something about how fast this is moving.)

Here’s what’s wild: back in 2017, when I first started talking about AI in testing on my TestGuild podcast, people thought I was overselling it. “Joe, is this just hype?” they’d ask. In our State of Automation 2027 survey, only 6.4% of the 485 testers who answered said their team isn’t using AI in testing at all. 40.8% said it’s already standard practice across their teams.

The question isn’t “should we use AI?” anymore. It’s “which AI tool won’t waste our time?”

AI test manager

testers.ai (CARBON)

Open source, runs in your coding agent

Agentic workflows

Mabl

Autonomous test agents

QA agents in your workflow

Amikoo

Agents test every PR and Jira story

Visual testing

Applitools

Visual AI pioneer

Codeless automation

ACCELQ

Generative AI test creation

AI failure analysis

Testkube

AI root cause analysis in your own cluster

Agentic, no script

Perfecto

Goal-based tests across web and mobile

LLM-powered

TestMu KaneAI

Natural language tests

Open source AI

Alumnium

AI layer on Selenium, Playwright, Appium

Plain-English agents

Thunders.ai

Tests written in business language

PR verification

QA.tech

Agents verify every pull request

Mobile AI

Maestro

AI-authored, deterministic mobile tests

Find the Right Test Tool For You Now with our Tool Matcher

Frequently Asked Questions About AI Test Automation Tools

Will AI test automation replace QA engineers?

No, and I’ll tell you why after interviewing 600+ testing leaders on TestGuild Podcast, AI handles the grunt work that burns out good engineers. We’re talking test generation, flake detection, self-healing selectors, the repetitive stuff that makes testers want to quit. But here’s what AI can’t do: understand business risk, know which bugs actually matter to users, or make judgment calls when your test results are ambiguous. The teams I’ve talked to at Automation Guild who are winning with AI? They’re using it to amplify their engineers’ impact, not replace them. Test everything and keep the good, including the human expertise.

Are AI test automation tools ready for real production use?

Some are, most aren’t. I’ve seen this firsthand covering AI testing tools for TestGuild over the past few years,the vendors who are honest admit they’re solving specific problems, not replacing your entire test stack. Tools focused on targeted use cases like Selenium self-healing, visual regression analysis, or Playwright test generation are in production CI/CD pipelines right now. Full autonomous testing with zero human oversight? That’s mostly conference demo magic. On TestGuild Podcast, I’ve talked to engineering teams at companies like Microsoft and Google who use AI in production, but they’re extremely selective about which problems they’re solving with AI versus traditional automation.

How do I choose the right AI testing tool for my team?

Start with the pain, not the hype. What’s actually breaking your workflow right now, flaky tests eating your CI/CD time? Maintenance hell every time the UI changes? Coverage gaps you can’t close with your current headcount? Then find the tool that solves that specific problem for your stack. Don’t buy a “complete AI testing platform” that does everything poorly. I built the TestGuild Tool Matcher specifically for this: answer a few questions about your tech stack and pain points, and it’ll recommend tools that actually fit your situation. And if you’re evaluating tools, ask vendors the hard questions I ask in my podcast interviews: How does this integrate with our existing framework? What’s the learning curve for my team? Can we export the tests if we need to leave?

Do AI testing tools work with Playwright, Selenium, or Cypress?

Most modern AI tools integrate with the big frameworks, Playwright, Selenium, Cypress, and even older stacks like WebDriver, but the quality of that integration varies wildly. Some tools bolt onto your existing tests through plugins or APIs. Others generate tests in your framework’s native language (like JavaScript for Playwright or Python for Selenium) so you own the output. The sketchy vendors lock you into their proprietary format and make migration painful. When I interview AI tool founders on TestGuild Podcast, I specifically ask them about framework compatibility because I know that’s where teams get burned. Before you commit to any AI testing tool, run a proof of concept with your actual application and framework, not just their demo app.

The Three Waves: How We Got Here

Before I dive into tools, you need to understand this framework.

It’ll help you cut through the AI washing and marketing BS that’s everywhere now.

First Wave: The Vendor Lock-In Era (1990s-2000s)

I cut my teeth on WinRunner. Man, I loved that tool. Then Mercury killed it for QTP, and my heart broke a little.

That was the first wave – proprietary tools that locked you in:

The problem? When the vendor pivoted (or died), you were screwed. Your entire test infrastructure could become obsolete overnight.

Enter the second wave.

Second Wave: Open Source Changes Everything (2004-2020)

Then Selenium happened. And Selenium changed everything.

I’ve interviewed dozens of people on my podcast about this shift – from Jason Huggins (Selenium’s creator) to folks building Cypress and Playwright.

The second wave was all about:

Open source – Free, community-driven, no vendor lock-inWeb-first – Built for the modern web app explosionDeveloper-focused – Real programming, not wizard-driven nonsenseExplosion of tools – Cypress, Playwright, Appium, and hundreds more

But here’s what nobody talks about: this wave just moved the pain. Instead of paying vendors, you paid engineers. Instead of brittle record-playback, you got brittle selectors. Different problem, same headache.

By 2017, we were seeing early ML attempts – basic self-healing, visual AI.

But the real AI explosion? That didn’t hit until ChatGPT launched in late 2022.

Suddenly everyone was racing to add LLMs to their testing tools.

Third Wave: AI That Actually Works (2020-Present)

Here’s where we are now. And I’m not gonna lie – after interviewing hundreds of testing experts on TestGuild, I’m cautiously optimistic about this wave.

What makes a tool “third wave”?

The big shift? Third wave tools don’t just run your tests faster. They actively reduce the maintenance burden that’s been killing teams since the Selenium days.

Look, I’m a skeptic by nature. But after testing these tools myself and talking to teams using them in production, this is real. Not perfect.

Not magical. But real.

Join Test Automation Training

The 12 Tools Actually Worth Your Time

Alright, let’s get into it.

I’ve personally tested, used, or extensively interviewed founders/users of every tool here.

No fluff, just what I’ve learned.

This is not a ranking. The numbers are only there to keep count. A tool showing up before another one doesn’t mean I think it’s better. Each tool is here because it’s a strong fit for a specific problem, so start with your pain point, not the number.

1. testers.ai CARBON: An Open Source AI Test Manager Inside Your Coding Agent

Podcast Connection: Founder Jason Arbon tested Chrome at Google and co-wrote How Google Tests Software. He knows what testing at real scale looks like and has been on the TestGuild Automation Podcast many times – most recently episode 600 Testing AI: Engineering Confidence.

Why It’s Here: When we asked testers in our State of Automation 2027 survey to name their team’s primary testing tool, 1 in 6 (64 of 388) named an AI coding assistant like Claude, GitHub Copilot or Cursor instead of a testing tool. That sounded weird to me at first. Then it clicked: that’s where the code is getting written now, so that’s where testing has to happen. CARBON is Jason’s answer to that shift.

What It Is: CARBON is an open source testing harness that runs inside your coding agent. Its AI test manager, Jay, reads your project and your recent changes, decides what needs testing, runs the checks, and reports back with an honest read on how confident you should be about shipping. It runs in Claude Code, Cursor, Codex and other MCP clients, and it works with what you already have: Playwright, Selenium, pytest, Jira, GitHub Issues, TestRail and Xray.

This is basically the opposite of vibe testing. testers.ai makes the same point I keep making on the podcast: a test your agent generated isn’t a test that passed. And CARBON doesn’t try to replace your framework. It sits on top of it.

Real Talk: Jason has been on multiple TestGuild podcasts and Automation Guild sessions, and his own rule applies here. As he told me on episode A600: “strong claims require strong evidence.” So run it on your own app before you believe anyone’s claims, including his.

Best For: Teams where developers and agents are writing code faster than QA can review it, and testers who already live in Cursor or Claude Code

Pricing: Free, MIT-licensed open source core. Paid Pro, Team and Enterprise tiers add more commands, specialist investigations and test history. Model usage is billed by your agent’s provider.

Check it out: testers.ai · Listen to episode A600 with Jason Arbon (testguild.com/podcast/a600-jason/)

2. Mabl: Agentic AI (Finally Living Up to the Hype)

Podcast Connection: I’ve been following mabl since they started, and had them on TestGuild to talk about their agentic workflows.

When mabl talks about “agentic” now, it doesn’t just mean an AI that drafts a test for you. In 2026 mabl started calling itself an “agentic testing harness”: agents that build, run, investigate and maintain tests, built to independently check what your coding agents ship. In October 2026, Gartner named mabl a Challenger in its first Magic Quadrant for Agentic Software Quality Assurance Platforms.

What’s New in 2026:

My Take: I’ve seen a lot of tools claim “autonomous” testing. Mabl is one of the few actually delivering on it. Their approach to test creation from user stories is legitimately impressive.

Real Results I’ve Heard:

Best For: Teams adopting coding agents that want an independent check on what those agents ship, and teams that want web, mobile, API, accessibility and performance testing in one platform

Pricing: Quote-based, with a free trial.

Learn more: mabl.com

3. Amikoo: QA Agents That Run On Every Pull Request (NEW)

Podcast Connection: Ivan Barajas co-founded MuukTest, which launched a no-code AI test automation platform in early 2020, years before ChatGPT. He came on episode A592 to show me Amikoo, which his team built from scratch on six years of those lessons. (Amikoo means “friend” in Mayan, which tells you how they think about it.)

What It Is: You connect your test repo (Selenium, Cypress or Playwright), and optionally your app repo, Jira, Azure DevOps or GitHub. In about 15 minutes you can ask it anything about your testing: what’s my coverage, what negative tests am I missing, what should I run for these pull requests. Under the hood, as Ivan put it, “we design a harness of agents and tools. We have 12 agents and 43 tools.” There’s a test design agent, an SDET agent that writes Playwright with page objects, and a repair agent that’s always hunting for flaky tests.

My Take:

What Was Coming: In the interview, Ivan said MCP support, a Slack integration, automatic test suggestions on new pull requests, and confidence scores on whether a failure is flaky or a real defect were all weeks away.

Best For: Testers and SDETs stuck deciding what to run for each change, teams whose developers now ship more AI-written code than QA can keep up with, and non-technical testers who want an assistant that explains itself

Pricing: Free tier plus paid plans. Ivan offered TestGuild listeners double the free tier if you mention the podcast when you sign up. [CONFIRM the offer still stands]

Check it out: amikoo.ai · Episode A592 with Ivan Barajas · My write-up

4. Applitools: Visual AI That Made Me a Believer

Podcast Connection: I interviewed founder Adam Carmi back in the early days (listen to episode 43), and he’s been back on the show multiple times.

I’ll be honest – when Adam first told me about visual validation testing in 2015, I thought it was BS. “An algorithm that finds bugs without explicitly defining elements? Come on.”

Then I tried it. And my skeptical mind was blown.

Why Applitools Is Different: No pixel-by-pixel comparisons. No fragile baseline images. Their Visual AI actually understands what matters visually and what doesn’t.

What’s New in 2025:

Real Story: One company saved a million dollars a year by replacing thousands of assertion lines with visual checkpoints. A MILLION. DOLLARS.

My Take: If you’re doing any UI testing and not using Applitools, you’re working too hard.

Best For: Visual regression testing, cross-browser validation, teams obsessed with UI/UX quality

Pricing: Starts at $199/month

Try it: applitools.com

5. ACCELQ: Generative AI Gets Real

What I’ve Seen: ACCELQ’s approach to generative AI is different than most. They’re using LLMs to actually understand test intent, not just generate scripts.

Key Features:

My Take: The “logic insights” feature is underrated. It’s like having a senior test engineer review your work and suggest improvements.

Best For: Teams wanting to scale test coverage fast, organizations moving from manual to automated

Pricing: Custom enterprise pricing

Learn more: accelq.com

6. Testkube: AI Test Agents That Run Inside Your Own Infrastructure (NEW)

Webinar Connection: I hosted a TestGuild webinar with Testkube CTO Ole Lensmar (yes, the guy who created SoapUI) about Using AI to Close the Gap Between AI-Written Code and Human-Tested Code where they highlight all the new development with TestKube.

What It Is: Testkube runs the tests you already have (Playwright, Cypress, Selenium, k6, JMeter, Postman, pytest and more) inside your own Kubernetes clusters. You trigger them from any CI, a schedule, the CLI, the API or a change in your cluster, and every run, log and artifact lands in one dashboard. Ole was clear about what it isn’t: “We’re also not a testing tool. We’re not here to replace Playwright.”

What’s New: AI On Top Of The Runner. This year Testkube added an AI layer that works on all that execution data:

My Take:

Best For: Platform and DevOps-minded QA teams on Kubernetes, regulated teams that can’t send test data to a vendor cloud, and anyone whose CI time goes to triaging failures instead of fixing them

Pricing: The open source agent is free to self-host. The dashboard has a free tier, and paid tiers are priced by environments and users.

Check it out: testkube.io 

7. Perfecto by Perforce: Agentic AI With No Script At All (NEW)

Podcast Connection: Don Jackson walked me through Perfecto’s agentic AI on episode A554, and I did a hands-on review on the TestGuild YouTube channel.

This is the tool that made me add a whole “fourth wave” section to this article. Instead of AI that writes and heals scripts, you give it a goal in plain language and it figures out how to reach it at runtime, by looking at the screen.

What It Is: An enterprise web, mobile and desktop testing platform. Perforce says you can create one AI-generated test and run it across iOS, Android, web and desktop.

My Take: It’s slower than a script and you have to learn to write good goals. But for apps where scripting is miserable (Salesforce Lightning, dynamic data, mobile flows that change every sprint), it’s the most interesting thing I’ve seen from an enterprise vendor. Full breakdown, including the weather-app demo that blew my mind, is in the Fourth Wave section below.

Best For: Enterprise teams testing on real devices, Salesforce Lightning and other dynamic UIs, non-scripters who know the business

Pricing: Custom enterprise pricing

Check it out: perfecto.io 

8.  TestMu (Formally LambdaTest) KaneAI: Modern LLMs Meet Testing

What’s Different: LambdaTest is now TestMu AI, and KaneAI is their end-to-end testing agent. You describe a test in plain English, and KaneAI plans, writes, runs and updates it across web and mobile. Because it’s TestMu, you get their whole cross-browser and real device cloud behind it.

Key Features:

My Take: This is where testing is heading: conversational interfaces powered by modern AI. Check out my podcast episode all about this, AI as Your Testing Assistant with Mudit Singh (A520).

Best For: Teams wanting cutting-edge LLM tech, cloud-based cross-browser testing

Pricing: See testmuai.com

Check it out: testmuai.com/kane-ai/

9. Alumnium: The Open Source One (NEW)

Podcast Connection: Alex Rodionov, a tech lead on the Selenium project, came on episode A587 to talk about Alumnium, which he built on the side.

If Cucumber plus AI or “no more locators” is why you came to this page, start here. It’s free, and you don’t have to talk to a salesperson. Alumnium sits on top of Selenium, Playwright or Appium, so you keep the framework you already have, and it works out what to click at runtime from the accessibility tree. What Alex told me about its limits, which vendors rarely admit, is in the Fourth Wave section below.

Best For: Anyone who wants to try AI at runtime without a procurement conversation, and the honest answer if your boss asks whether there’s a free option

Pricing: Free (you pay your own LLM costs)

Check it out: github.com/alumnium-hq/alumnium

10. Thunders.ai: English Is The Program (NEW)

Podcast Connection: Founder Karim Jouini came on episode A578. He spent years at Microsoft on Visual Studio and sold his last company, Expensya.

If you came here for BlinqIO’s “virtual tester” idea, this is the closest thing on the list. You write what you want checked in plain language, and AI test agents run it every time. No code generated, nothing to heal. Why Karim refuses to generate any more Playwright is in the Fourth Wave section below.

What It Is: Thunders says its AI test agents build, run and maintain end-to-end tests across web, mobile, API and desktop.

Best For: Teams drowning in maintenance after a redesign or platform migration, and teams where the domain experts aren’t the automation experts

Check it out: thunders.ai

11. QA.tech: Validating Intent Instead Of Clicks (NEW)

Podcast Connection: I’ve worked with the QA.tech team more than once, and co-founder Vilhelm von Ehrenheim came on episode A590.

QA.tech crawls your app to build what they call a behavioral knowledge graph, then agents use it to check changes on every pull request without scripted tests. Vilhelm’s case for why the bottleneck moved to review is in the Fourth Wave section below.

Best For: Teams that have already sped up development with AI and are now choking on verification

Check it out: qa.tech

12. Maestro: Mobile Testing With AI in the Authoring, Not the Run (NEW, mobile category)

Podcast Connection: Co-founder and CEO Leland Takamine came on episode A589 and built a mobile test live with an AI agent.

This list didn’t have a real mobile pick until now. Maestro is open source, mobile-first, and the tests are human-readable YAML files that live in your repo. A coding agent like Claude or Cursor writes the flow through the Maestro MCP server, then the flow runs the same way every time in CI. Why Leland pulled AI out of the runtime is in the Fourth Wave section below.

Best For: Mobile teams on iOS and Android, and anyone who wants AI in authoring rather than execution

Check it out: maestro.dev

What Happened to BlinqIO, Testim, Katalon, TestResults.io, Parasoft and BrowserStack Test Observability?

If you’ve been here before, you’ll notice some tools moved off the numbered list. Here’s why, so nobody thinks I quietly deleted them.

BlinqIO (shut down, 2026). BlinqIO used to be #1 on this list. Guy Arieli and Tal Barmeir told me directly they were shutting down. They worked with me every month for a couple of years, and then they were gone. Their idea, treating Cucumber as a language for talking to AI rather than a language for writing tests, was one of the more interesting ones anybody had in this space, and our conversation on episode A485 still holds up. If you came for the Cucumber plus AI angle, try Alumnium (#9). If you want the virtual tester model, look at Thunders.ai (#10) or QA.tech (#11).

If you’ve been here before, you’ll notice some tools moved off the numbered list. Here’s why, so nobody thinks I quietly deleted them.

BrowserStack Test Observability (discontinued, 2026). BrowserStack’s product team reached out to let me know Test Observability was discontinued, which I appreciate. Not every vendor bothers. They pointed me toward BrowserStack Low Code Automation, which is a different product solving a different problem, so I’m not swapping it in until I’ve used it. (If you go looking, the old Test Observability URL now redirects to BrowserStack’s Test Reporting & Analytics page.) (TestGuild sponsor.)

Testim (part of Tricentis). Testim was one of the early ML-locator tools, and I respect how focused they were on flaky tests. But I haven’t seen meaningful updates in a long time, and it now sits inside Tricentis.

Tricentis. Tricentis is still the big enterprise suite for SAP, Salesforce and large packaged-app portfolios. If that’s your world, put it on your shortlist. I took it off the numbered list because none of my 2026 interviews covered it, so I can’t vouch for its newer AI features firsthand right now.

Katalon. Katalon shows up on almost every list like this one, and it used to be on mine. But I haven’t used it myself or talked to teams running it in a ling time, and this list is only tools I can vouch for. If it’s on your shortlist, run the same proof of concept I’d run on anything here: your app, your framework, your five hardest tests.

TestResults.io. Still around, and still an interesting selector-free approach. I’d been on the fence about whether it was an “AI” tool as much as a different way to test, so it made room for Perfecto. My full write-up is here: TestResults.io review (testguild.com/testresultsio-review/).

Parasoft Test Impact Analysis. It’s a good tool, but it solves a different problem: picking which tests to run when code changes. That’s not AI test automation, so it didn’t belong in the numbered list. If you’re running your whole regression suite on every commit, it’s still worth a look, and my episode with Wilhelm Haaker and Daniel Garay is here (A569). (TestGuild sponsor.)

Overwhelmed? Use My Free Tool Matcher

Look, I get it !

That’s alot of tools is a lot to process. That’s exactly why I built the TestGuild Tool Matcher.

Answer a few quick questions about your tech stack, budget, and testing goals, and it’ll shortlist the best options from over 300 tools (including all the ones in this article).

Takes about 60 seconds. Completely free. No email required, no sales BS. Just a straight answer about what tools actually fit your needs.

Try the Tool Matcher now →

How to Actually Choose (By Pain Point)

By Team Size

By Primary Pain Point

By Technical Skill

Non-Technical Team:

Mixed Skills:

Highly Technical: Any of these work, and testers.ai CARBON, Alumnium, Maestro and Testkube will feel most at home. Focus on integration capabilities.

Real Talk: Is AI in Testing Just Hype?

I get asked this on every podcast episode. Here’s my honest answer:

In 2017? Yes, mostly hype.

In 2023? Getting real, but oversold.

In 2026? It’s mainstream, and now we’re finding out which vendors can survive it. Two tools on this list shut down or got discontinued in the past year. The question isn’t “is it hype?” anymore. It’s “which of these will still be here in eighteen months?”

After interviewing Jason Huggins (Selenium creator), Jason Arbon, Alex Rodionov, Ivan Barajas, Vilhelm von Ehrenheim, Leland Takamine, Jim Trentadue, and dozens of other testing leaders on TestGuild, here’s what I’ve learned:

AI won’t replace QA engineers. But it’s already changing what they do:

The teams winning aren’t the ones avoiding AI. They’re the ones figuring out how to work WITH it.

Wait… Is There Already a Fourth Wave?

Podcast Connection: I just had Don Jackson on episode A554, and what he showed me made me question everything I thought I knew about where automation testing is heading.

Don recently joined Perfecto (now part of Perforce), and their new agentic AI approach is fundamentally different from every tool in this article. Here’s why:

Third Wave vs Fourth Wave: The Critical Difference

Third Wave Tools (everything above in this guide):

Perfecto’s Fourth Wave Approach: No script. Ever.

Instead, you write goal-oriented prompts in natural language. Don’s example from our podcast:

“Book a flight from San Francisco to New York in business class, prefer an aisle seat, second preference is window seat. If there are no flights available that have one of those seats, I don’t want to sit in the middle. Come back with an error message.”

That’s it. That’s your entire “test.”

How It Actually Works

At runtime, the AI:

Don’s tagline says it all: “No scripts, no frameworks, no maintenance”

Real Example That Blew My Mind

I asked Don about reliability concerns. He told me about testing a weather app:

He wrote: “If the app isn’t installed, go install it.”

What the AI did autonomously:

No explicit loop scripting. No device-specific logic. No progress bar waits coded. Just one simple prompt.

As Don said on the podcast: “Think about how hard that would be to script today.”

What Makes This “Fourth Wave”?

The difference is agency – real, autonomous decision-making:

Third Wave Example:

AI generates: click(‘#login-button’)type(‘#username’, ‘[email protected]’)type(‘#password’, ‘password123’)click(‘#submit’)

Script is created and executed.

Fourth Wave Example

Prompt: “Log into the application”

AI figures out HOW at runtime based on what it sees.

Things That Were Previously “Untestable”

Don mentioned several beta customers finding use cases nobody expected:

Financial Services Company: Stock trading app with dynamic graphs. The AI can now validate:

E-commerce Company: Product images with descriptions

Accessibility Testing: One prompt: “Make sure this page matches WCAG 2.0 standards”

The AI grabs those standards, checks compliance, reports back. Done.

My Honest Take (The Skeptic’s View)

Look, I’ve been in automation for 25+ years. I’ve seen a LOT of “revolutionary” promises turn into vaporware.

When Don first described this, I thought it was interesting theory. When he demoed it, I was intrigued. Now that it’s shipping, here’s where I’ve landed.

The idea I keep coming back to in my talks is the Jevons Paradox of testing. When something gets cheaper, you don’t use less of it. You use more. AI made writing code cheap, so there’s more code than ever, and every line of it needs testing. More AI means more testing, not less. That’s why goal-based tools like this matter: nobody can hand-script their way through that volume.

But testers aren’t fully sold yet, and they shouldn’t be. In our State of Automation 2027 survey, 40.8% of teams say AI is already standard practice in their testing. Yet “trust in the output” is still one of the top reasons teams hold back from using more of it. That’s the right instinct. My rule: don’t vibe test. If you can’t see why an agent passed or failed, you don’t have a test. You have a guess.

The Good:

The Trade-offs:

The Real Question: Is this production-ready today?

For some use cases, absolutely. For dynamic UIs like Salesforce Lightning (Don’s example), for exploratory testing, for applications that change frequently.

For high-speed regression suites where you need maximum performance, or a release gate where you need the exact same result every run? Maybe not yet.

The Controversial Take: Scripters vs Testers

Don said something on the podcast that’s going to upset some people:

“Some of the best testers I’ve known in my career are the worst scripters. And conversely, some of the best scripters were the worst testers because they didn’t have that destructive mindset. Wouldn’t it be amazing if I could have my best testers be able to do automation?”

I’ve seen this my entire career. The business experts who understand the domain can’t automate. The automation experts don’t understand the business context.

In 2026, that gap is closing from both sides. Domain experts can now write tests in plain English. Karim Jouini’s point stuck with me: a mortgage is a mortgage, no matter which version of the core banking system you’re on. And scripting itself got cheap. As Ivan Barajas told me, AI helpers are the new no-code tools, so what’s scarce now isn’t writing scripts. It’s the tester’s mindset and knowing how to architect automation that scales.

Skills are still the sticking point, though. In our State of Automation 2027 survey, lack of skills or training was the second most common reason teams gave for not using AI more widely. Only “nothing is blocking us” ranked higher.

Exploratory Testing, Automated

This is what really got me excited. Don described a beta customer who asked the AI:

“Find all the different paths to get to the shopping cart.”

The AI found 12 paths.

The customer only knew about 9.

Think about that. Automated exploratory testing that discovers things your manual testers missed.

Should You Adopt This Now?

Immediate Use Cases:

Wait a Bit If:

My Prediction

In our podcast conversation, Don mentioned he’d been talking about this concept for 18 months and calling it “goal-oriented testing.” The fact that multiple companies (including Perfecto) are now building this approach tells me something:

This is where testing is going.

Not in 10 years. In the next 2-3 years.

The tools in the main part of this article (third wave) are amazing and will continue to evolve. But I think we’re watching the fourth wave emerge right now.

Check it out: Perfecto – Look for their Agentic AI features (released July 15, 2025)

Watch my hands-on review: TestGuild YouTube Channel – I did a deep dive showing this in action

Hear the full conversation: TestGuild Podcast Episode A554 with Don Jackson

Five More Fourth Wave Tools I’ve Had On The Podcast This Year

Since I wrote the Perfecto section, I’ve had a run of conversations with people building in this exact space. Some of them agree with Don. Some of them think he’s wrong. That disagreement is the most useful thing I can show you, because it means nobody has this solved yet. (All five made the numbered list above.)

Thunders.ai: English Is The Program

TestGuild sponsor. (See #10 above)

Podcast Connection: I talked with founder Karim Jouini on episode A578. He spent years at Microsoft on Visual Studio and sold his last company, Expensya.

Karim’s position is the most extreme version of fourth wave I’ve heard. There’s no code generated. There’s no code cached. When I asked what’s underneath, his line was blunt: “There is nothing underneath.”

He calls it a manual tester on steroids. You write what you want checked, in plain language, and agents go do it every time you run. He had over a million lines of Playwright at Expensya, which is exactly why he refuses to generate any more of it.

The number that got my attention: a bank planned three years of testing for a core banking upgrade and did it in four months, because the tests were written in business language and survived a UI that changed completely. As Karim put it, a mortgage is a mortgage regardless of what version of the core banking system you’re on.

Best For: Teams drowning in test maintenance after a rebrand or a platform migration, and teams where the domain experts aren’t the automation experts.

QA.tech: Validating Intent Instead Of Clicks

TestGuild sponsor. (See #11 above)

Podcast Connection: I spoke with Vilhelm von Ehrenheim on episode A590 about why the bottleneck in software moved instead of disappearing.

Fully agentic, but the piece that’s different is the knowledge graph underneath. Most agentic tools get more expensive as your app gets bigger, because every run means feeding more context to the model. QA.tech builds a map of your application, and Vilhelm’s claim is that the map cuts token usage rather than growing it.

His broader point stuck with me more than the product did. AI sped up coding, but as he put it: “The bottleneck didn’t go away completely.” If your team is now producing ten or fifteen pull requests a day, who is reading all of that? You didn’t remove a bottleneck. You moved it to review.

Best For: Teams that have already sped up development with AI and are now choking on verification.

Alumnium: The Open Source One

Not a sponsor. (See #9 above)

Podcast Connection: Alex Rodionov is a tech lead on the Selenium project. He built Alumnium on the side and it’s free.

This is the one I’d try first if you’re curious and don’t want to talk to a salesperson. Alumnium sits on top of Selenium, Playwright or Appium, which means you keep the framework you already have. There are no locators. It works out what to click at runtime and heals on the fly, and Alex believes it’s the only open source project doing that across iOS, Android and Chrome.

It runs on cheap low tier models.

He’s also honest about the limits in a way vendors aren’t: “You can’t just throw things at LLM and hope that it will work as accurately and as reliably as it works in small contexts.” Models degrade well before their context window fills, which is why a hundred step test falls apart when a ten step test works fine. That’s the kind of thing you only hear from someone with nothing to sell.

Best For: Anyone who wants to try AI at runtime without a procurement conversation. Also the honest answer if your boss asks whether there’s a free option.

Maestro: The Tool That Killed Its Own AI Feature

TestGuild sponsor. (See #12 above)

Podcast Connection: Leland Takamine built Maestro. It’s used for mobile end to end testing at Microsoft, Amazon, DoorDash and Meta, and the React Native core team uses it to test the framework itself.

I’m including Maestro on an AI tools list for a reason that might seem backwards. Leland built exactly the product everyone in this section is selling. Write your test in plain English, agent runs it, code cached underneath. He shipped it. Then he shut it down, because his users kept coming back to plain YAML.

His conclusion is worth more than most product pitches: use AI to author the test, then run it deterministically. As he told me, “we don’t want AI at runtime.” Putting the model at runtime costs you the one property an end to end test is supposed to have.

There’s a Maestro MCP server, so Cursor or Claude Code can drive your app, explore it, and write the flow for you. The flow is a YAML file that sits in your repo next to your code. No SDK, no hosted interface required, no proprietary syntax.

Best For: Mobile teams, and anyone who wants the AI in authoring rather than execution.

Amikoo: The Intern You Have To Train

TestGuild sponsor. (See #3 above)

Podcast Connection: Ivan Barajas co-founded MuukTest and built Amikoo on six years of AI test automation lessons. He came on episode A592.

Ivan sees more testing coming, not less. His customers are shipping ten times more code with Claude Code and Codex, and every line of it needs checking. In his words, “testing becomes more critical.” Amikoo is his bet on how testers keep up: an assistant you ask what to test, not a robot that tests without you.

Ivan gave me the most sensible framing anybody has for how to actually deploy these things. In his words, “you can see a QA agent like [Amikoo], like an intern or a new software engineer right out of school.” Give it your P2s and P3s. Keep a human on P0 and P1, always. That’s advice from someone selling the agent, which is why I trust it.

He also warned about something I’ve watched teams walk into: the jump from a few dozen AI-written tests to a few thousand. It’s easy to get 10 or 20 running with an LLM. Past that, as he put it, “Test automation at hundreds or thousands of tests, it’s a formal software project.” His advice for SDETs follows from that: writing scripts is cheap and fast now, so learn test automation architecture.

Best For: Teams that want an AI helper for testers, not a replacement for them.

One More, Different Category: Ito.ai

Not a test automation tool, so it doesn’t get a number on this list, but it’s solving something adjacent that I keep hearing about.

Ito.ai reviews pull requests, except it actually runs your code in a throwaway environment instead of reading it. It collects screenshots, logs and console output, so when it flags something it has evidence rather than a guess.

Evan Marshall, who built it, gave me the single most useful piece of advice in any of these conversations, and it costs nothing to apply: “separate your end to end test layer away from your code generation.” What happens otherwise is an agent implements a feature, the tests fail, and the agent rewrites the tests to confirm whatever it just did.

If you’re letting agents open pull requests, do that one thing today.

Check it out: ito.ai · Listen to Podcast episode A593 with Evan

My Actual Recommendation

Stop overthinking it. Pick 2-3 tools from this list based on your primary pain point. Get trial access. Build the same 5 tests in each. See which one clicks with your team.

For the adventurous: Try Perfecto’s new agentic AI on one particularly painful automation scenario. See if runtime decision-making works better than scripting.

Don’t wait for perfect. Start experimenting this quarter.

The teams I see succeeding with third-wave tools aren’t necessarily the ones with the biggest budgets or most engineers. They’re the ones who started early and learned by doing.

And the teams that will lead in the fourth wave? They’re experimenting with these agentic approaches RIGHT NOW.

Free TestGuild Courses

Stay Connected

Want more? Here’s how to keep learning:

TestGuild Podcast: Every week I interview testing leaders about what’s actually working. We’ve covered AI testing extensively with folks from BlinqIO, Applitools, Testim, Reflect, and many more. Subscribe here

Automation Guild Conference: My annual online conference brings together the biggest names in test automation. We’ll have sessions specifically on AI testing tools. Learn more

Weekly Newsletter: I send out weekly updates on the latest tools, trends, and techniques. No BS, just actionable insights. Join 40,000+ subscribers