What Is Smoke Testing? A Complete Guide for SaaS Teams

You shipped a change an hour ago. CI was green, the deploy went through, and your status page still says everything is fine. Is signup actually working right now? Most teams can't answer that without clicking through it by hand.

Smoke testing is how you answer it automatically. A smoke test is a short set of automated checks that run against a build to confirm its most critical functions still work. Can a user log in, finish onboarding, pay? If the smoke tests pass, the build is stable enough to ship or test further. If they fail, you stop and fix before spending time on anything deeper.

The name comes from hardware. When you power on a new circuit board for the first time, you watch for smoke. If it smokes, you stop. There's no point testing anything else until the basic problem is fixed. Software smoke testing works the same way: prove the basics before you trust the rest.

Where smoke testing fits

Smoke testing sits between a new build and everything else. It runs before regression testing, before manual review, before a release candidate gets promoted.

Code commit
   ↓
Build
   ↓
Smoke tests  ← here
   ↓ (pass)
Regression / deeper testing
   ↓
Staging
   ↓
Production deploy

If you ship through CI/CD, smoke tests are the first automated gate after a build succeeds. They don't cover everything. They confirm the product can do the handful of things it exists to do, before any slower, deeper testing runs.

For a SaaS product, that usually means four questions:

• Can a user log in?
• Does the core feature load and work?
• Can a user complete the primary action, like create a project, run a report, or send a message?
• Does billing go through without errors?

If those four work, the build is stable. Everything else is refinement.

What a smoke test actually checks

Smoke tests check the critical path: the sequence of actions a user has to be able to take for your product to do what you say it does.

For most SaaS products, that path looks like this:

None of these are edge cases. They're the tests your product lives and dies by, and if one of them is broken you have an incident, one your uptime monitor will happily report as 200 OK.

A smoke test doesn't check every browser, every permission, every odd state. It checks the critical path, fast, after every meaningful change.

Smoke vs. sanity vs. regression testing

These three terms get used interchangeably, which causes real confusion. They answer different questions:

The order in practice is smoke, then sanity, then regression. You run smoke first because there's no point running a two-hour regression suite on a build where login is broken.

When to run smoke tests

After every deploy. That's the short answer.

In a CI/CD pipeline, smoke tests are the first automated check on every branch push, pull request, and production deploy. They're fast enough, under five minutes, that they don't slow anything down. Specific triggers worth wiring up:

• Pull request opened. Run smoke against the preview URL before anyone reviews it.
• Merge to main. Run smoke against staging before you promote.
• Production deploy. Run smoke against production the moment it's live. This is the core of post-deploy testing.
• On a schedule. Run smoke against production every few hours, independent of deploys.

That last one is the trigger most teams skip, and it's the one that catches the failures you didn't cause. Third-party scripts change, DNS drifts, a payment provider has an incident. None of those show up as a deploy, so nothing in your pipeline reruns. A scheduled smoke test running every hour tells you whether production is healthy right now. Your uptime monitor won't.

Manual vs. automated smoke testing

Manual smoke testing means a person clicks through the critical tests before calling a build ready. It works, and it doesn't scale. A manual pass on every pull request is a full-time job, and it's the first thing to get skipped when you're shipping under pressure.

Automated smoke testing runs the same checks on every build with no one watching. The suite runs, returns pass or fail, and either blocks the deploy or lets it through.

If you ship more than a couple of times a week, automation is the only version of this that survives contact with a real schedule.

Smoke testing tools

The common ways to automate smoke tests for a web app:

• Playwright. Open-source browser automation from Microsoft. You write tests in TypeScript or JavaScript and run them in CI. Powerful, and it's now the default for most teams. (We wrote a longer take on where it fits and where it hurts in our Playwright alternative breakdown.)
• Cypress. A popular alternative with a strong local debugging experience for teams writing end-to-end tests.
• Selenium. The original. Still common in larger organizations, with more setup overhead than the newer tools.
• AI-driven platforms. You describe a test the way you'd explain it to a teammate, and an agent runs it in a real browser. There's no test code to write or selectors to keep current; the agent reasons about the live page instead.

For a team without a dedicated tester, the deciding factor is almost always maintenance. Playwright and Cypress give you full control but need someone to keep the suite green every time the UI moves. The tests in an agent-driven tool only fail when the behavior actually breaks. There's no selector to update when you rename a button.

How long should a smoke test take?

Under five minutes. Ideally under three.

The reason is simple: smoke tests run on every build. A twenty-minute smoke suite becomes a bottleneck, and bottlenecks get bypassed. A suite covering five to seven critical tests usually runs in two to four minutes in parallel. If yours is creeping past five, look for:

• Tests waiting on network calls with hardcoded sleeps instead of proper waits.
• Tests running one after another that could run in parallel.
• Tests that belong in a regression suite, not a smoke suite. Smoke is critical path only.

A smoke test checklist for SaaS web apps

Use this as a starting template. Cover every test on it that applies to your product, and resist adding more. The value of this list is that it's short.

Authentication

• Sign up with email and password succeeds
• Email verification link arrives and works
• Log in with valid credentials succeeds
• Log in with invalid credentials fails gracefully, with an error, not a crash
• Password reset email arrives and the link works
• Magic link / SSO login works, if you have it

Onboarding

• A new user can finish the setup or onboarding test
• The empty state loads without errors
• The invite test works, if you have one

Core product

• The primary feature loads without errors
• A user can perform the main action (create, run, send, generate)
• Data created in that test shows up correctly afterward

Billing

• The upgrade test completes with a test card
• Paid features unlock after upgrade
• The billing portal loads

Navigation and session

• Main navigation loads with no 404s
• A user can reach settings and account pages
• Log out works, and the session persists across a reload

This isn't a full test suite. It's the minimum that tells you your product works for real people right now.

Common smoke testing mistakes

Testing too much. A smoke suite that takes twenty minutes isn't a smoke suite. It's a regression run wearing the wrong name. Keep the scope to the critical path.

Testing staging only. Staging differs from production in quiet ways. Run smoke against production too, on a schedule.

Skipping it when you're in a hurry. The rushed late-night deploy is exactly when a regression slips in, and exactly when you're most tempted to skip the check that would catch it.

Writing tests that fail on every UI change. A test that breaks when someone renames a button creates noise, and noisy checks get muted. A smoke test should fail when the test breaks, not when the markup moves.

Treating a red smoke test as optional. If a failing smoke test doesn't block the deploy or page someone, the signal is worthless. Make it a hard gate or don't bother.

How AI changes smoke testing

Traditional automated smoke tests are code. You write TypeScript, maintain selectors, and keep the suite passing as the UI changes. With dedicated engineers and time, that's a perfectly good approach.

For everyone else (founders, eng leads at a five-to-twenty-person SaaS, anyone shipping without a dedicated tester), maintaining that suite is the chore that keeps getting pushed to next sprint until something breaks in production.

Agent-driven smoke testing changes the math. Instead of writing test code, you write what to check, in words:

User logs in with their email and password.
Goes to the billing page.
Upgrades to the Pro plan with the test card.
Confirms Pro features are now enabled.

An agent runs that test in a real browser, records the session, and tells you whether it passed, with a screenshot, a step-by-step transcript, and the exact point of failure if it didn't. Rename the button and the test doesn't fall over, because there's no selector hard-coded to it. It only goes red when the test actually breaks.

That's what Smoketest does. You describe what a real user would do, we run it in a real browser, and you find out whether your critical paths work, after every deploy, on a schedule, or on demand.

FAQ