Smoketest vs Checkly

Smoketest vs Checkly: monitoring as code, or from a sentence.

Checkly is monitoring-as-code: engineers write checks in Playwright and TypeScript and deploy them from git. Smoketest monitors the same kind of real-browser test from a sentence, so anyone on the team can add one.

No Playwright, no TypeScript, no git. Your first run in about two minutes.

smoketest.sh/dashboard
Board
Tickets currently moving through QA
Development1
Jira · APP-917
Invite member from workspace settings
App QANot tested
Testing2
Linear · WEB-284
Apply annual discount at checkout
Checkout QARunning
Linear · AUTH-126
Recover account with magic link
Authentication QANeeds attention
Passed1
Jira · BILL-442
Show updated invoice after seat change
Billing QAPassed
The problem

Both do monitoring. They differ on who can author a check.

Checkly and Smoketest both run real-browser tests against production on a schedule and on every deploy. The difference is how a check gets written and who can write one.

How a check is authored

Checkly checks are Playwright and TypeScript, committed to git. Smoketest tests are a sentence or two.

Even with AI

Checkly's AI can generate a check, but it produces code an engineer reviews and commits. Smoketest needs no code at any step.

Who can add one

Checkly is built for engineers and coding agents. With Smoketest, a PM, founder, or support lead can add a test.

What a failure shows

Checkly returns logs, traces, and video. Smoketest hands you a recording, a transcript, and a pass or fail result with reasoning.

What a test looks like

No selectors. No test file.

you write
Start on the login page.
Log in with the test account.
Open the orders page.
Confirm the latest order is visible.
smoketest does the rest
Opens a real browser and runs each step
Pulls test credentials from your Persona secrets
Records the full session
Returns a pass or fail result with its reasoning
Saves the transcript, screenshots, and a Playwright trace
Smoketest vs Checkly

Smoketest vs Checkly, at a glance.

Checkly
Smoketest
Author a check
Playwright in TypeScript, committed to git
A sentence or two
With AI assistance
Generates code an engineer reviews and commits
No code at any step
Who can author
Engineers and coding agents
Anyone on the team
Also covers
API and multistep checks, IaC, alerting
Real-browser user tests with a result
A run gives you
Logs, trace, and video
Recording, transcript, and a pass or fail result
Pricing
Free tier, then $24 to $64/month plans plus usage overages
$39/month flat, listed publicly
Who it's for

Checkly or Smoketest?

Smoketest is the better fit when you

Want non-engineers to add and read the checks
Would rather describe a test than write or review Playwright
Want a watchable recording and a result, not just logs and a trace
Don't want to own a monitoring-as-code repo

Checkly is the better fit when you

Want monitoring versioned as code in git, with Terraform or Pulumi
Need API and multistep checks alongside browser checks
Have engineers (or coding agents) who own reliability as code
Want SRE-grade alerting and dashboards across many endpoints and locations
Use cases

Cover the tests that matter.

Login

→User logs in with email and password and lands on the dashboard.

Checkout

→User adds an item, checks out with the test card, and sees the order confirmation.

Onboarding

→New user signs up, completes the setup wizard, and reaches an empty project.

Billing

→User upgrades to Pro and the workspace shows Pro features enabled.

Magic link

→User requests a magic link, opens it, and lands authenticated.

PR previews

→Your critical tests, pointed at the pull request's preview URL.
When it fails

The whole story, not just a red X.

Every failed run hands you a recording, a transcript, the result, and the downloadable Playwright trace.

signup-test.webm
00:00↑ 00:06 · sign-up00:07
step log
Opened /pricing
Clicked “Start free”
Filled email · password
Submitted sign-up form
Expected dashboard · got 500 error
Session recording
Screenshot at failure
Step-by-step transcript
Downloadable Playwright trace
How it works

Jira or Linear starts the Test.

Keep the workflow where QA already works. Smoketest adds execution, evidence, and routing without becoming another board.

workspace · connections
Lhealthy
Linear
Acme workspace
Jhealthy
Jira
acme.atlassian.net
01

Connect Jira or Linear

A workspace owner authorizes Jira or Linear once. Smoketest keeps Connection identity, permissions, token health, and webhook health visible.

new test source
Listen to
Checkout board · QA
Passed
Ready to release
Failed
Development
Run at
staging.acme.com · Desktop
Permissions and direct transitions validated
02

Define the QA handoff

Choose the team or board, exact QA status, direct Passed and Development statuses, then set the URL, Environment, device, and access settings.

test request · attempt 1
WEB-284 enters QA
Ticket + linked PR frozen
Generated Test created
Browser Run in progress
03

Let the ticket start testing

When a ticket enters QA, Smoketest freezes current context, enriches it from an eligible ticket-linked pull request, generates one Test, and runs it automatically.

result · synchronized
PASSEDWEB-2841m 42s
Attempt comment updated with evidence
Authenticated recording and transcript linked
QAReady to release
04

Return the Result safely

Smoketest posts evidence to one evolving Attempt comment. Passed moves forward, Failed returns to Development, and No Result stays in QA. Newer human changes always win.

FAQ

Teams comparing us to Checkly ask these.

Yes, and it's good. Checkly's AI can generate a check and help investigate failures. But the check it produces is Playwright and TypeScript that an engineer reviews and commits. Smoketest needs no code at any step: you describe the test and the agent runs it.

No. A Smoketest test is a sentence or two, so a PM, founder, or support lead can add one. Checkly checks are written in Playwright and TypeScript and deployed from git.

For real-browser user tests, yes. Run them against production on a schedule or on every deploy, and get alerted when one breaks. Checkly also covers API and multistep checks and infrastructure-as-code, which Smoketest does not.

Yes. Every run saves a downloadable Playwright trace alongside the recording, transcript, and result.

About two minutes: paste a URL, write the test, run it once, then schedule it. No repo, pipeline, or deploy step to wire up.

Add a check without writing one.

Describe a test in a sentence, run it against your live app, and get a recording and a result. No Playwright, no TypeScript, no git.

Read the docs