Smoketest
Changelog

Shipped recently

What you can do today that you couldn't last week.

35 releasesRSS
New
Permalink

Owner, admin, and viewer, per project

Access inside a project is now a role, not a yes or no.

  • Workspace owners assign owner, admin, or viewer per project, at invite time or later from team settings.
  • Project admins can see and manage Integrations and API keys for their own projects.
  • Workspace-wide credential changes stay owner-only.

This finishes what team access started in July.

Improved
Permalink

Steadier runs on messy pages

Two hardening passes on how the agent observes pages and recovers from dead ends.

  • Observation stays bounded on very large pages, so runs neither stall nor drown in markup.
  • Stale links recover conservatively instead of guessing at a replacement.
  • Evidence is gathered per requirement, so each Result cites the observations that actually support it.

No timeouts or budgets were raised. The agent just wastes less of them.

New
Permalink

Run digests: twice daily to monthly

Instead of an email per run, get a summary when you want one.

  • Choose twice-daily, daily, weekly, or monthly, delivered in your timezone.
  • Each digest compares against the previous period and summarizes status per test.

Configure it under notification settings. Per-run alerts still work if you want both.

New
Permalink

A 10,000-credit boost pack

Scheduled monitoring burns credits at a predictable rate. Release weeks do not. There is now a bigger top-up for the spikes.

  • Buy 10,000 credits at once from Usage & billing.
  • Boost credits stack on top of your plan's included balance.

One purchase covers the push, then your plan carries on as usual.

New
Permalink

A Board for your QA tickets

The dashboard now opens on Overview, and testing work lives on a Board that mirrors Jira or Linear.

  • The Board shows your Jira or Linear tickets in their real columns, with each card marked by where its testing stands.
  • Overview is the new home: iteration KPIs with filters that drill through to the runs behind each number.
  • The old Queue's filters and links carry over.

Jira or Linear still owns the workflow. The Board is Smoketest's view of it.

Fixed
Permalink

One email per failing test

A test that kept failing could email you for every single failure. Now it does not.

  • Repeated consecutive failures collapse into one email that counts them, instead of a fresh alert per run.
  • You still hear immediately on the first failure and on the recovery.

Your inbox tells you what changed, not what stayed broken.

Breaking
Permalink

Move a ticket to QA, get a tested Result back

Connect Jira or Linear, pick the status where testing begins, and moving a ticket into QA is enough to start the test. This is now the main way Smoketest works.

  • A ticket gets one durable Test Request. Each trip through QA becomes a new Attempt under it, so the current run and the history stay connected.
  • Each Attempt reads the ticket, its comments, and a linked pull request, generates a Test, runs it in a real browser, and posts the Result back on the ticket with an evidence link.
  • Passed moves the ticket to your success status. Failed returns it to development. No Result stays in QA and says exactly what is missing.
  • Human changes win: if someone moves the ticket out of QA mid-run, the Attempt cancels and Smoketest stands down.

Reusable Tests keep working on demand, on schedules, and from webhooks, API, CLI, and MCP. Start with the Quickstart.

Breaking
Permalink

Flows are called Tests now

The product now uses the words QA teams already use. This release completes the rename.

  • Flow is Test, Subflow is Shared Step, and Verdict is Result.
  • Manual is On demand, Testing Rule is Test Source, and one-time Flows are generated Tests.
  • Old API routes, CLI commands, MCP resources, and compatibility aliases are gone.

Your existing data is preserved under the new names. Only the words changed.

New
Permalink

An email when credits pause your schedules

Scheduled runs stop quietly when a workspace runs out of credits. Now the owner hears about it.

  • Workspace owners get an email the moment scheduled runs pause for lack of credits.
  • No more discovering a silent gap in your run history a week later.

Top up and the schedule picks back up on its own.

Improved
Permalink

The agent works in smaller steps it can check

We replaced the layer that drives the browser. The agent now acts through small, typed steps with bounded observations, and the verifier has to show its evidence.

  • Navigation at the start of a run is deterministic instead of best-effort.
  • Every Result comes from a structured verdict backed by observed evidence, not a general impression.
  • Browser errors fail fast with a named cause instead of timing out quietly.

Same tests, same runs. They just hold up better on difficult pages.

New
Permalink

Analytics for the whole project

A new analytics view answers the questions you were eyeballing from run lists: how reliable is this, how fast, how often does it recover.

  • Reliability, performance, recovery, and run-history metrics, per project.
  • Tables sort by name, pass rate, p95 duration, and last run.
  • Search finds a test by name or tag, and device and trigger now have their own labeled columns.

All of it comes from the runs you already have. Nothing to configure.

New
Permalink

Tests can use more than one tab

OAuth popups, payment windows, links that open a new tab: the agent now treats tabs as real tabs instead of pretending everything happens in one.

  • A new tab opened by a click becomes the active tab automatically.
  • The agent can list, switch, and close tabs mid-flow.
  • A login popup that closes itself hands control back to the window that opened it.

The run log labels each step with the tab it happened in, so the story stays readable.

Improved
Permalink

MCP clients can tag and schedule tests

The MCP surface caught up with the dashboard.

  • Create or update a test over MCP and assign its tags in the same call.
  • Configure hourly, daily, and weekly schedules with real IANA timezones.

Whatever your agent sets up shows in the dashboard like any other schedule.

Improved
Permalink

Pass rates on every test

Every test now shows how it has actually been doing, in the run views and on the dashboard.

  • Pass-rate stats appear per test, computed from your real run history.
  • Recordings play reliably now: files are indexed for instant start, so no more blank players.

The numbers come from every run, not a sample.

Improved
Permalink

Live runs narrate the finish

The end of a run used to sit on a stale "finishing" message. The live view now names each closing step as it happens.

  • Result checking, browser cleanup, artifact saving, and usage recording each report in turn.
  • Recordings now stream with byte-range support, so seeking in the player is instant.

No more wondering whether a run is finishing or stuck.

Improved
Permalink

Switch projects across workspaces

The project switcher now lists every project you can reach, from every workspace you belong to.

  • Projects are grouped by workspace, one click to jump between them.
  • Smoketest switches the workspace safely before opening the project, so you never land somewhere half signed in.

Consultants and agencies, this one is for you.

New
Permalink

A run can trigger other runs

Automations connect tests to each other. When a run passes, fails, errors, or recovers, Smoketest can start more runs in response.

  • Watch a whole project, one test, or a tag. When the event fires, trigger a test or an entire tag.
  • A failed checkout can immediately run the rest of your billing tag, so you know the blast radius before you open the dashboard.
  • Set them up from Integrations, next to your alert channels.

Every execution is tracked, so you can see what an automation actually did.

New
Permalink

Invite your team

Projects are team-scoped now. Invite people to your workspace and everyone works from the same tests, runs, and environments.

  • Invite someone and they see the projects their team owns, nothing else.
  • The API, CLI, and dashboard all enforce the same membership rules.
  • Accepting an invite drops you straight into the right project.

Roles within a project arrive later. For now, membership decides access.

Fixed
Permalink

Fixes: recordings, viewports, and an auth loop

A round of fixes for walls people actually hit.

  • Browser session runs record reliably again, and live sessions stay isolated from each other.
  • Runs on a phone preset now measure like the phone, not the desktop window.
  • HTTP basic auth no longer retries in a loop when the credentials are wrong.

Keep the reports coming.

New
Permalink

Support chat, straight to the founder

A chat widget now lives in the app and on the site. Write a message and it reaches the founder directly. Replies stream back in place.

  • Works signed in or anonymous, with an optional email if you want an answer later.
  • Attach a screenshot when words are not enough.
  • An unread badge and a quiet ping tell you a reply arrived, even if you closed the tab in between.

No ticket forms and no portal. It is a conversation.

New
Permalink

Log in once per environment

Each environment can now keep a signed-in browser session. Do the login once and every test in that environment starts already authenticated.

  • Save the session from the environment panel or the API.
  • Runs skip the login steps and land straight in your app.
  • This helps most behind SSO, magic links, or any auth that is slow or rate-limited to repeat.

Sessions belong to the environment, so staging and production stay separate.

New
Permalink

A guided start, with credits for finishing it

New accounts now open on a start guide that walks the first few milestones and pays you for them.

  • Create a test, run it, connect an alert. Each milestone checks itself off as you go.
  • Finishing milestones adds bonus credits to the workspace.

Skip it anytime if you would rather explore on your own.

New
Permalink

Trigger runs from Claude and Cursor

Smoketest now speaks MCP. Connect it to Claude, Cursor, or any MCP client and drive your tests from the tool you already work in.

  • Trigger runs and read results without leaving your editor.
  • Sign in with OAuth from the browser. There is no API key to copy around.

Anything that speaks MCP can connect the same way.

New
Permalink

Alerts for failures, and for recoveries

You now choose what you hear about, and Smoketest tells you when a failing test starts passing again.

  • Per project, alert on every run or only on failures and recoveries.
  • A recovery alert arrives the moment a previously failing test passes, so you know when the fix actually landed.

Works across email, Slack, and Discord.

New
Permalink

A check on every pull request

Install the Smoketest GitHub app, point it at a test, and every pull request gets checked before it merges.

  • The run shows up as a status check on the PR: green when it passed, red when it did not.
  • The check links to the recording, so you can see exactly what the agent saw.
  • No YAML to write and no action to keep updated.

Smoketest waits for your deploy to be ready before it starts the run.

Improved
Permalink

Every alert channel gets a send test button

Last week's Discord webhook is now a proper bot, and every alert channel can prove it works before you rely on it.

  • Install the Discord app once and pick the channel. No webhook URL to manage.
  • Slack, Discord, and email each have a send test button now.

Check that the alert lands today instead of discovering a typo during an incident.

New
Permalink

Trigger a run from any deploy

Tests can now start from your own pipeline. Wire a webhook or call the API, and a deploy kicks off the run.

  • Test settings show a Via API option with the exact endpoint to call. Copy, paste, done.
  • Deployment webhooks cover Vercel, Netlify, and anything that can send a POST.

This works alongside pull request checks for teams that ship outside GitHub.

New
Permalink

Set where a test starts

Every test can now declare its own start URL. Point it at a preview deploy, a staging branch, or a specific page.

  • Set it in the dashboard, the API, or the CLI.
  • The rest of the test stays the same. Only the starting point moves.

Useful when the same flow needs checking on more than one deployment.

New
Permalink

Schedules go hourly, daily, and weekly, in your timezone

The flat list of six preset times is gone. Pick an interval, a daily time, or a day of the week, and the scheduler uses your local timezone.

  • Run every 1, 3, 6, or 12 hours, once a day at a fixed time, or weekly on a specific day.
  • "9 AM daily" now means 9 AM where you are, not UTC. The dashboard picks up your timezone automatically.

This release also fixes a dispatch bug that could crash a run before it started.

New
Permalink

Tests can receive email

Some flows stop at a magic link or a one-time code. A test can now have its own inbox.

  • Opt a test in and it gets its own email address.
  • The agent sends the message, waits for it to arrive, reads the code or link, and carries on with the flow.

No forwarding rules and no manual steps on your side.

Improved
Permalink

Tag tests, describe them in Markdown

Past a handful of tests, one long list stops working. Tags and richer descriptions keep the library legible.

  • Add tags like "checkout" or "auth" to any test and filter the list by them.
  • A dedicated Tags page manages every tag in the project.
  • Descriptions render Markdown, with headings, lists, and links, so "passing" means the same thing to everyone.

Anyone on the team can open a test later and know exactly what it checks.

Improved
Permalink

Every step gets a screenshot

The run page now shows what the browser saw at each step, not only the final recording.

  • Each step captures a screenshot, so you can walk a failure moment by moment.
  • Runs use Playwright's own device presets, so viewports and user agents match real devices instead of approximations.
  • The run page now says exactly which evidence exists for a run, so you know when a trace is on hand.

And if something looks wrong, the new feedback dialog in the dashboard header reaches us directly.

New
Permalink

Run alerts land in Discord

If your team lives in Discord, Smoketest can now be the one to say something broke. Paste an incoming webhook URL into project settings and alerts arrive in your server.

  • Embeds are color-coded: green for a pass, red for a fail.
  • Every alert links straight to the run and its recording.

Slack and email alerts work exactly as before. Discord joins them.

Improved
Permalink

Watch a run live from the terminal

Trigger a run from the CLI and follow it while it happens, without switching to the dashboard.

  • smoketest runs watch streams each step as the agent works through your test. When it finishes, runs get prints the result with a full breakdown.
  • API requests now time out instead of hanging, and --insecure lets the CLI talk to a local server over plain HTTP.

Every command also accepts the short form of any resource ID, straight from the dashboard URL.

New
Permalink

Smoketest is open to everyone

Describe a user flow in a sentence or two and Smoketest runs it in a real browser. When it breaks, you find out before your users do.

  • Tests, environments with secrets, on-demand and scheduled runs, and full run history ship today.
  • Every run leaves evidence: a video recording cued to the failure and a step-by-step log of what the agent did.
  • Alerts land in Slack and email, and any CI or deploy webhook (Vercel, Netlify, anything that can POST) can start a run.

There is a public API and a CLI for teams that want to fit us into an existing workflow. Start free, no card required.

The QA column is no longer where the sprint goes to die.

Move one ticket. Watch it come back tested. Then decide.