Claude Code
CursorYour server
Real clients
It runs your server through real MCP clients, across several models.
MCP server testing
Can a model useyour MCP server?
Smoketest runs your server through real MCP clients, across several models, and watches what they do with your tools. You get a result for each model and the tool-call transcript on the ticket.
Book a callTalk to our founder. Test a real ticket the same day.
Acme › ACM-561In Progress
Add a create_invoice tool to the MCP server
A model asked to invoice a customer finds create_invoice, fills it in correctly, and the invoice appears in the account.
Activity
SKSara K assigned this to Smoketest
Smoketest Read the ticket, 2 comments, and PR #1374. The change adds one tool.
Ticket2 commentsPR #1374Tool list
Test plan1m 58s
1The server lists create_invoiceBothPassed
2Claude picks the right toolClaudePassed
3The invoice is in the accountClaudePassed
4GPT picks the right toolGPTFailed
Failed · Passed on Claude, failed on GPT · 1m 58s
GPT called create_draft, because both tool descriptions say they create an invoice. Step 6 in the transcript.
Not tested: expired tokens, because the ticket does not cover auth.
RecordingStepsReasoning
Smoketest moved this to In Progress and assigned Kevin
GPT called create_draft
A result for each model lands on the ticket you assigned.
It uses your server the way a model does.
A schema can pass every check and still leave a model stuck, so it tries the real thing.
Claude CodeCursorInvoice Globex
search_customerscreate_draftcreate_invoicesend_reminder
Picking the right tool
It checks that a model finds the right tool from your descriptions alone.
create_invoice
customer"Globex"Accepted
amount312.40Accepted
amount"lots"Refused
dueyesterdayAccepted
Schemas
It sends good and bad arguments, and checks each gets the right answer.
Valid token200
Expired token401
No token401
Auth
It connects with and without a token, and checks who gets refused.
customer_not_foundNo match for Globex. Try search_customers.
search_customersErrors a model can read
It checks that a failed call says enough for the model to recover.
create_invoice
INV-2041$312.40
GlobexIn the account
What happens next
It follows each call through, then checks what changed in your product.
Every ticket comes back Passed, Failed, or No Result.
If a person moves a ticket while it is testing, it stops and never moves it back.
Acme › ACM-561In Progress
Add a create_invoice tool to the MCP server
A model asked to invoice a customer finds create_invoice and the invoice appears.
Smoketest3 min ago
Failed 3 of 4 checks passed on PR #1374, 1m 58s
- The server lists create_invoice
- Claude picked create_invoice
- Invoice INV-2041 is in the account
- GPT picks the right toolExpected create_invoice, it called create_draft. Step 6 in the transcript.
- Not tested: expired tokensThe ticket does not cover auth.
RecordingStepsReasoning
Smoketest moved this issue to In Progress and assigned Kevin
One team member for everything you ship.
When a ticket touches more than your MCP server, the same ticket runs on the other surfaces too.
| Surface | Sandbox | What it checks | Proof you get | |
|---|---|---|---|---|
| Web apps | SandboxCloud browser at any width | What it checksFlows, forms, logins, downloads | Proof you getVideo, steps, transcript | → |
| Mobile apps | SandboxiOS simulator, Android emulator | What it checksScreens, gestures, deep links | Proof you getVideo, steps, device log | → |
| APIs | SandboxAPI runner against staging | What it checksStatus, contracts, auth, side effects | Proof you getRequest and response log | → |
| MCP servers | SandboxMCP client with a real model | What it checksSchemas, auth, real tool calls | Proof you getTool-call transcript | → |
| AI agents | SandboxYour agent's real interface | What it checksReplies, tool calls, side effects | Proof you getConversation transcript | → |
Building something else? One team asked us to test their voice agent, so we are adding it. Propose your surface
Testing an MCP server comes down to five checks.
Whether a person or an agent does it, a good pass covers the same ground. Smoketest works out which of these a ticket needs, then runs them.
- 1
Discovery
List the tools, and check that each name and description is clear.
- 2
Tool choice
Give a model a task, and see which tool it reaches for.
- 3
Arguments
Send good, bad, and missing values, and read what comes back.
- 4
Errors
Break a call on purpose, and see whether the model can recover.
- 5
What happens next
Check what changed in your product after the call finished.
Questions about testing MCP servers
It runs your server through real MCP clients, across several models, then checks which tools they picked and what changed in your product.
Keep it. Inspector is how you call a tool by hand while you build it. Smoketest shows whether a model finds the tool on its own, and what happens next.
Several, so you can see where one model gets it right and another does not. Tell us on the call which ones your users run.
Yes. Save a token or a test account, and it checks who gets in and who gets refused.
A result for each model on the ticket, with the tool-call transcript and what it did not test.
It starts at $399 a month for 1,000 runs. Each run is up to 10 minutes, and a longer test counts as two runs or more, one for every 10 minutes. If your team needs more, we design a Scale plan with you, so you stop buying extra runs.
Every result links to the recording, the steps, and the reasoning, and says what it did not test, so you can check its work the way you would check a teammate's.
Your next MCP ticket comes back tested.
Book a call and we will run one ticket through your server with you. You get the result and the transcript the same day.
Ready for QA
Done
ACM-561
Add a create_invoice tool to the MCP server
Passed, recording