Build APIs that stay correct after they ship.
PreMan helps teams discover and test API surfaces, simulate changes against trusted evidence, monitor production, repair regressions, and verify that AI-agent actions reached the correct state in the system of record.
npm exec -y premanmcp@latest -- onboardTest
Discover endpoints, save requests, assert responses, and run safely from cloud or Desktop.
Read the guideGuard
Diff an exact Git commit, test affected routes, and explain release evidence before deployment.
Read the guidePulse
Monitor endpoint health, production logs, dependencies, alerts, and repair work.
Read the guideVerified Actions
Read the system of record after an agent acts and store a check-by-check verdict.
Read the guideHosted MCP
Convert APIs into remote MCP tools with access controls, credentials, audit, and metrics.
Read the guideManaged MCP profiles
Import several existing MCPs and expose one policy-controlled profile. Not a production API yet.
Read the guideConnect PreMan to your coding agent
The MCP package is the fastest way to use PreMan from Cursor, Claude Code, or Codex. The installer handles authentication and writes the correct agent configuration.
- 1Connect the agent
Run the installer and choose your coding agent.
- 2Restart the agent
Configuration changes are loaded when the host restarts.
- 3Confirm the workspace
Ask the agent to run
preman_status. - 4Start with your API
Discover routes, test them, or run a Guard readiness check.
$ npm exec -y premanmcp@latest -- onboardRun preman_status, then discover the API surface in this repository and show me which endpoints are ready to test.The installer stores your credentials outside the project configuration. For CI or a non-interactive environment, create a pm_live_… key in PreMan settings.
Core concepts
These objects connect the developer workflow to production evidence.
- Workspace
- The shared place for requests, secrets, conversations, integrations, and team settings.
- Project
- The access boundary for Pulse runs and production logs. Link a workspace to a project explicitly.
- Endpoint
- A normalized method, route, schema, upstream, and source record discovered from code or imported definitions.
- Guard simulation
- An exact-commit contract comparison plus the runtime evidence available for that candidate.
- Action contract
- A versionable definition of the state that must exist after an agent claims an action succeeded.
- Verification run
- A verdict, check results, and bounded evidence for one reported agent action.
Discover and import API surfaces
PreMan normalizes routes from the tools teams already use, then keeps source information attached to every endpoint.
Discover routes and schemas from supported application code.
Import a machine-readable specification or public reference URL.
Connect with an API key or upload collections and environments.
Normalize request definitions into the same endpoint registry.
Keep repository-owned endpoints aligned with exact commits and the published baseline.
Let the agent inspect the open repository and share a reviewable endpoint session.
discover_endpoints_from_codebase → verify_endpoints_live → mcp_preview → mcp_deployRun saved requests with explicit assertions
A saved request contains the method, URL, body, expected status, headers, provenance, and risk classification used for repeatable testing.
Cloud runner
Runs from PreMan infrastructure against publicly reachable endpoints. It cannot reach localhost, private network addresses, or VPN-only services.
Desktop runner
Runs from your machine, so it can reach localhost and private services. Results can still be recorded in shared history and Pulse.
Store credentials as workspace secrets and reference them with tokens such as {{API_TOKEN}}. Values are substituted at execution time instead of being copied into the shared request definition.
| Assertion | Use it for |
|---|---|
status / status_in | Expected HTTP outcomes |
max_latency_ms | A response-time budget |
json_path | Dotted-path value equality |
required_fields | Required response shape |
preman assert | Read-only GET/HEAD state checks in local development or CI |
Generate bounded tests, monitor continuously, and alert
Generated cases and scheduled probes turn one saved request into repeatable release and production coverage. Some monitoring and alert features are plan-gated.
- Generate persisted happy-path, missing-field, invalid-value, boundary, pagination, and injection cases.
- Choose
read_only,allow_writes, orallow_destructiveas the standing authorization. - Schedule endpoint probes from 30 to 3,600 seconds and inspect result history.
- Trigger rules for consecutive failures, error rate, or p95 latency.
- Send alerts through email or Slack, including bounded evidence and recovery notices.
$ npx preman-sdk monitor --endpoint-id <id> --interval-seconds 60 --expected-status 200Simulate the exact Git commit before release
Connecting the GitHub App enables durable push simulations. PreMan checks out the pushed SHA, compares its API contract with the trusted baseline, and runs only eligible evidence against the candidate environment.
- 01Signed push
GitHub sends an authenticated delivery for the selected repository and immutable repository id.
- 02Exact source scan
PreMan checks out the full SHA and produces a bounded field-level contract diff.
- 03Runtime scenarios
Read-only probes or approved suites exercise the candidate environment.
- 04Build attestation
Every runtime response must identify the exact commit before it becomes trusted evidence.
- 05Verdict
Guard separates impact, green evidence, missing coverage, and repositories with no API surface.
The first successful default-branch simulation establishes the baseline. Later default-branch pushes advance it only when GitHub still reports that commit as the branch head. Other branches are previews and do not replace the published baseline.
Read evidence modes and verdicts honestly
Static contract evidence and runtime behavior are evaluated separately. A matching schema cannot override a failed or unverified runtime result.
Contract + synthetic
Uses the exact-SHA contract diff plus configured, approved, read-only probes and generated scenarios.
Observed behavior
Builds privacy-safe, aggregate scenarios from an explicitly selected healthy CloudWatch source. It never replays raw customer requests.
| Result | Meaning |
|---|---|
| Green | No breaking contract change; intended runtime coverage ran, passed, and attested the exact commit. |
| Impact detected | A breaking contract change or failed runtime scenario was found. |
| Inconclusive | The simulation finished without enough trusted evidence to call it green. |
| No API surface | The signed commit scan completed, but the repository did not own an API candidate to exercise. |
Candidate responses can attest the commit through headers including X-PreMan-Build-Commit, X-Git-Commit, or X-Commit-Sha, or through standard JSON health fields such as commit_sha. Full 40-character SHAs are required.
When responses attest a different build, Impact Simulation reports the active commit serving that traffic alongside the selected candidate. If traffic spans a rolling or split deployment, it reports each observed commit and its response count instead of choosing one arbitrarily.
Hand a finding to a coding agent
A completed simulation or fired incident can become a repository-bound fix task with the evidence needed to reproduce and validate the failure.
Connect production behavior to endpoint health
Pulse is project-scoped so test runs, production logs, and the routes they describe share one access and evidence boundary.
Availability, p95 latency, check count, latest result, and recent history.
Authenticated, redacted CloudWatch streaming with health and freshness signals.
Explicit source_depends_on_target edges and evidence-backed blast radius.
Alerts, correlated evidence, investigation context, and repair handoff.
- Link the workspace to a project. Workspace membership and project membership are separate; production log access follows the project.
- Connect AWS. Deploy the generated read-only cross-account role, select a CloudWatch log group, and test reachability.
- Inspect the live signal. Use Graph, List, and History without losing the selected route.
- Reproduce or investigate. Open the saved request, correlated logs, or an agent task from the same endpoint context.
Verify the business outcome after an agent acts
The agent saying “done” is the claim. PreMan independently reads the system of record, compares observed state with an action contract, and stores the result.
Use Verified Actions when a valid tool response is not enough—for example, when a refund can have the wrong amount, an appointment can be missing from the scheduler, or a CRM update can land on the wrong record.
Define an action contract
A contract names the action, system, consistency window, and checks that must hold. Adapter credentials are write-only, encrypted at rest, and never returned by contract APIs.
{
"key": "shopify_refund_v1",
"name": "Verify Shopify refund",
"system": "shopify",
"consistency_window_seconds": 120,
"adapter": {
"system": "shopify",
"config": { "shop_host": "acme.myshopify.com" },
"secret": "<read-only Shopify token>"
},
"checks": [
{
"type": "linked_state",
"resource": "order",
"id_from": "claimed.order_id",
"field": "financial_status",
"expect": "refunded"
},
{
"type": "amount_equals",
"field": "refund.amount",
"source": "request.amount"
}
]
}| Check | Purpose |
|---|---|
object_exists | Confirm that a claimed object is present. |
field_equals | Compare a field with a literal or event source value. |
amount_equals | Compare numeric values with an optional tolerance. |
unique_match | Require exactly one matching record. |
linked_state | Read the attached system-of-record adapter and grade observed state. |
Report an action and inspect its run
Action events are asynchronous claims. Use an idempotency key so a retry cannot create a second logical action.
{
"action_type": "issue_refund",
"contract_key": "shopify_refund_v1",
"source": "customer-agent",
"external_action_id": "refund_1049",
"customer_ref": "store_842",
"request_payload": {
"order_id": "1049",
"amount": 82
},
"claimed_result": {
"order_id": "1049",
"refund": { "amount": 82 },
"status": "refunded"
},
"idempotency_key": "refund_1049"
}The create call returns both the action event and its verification run. A run may remain pending while the consistency window is open. Read it with GET /verification/runs/{run_id}, or use POST /verification/runs/{run_id}/grade when an operator needs an immediate terminal attempt.
| Verdict | Meaning |
|---|---|
passed | Every supported check passed. |
failed | At least one check contradicted the expected outcome. |
inconclusive | The evidence could not prove or disprove the outcome. |
pending | The run is waiting for its next grading attempt. |
blocked_missing_credentials | The read-only adapter could not authenticate. |
blocked_rate_limited | The upstream system prevented a reliable read. |
Read supported systems of record
Adapters return a bounded, secret-free observation: found, missing, or read failed. The raw credential and complete upstream body are not exposed as evidence.
Orders, products, and customers. Configure a *.myshopify.com host and read-scoped token.
Deals, contacts, companies, and tickets. Configure a private-app token and optional property list.
Runs the production grading path against deterministic records without a network or credential.
Shopify and HubSpot reads use bounded timeouts and response limits. Authentication failure, missing objects, unreachable systems, and upstream errors remain distinct evidence states.
Convert an API into a remote MCP server
Select discovered endpoints, review their generated tool schemas, and deploy one Streamable HTTP MCP that agents can install.
Scan code, import a spec, or use a public reference URL.
Review tool names, descriptions, schemas, and endpoint mappings.
Choose the upstream base URL, auth mapping, and access mode.
Share the remote URL or mint scoped consumer tokens.
{
"mcpServers": {
"payments-api": {
"url": "https://api.preman.live/h/<hosted-mcp-id>/mcp",
"headers": {
"Authorization": "Bearer pm_hmcp_<consumer-token>"
}
}
}
}Hosted runtimes support initialize, tools/list, tools/call, ping, and JSON-RPC batches. Public mode needs only the URL; token mode requires a scoped consumer token.
Operate credentials, consumers, and tool quality
The hosted control plane separates upstream credentials from the credentials agents use to call the MCP.
pm_hmcp_… tokens shown once, stored hashed, revocable, rate-limited, and scoped to selected tools.Remote Streamable HTTP MCP import is currently Beta: PreMan can wrap a compatible remote server in the hosted gateway. The broader experience for importing multiple stdio, SSE, and HTTP servers into one governed profile is still Preview and has no stable public API.
Read contract-gated hosted MCP verificationConnect the systems already in your workflow
Each integration has a narrow job and an explicit access boundary.
Choose the package for the job
The SDK and MCP package serve different integration paths.
$ npm exec -y premanmcp@latest -- onboardUse for Cursor, Claude Code, Codex, endpoint discovery, Guard readiness, and API-to-MCP workflows.
Open npm package$ npm install preman-sdkUse for endpoint registration, scheduled monitoring, state assertions, incidents, and repair task APIs.
Open npm package$ preman verify --pre-pushPre-push checks scan changed API sources, include shared-model neighbors, and can rank tests with observed route volume. Blocking is opt-in; crashes and timeouts do not block a push by default.
HTTP API reference
The live OpenAPI document is the source of truth for request and response schemas. Use this guide for workflow and safety semantics.
Interactive FastAPI reference for the deployed backend.
| Route group | What it controls |
|---|---|
/verification | Action contracts, action events, runs, grading, and evidence. |
/integrations/github | Repositories, automation, simulation policy, and push results. |
/workbench | Saved requests, secrets, conversations, auto-tests, and workspace settings. |
/monitoring | Probes, alerts, incidents, fix tasks, and remediation handoff. |
/projects/{id}/logs | Redacted production logs, health, queries, and authenticated streaming. |
/hosted-mcps | Deployments, credentials, consumers, invocations, metrics, and discovery. |
/mcp/call-tool | The authenticated bridge used by premanmcp. |
Security and evidence boundaries
PreMan treats execution authority, system-of-record access, and customer-facing MCP access as separate credentials.
pm_live_… keys authenticate SDK, CLI, and MCP control-plane calls. Raw keys are returned once and stored hashed.
pm_hmcp_… tokens authorize downstream agents to call one hosted MCP and can be scoped or revoked independently.
Verification and log connections should use least-privilege read access. Stored secrets are encrypted and not returned in plaintext.
Production rows are redacted before persistence; reports store bounded metadata and observations instead of unrestricted raw payloads.
- Manual destructive requests require one-run approval.
- Scheduled execution defaults to read-only authorization.
- Runtime attestation fails closed on missing, short, or mismatched commit ids.
- Repair automation opens review work; it does not merge or deploy.
Troubleshooting
Start with the boundary that produced the result: authentication, network reachability, evidence coverage, or repository state.
PreMan tools show a missing Authorization token
Run npm exec -y premanmcp@latest -- onboard, restart the coding agent, and run preman_status. In CI, set a complete pm_live_… key rather than a hex-only value or hosted-MCP consumer token.
A cloud request cannot reach localhost or a private API
Use PreMan Desktop so the request originates from your machine. Confirm that the selected local runner has access to the VPN or private host.
A Guard simulation is inconclusive
Check runtime coverage, exact-build attestation, candidate environment URL, observed-source health, and fallback policy. Inconclusive is intentionally different from green.
A GitHub push does not start a simulation
Confirm the GitHub App still has repository access, simulation-on-push is enabled, and PreMan has received a signed delivery. Opening GitHub configuration does not update PreMan until repositories are refreshed.
A verification run is blocked
For blocked_missing_credentials, rotate or replace the adapter credential. For rate limits or eventual consistency, wait for the next attempt or deliberately use the grade-now endpoint for a terminal operator check.