Create an eval case

Adds one case to the suite. Both title and a non-empty steps array are required — steps is optional on the shared case shape so PATCH can be partial, but a case persisted without executable steps could never run. When models is omitted the suite’s configured model is used.

POST /projects/{projectId}/eval-suites/{suiteId}/cases

Create an eval case

cURL

curl --request POST \
  --url https://app.mcpjam.com/api/v1/projects/{projectId}/eval-suites/{suiteId}/cases \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "title": "lists files",
  "steps": [
    {
      "id": "s1",
      "kind": "prompt",
      "prompt": "List the files in the project root."
    },
    {
      "id": "s2",
      "kind": "assert",
      "assertion": {
        "type": "toolCalledWith",
        "toolName": "list_files",
        "args": {
          "args": {}
        }
      }
    }
  ]
}
'

201

Example

{ "id": "<string>", "title": "<string>", "steps": [ { "id": "<string>", "prompt": "<string>", "serverName": "<string>", "toolName": "<string>", "arguments": {}, "action": {}, "assertion": {} } ], "iterations": 5, "isNegative": true, "models": [ { "model": "<string>", "provider": "<string>" } ], "expectedOutput": "<string>", "scenario": "<string>", "matchOptions": {}, "checks": { "list": [ {} ] }, "createdAt": 123, "updatedAt": 123 }

Authorizations

Authorization · string · header · required

MCPJam API key (sk_…). Create one at Settings → API keys. Guest sessions cannot use the API, and API keys cannot manage other API keys.

Path Parameters

projectId · string · required

ID of the hosted project that contains the server.

suiteId · string · required

Eval suite ID, as returned by POST /eval-runs.

Body

application/json

Create one case. title and a non-empty steps array are both required — a case persisted without executable steps could never run.

title · string · required

Minimum string length: 1

steps · object[] · required

REPLACES the case's test definition wholesale when provided.

Minimum array length: 1

Show child attributes

expectedOutput · string

iterations · integer

Required range: 1 <= x <= 10

isNegative · boolean

scenario · string

models · object[]

Show child attributes

matchOptions · object | null

null clears the case override and inherits the suite's.

checks · object | null

null clears the case override.

Show child attributes

Response

201

application/json

The created case.

A persisted eval case, in the public steps-first shape. Note this is NOT EvalTestCase, which is the INLINE authoring shape accepted by suite creation.

id · string · required

title · string · required

steps · object[] · required

Ordered test steps. A prompt step is a model turn; a single model-free toolCall step is a render-check; assert steps hold the expectations.

Minimum array length: 1

Show child attributes

iterations · integer · required

Required range: 1 <= x <= 10

isNegative · boolean · required

When true, the case passes if NO tools are called.

models · object[] · required

Show child attributes

expectedOutput · string

scenario · string

matchOptions · object | null

checks · object | null

Show child attributes

createdAt · number | null

updatedAt · number | null