apo
Demo · read-only
Sign in
Agent Testing
Tasks
Runs
Schedules
Observability
Traces
Toggle Sidebar
demo
Runs
Runs
demo-bch
real-agent/engineering/api-testing
demo-run_c
Failed
real-agent/engineering/api-testing
·
anthropic/claude-sonnet-4.5
·
cli
·
8w ago
Details
▸
Task
Run
Trace home
Delete
50%
pass rate
3
passed
·
3
failed
·
6
checks
48.1s
duration
$0.1281
25.4k tok
—
reasoning
Checks
6
Conversation History
Deliverables
Trace home
3
/6 passed
3 failed
Click to expand
✓
read-responses-and-schema
✗
caught-planted-violations
expected expected finding: empty user name (id=4) findings include empty user name (id=4); expected expected finding: malformed email (bob@example) findings include malformed email (bob@example); expected expected finding: invalid role (superadmin) findings include invalid role (superadmin); expected expected finding: negative order total (-15) findings include negative order total (-15); expected expected finding: empty items array (order 102) findings include empty items array (order 102)
✗
caught-majority-of-violations
assertion "caught at least 6 of 8 planted violations" failed
✗
analysis-is-comprehensive
Value 1 catches only 3 of the 8 required schema violations (user id=-3, user id=5 invalid date, order 102 dangling FK). It misses 5 violations: user id=4 empty name, malformed email 'bob@example', role='superadmin' invalid enum, order 102 total=-15 negative, order 102 items=[] minItems violation, and order 103 status='unknown' invalid enum. Value 2 is incomplete. The instruction requires ALL 8 violations to be caught without fabrication.
✓
findings-are-field-specific
✓
files-present