apo
Demo · read-only
Sign in
Agent Testing
Tasks
Runs
Schedules
Observability
Traces
Toggle Sidebar
demo
Runs
Runs
demo-bch
real-agent/engineering/api-testing
demo-run_f
Failed
real-agent/engineering/api-testing
·
openai/gpt-4o-mini
·
cli
·
8w ago
Details
▸
Task
Run
Trace home
Delete
50%
pass rate
3
passed
·
3
failed
·
6
checks
21.0s
duration
$0.003000
11.5k tok
—
reasoning
Checks
6
Conversation History
Deliverables
Trace home
3
/6 passed
3 failed
Click to expand
✓
read-responses-and-schema
✗
caught-planted-violations
expected expected finding: empty items array (order 102) findings include empty items array (order 102); expected expected finding: dangling user_id (99) findings include dangling user_id (99)
✓
caught-majority-of-violations
✗
analysis-is-comprehensive
The findings in Value 2 do not catch all 8 planted schema violations. Specifically, it fails to identify the dangling foreign key for order 102, the invalid enum for order 103, and the malformed email for user 2. Therefore, it does not meet the requirement to pass.
✗
findings-are-field-specific
Value 1 identifies specific records and fields that violate constraints, while Value 2 does not provide specific record and field details for the validation findings, making it generic.
✓
files-present