← Back to search

io.github.iris-eval/mcp-server

iris-eval Scanned 5h ago

The agent eval standard for MCP. Score every agent output for quality, safety, and cost.

MCP Registry ↗
D
40 / 100

Versions

0.1.3latest
first seen Jun 5, 2026
0.4.2
first seen May 19, 2026
PermissionsTool SafetyAuthAnnotationsCode QualityStabilitySpecVuln HistoryAuthorTransparencyCommunity

Tools 14

evaluate_output
annotations: verified low

readOnlyHint false destructiveHint false
evaluate_with_llm_judge
annotations: verified low

readOnlyHint false openWorldHint true idempotentHint false destructiveHint false
delete_rule
annotations: verified low

readOnlyHint false openWorldHint false idempotentHint false destructiveHint true
list_rules
annotations: verified low

readOnlyHint true openWorldHint true idempotentHint true destructiveHint false
get_traces
annotations: verified low

readOnlyHint true openWorldHint true idempotentHint true destructiveHint false
delete_trace
annotations: verified low

readOnlyHint false openWorldHint false idempotentHint false destructiveHint true
log_trace
annotations: verified low

readOnlyHint false idempotentHint false destructiveHint false
deploy_rule
annotations: verified low

readOnlyHint false openWorldHint false idempotentHint false destructiveHint false
compare_traces
annotations: verified low

readOnlyHint true openWorldHint false idempotentHint true destructiveHint false
verify_citations
annotations: verified low

readOnlyHint false openWorldHint true idempotentHint false destructiveHint false
compare_runs
annotations: verified low

readOnlyHint true openWorldHint false idempotentHint true destructiveHint false
evaluate_runs
annotations: verified low

readOnlyHint false idempotentHint true destructiveHint false
ping
annotations: none low

Answers pong.

get_weather
annotations: none low

Today\'s weather for a city

Permissions 5

network medium
Server uses network capabilities via: fetch(), http, httpx, net, requests, socket, urllib
filesystem low
Server uses filesystem capabilities via: fs, fs sync ops, fs/promises, open(), os, path, pathlib, shutil, tempfile
shell high
Server uses shell capabilities via: child_process, execSync(), spawn(), spawnSync(), subprocess
database medium
Server uses database capabilities via: better-sqlite3
env_vars low
Server uses env_vars capabilities via: os.environ, process.env

Scan Findings 56

info
Tool 'evaluate_output' annotations are consistent annotation_checker · 80%
info
Tool 'evaluate_with_llm_judge' annotations are consistent annotation_checker · 80%
info
Tool 'delete_rule' annotations are consistent annotation_checker · 80%
info
Tool 'list_rules' annotations are consistent annotation_checker · 80%
info
Tool 'get_traces' annotations are consistent annotation_checker · 80%
info
Tool 'delete_trace' annotations are consistent annotation_checker · 80%
info
Tool 'log_trace' annotations are consistent annotation_checker · 80%
info
Tool 'deploy_rule' annotations are consistent annotation_checker · 80%
info
Tool 'compare_traces' annotations are consistent annotation_checker · 80%
info
Tool 'verify_citations' annotations are consistent annotation_checker · 80%
info
Tool 'compare_runs' annotations are consistent annotation_checker · 80%
info
Tool 'evaluate_runs' annotations are consistent annotation_checker · 80%
low
Tool 'ping' has no annotations annotation_checker · 100%
low
Tool 'get_weather' has no annotations annotation_checker · 100%
medium
OAuth implementation without PKCE auth_checker · 75%
info
Sandbox failed to start for behavioral verification behavioral_verifier · 100%
medium
Suspicious package name: react-dom dependency_analyzer · 60%
medium
Suspicious package name: react-router dependency_analyzer · 60%
medium
Suspicious package name: react-markdown dependency_analyzer · 60%
medium
Vulnerable dependency: @modelcontextprotocol/sdk@1.30.0 (GHSA-6qxp-vccf-f47h) dependency_analyzer · 95%
medium
Vulnerable dependency: ajv@8.17.1 (GHSA-2g4f-4pwh-qvx6) dependency_analyzer · 95%
medium
Vulnerable dependency: httpx@0.27,<1 (GHSA-h8pj-cxx2-jfg2) dependency_analyzer · 95%
medium
Vulnerable dependency: httpx@0.27,<1 (PYSEC-2022-183) dependency_analyzer · 95%
medium
Buffer.from base64 in iris-eval-mcp-server-ade0c5e/tests/listings-hygiene.test.ts:75 entropy_analyzer · 75%
medium
Buffer.from base64 in iris-eval-mcp-server-ade0c5e/src/otel/ingest.ts:143 entropy_analyzer · 75%
medium
Buffer.from base64 in iris-eval-mcp-server-ade0c5e/src/notify/config.ts:76 entropy_analyzer · 75%
medium
Buffer.from base64 in iris-eval-mcp-server-ade0c5e/src/notify/webhook.ts:87 entropy_analyzer · 75%
medium
Hex string literal (>50 chars) in iris-eval-mcp-server-ade0c5e/tests/unit/security/keys.test.ts:132 entropy_analyzer · 70%
info
package.json metadata manifest_parser · 100%
info
pyproject.toml metadata manifest_parser · 100%
info
Tool: evaluate_output manifest_parser · 85%
info
Tool: evaluate_with_llm_judge manifest_parser · 85%
info
Tool: delete_rule manifest_parser · 85%
info
Tool: list_rules manifest_parser · 85%
info
Tool: get_traces manifest_parser · 85%
info
Tool: delete_trace manifest_parser · 85%
info
Tool: log_trace manifest_parser · 85%
info
Tool: deploy_rule manifest_parser · 85%
info
Tool: compare_traces manifest_parser · 85%
info
Tool: verify_citations manifest_parser · 85%
info
Tool: compare_runs manifest_parser · 85%
info
Tool: evaluate_runs manifest_parser · 85%
info
Tool: ping manifest_parser · 85%
info
Tool: get_weather manifest_parser · 75%
info
Transport: stdio manifest_parser · 90%
info
Required env vars (91) manifest_parser · 80%
info
Sandbox failed to start for output poisoning scan output_poisoning · 100%
medium
Permission: network access detected permission_analyzer · 90%
low
Permission: filesystem access detected permission_analyzer · 90%
high
Permission: shell access detected permission_analyzer · 95%
medium
Permission: database access detected permission_analyzer · 90%
low
Permission: env_vars access detected permission_analyzer · 90%
info
SBOM generated: 1201 components sbom_generator · 100%
high
Generic API Key Assignment found in iris-eval-mcp-server-ade0c5e/proof/corpus/pii.json secret_scanner · 75%
critical
Database URL with Password found in iris-eval-mcp-server-ade0c5e/proof/corpus/pii.json secret_scanner · 85%
medium
No build provenance detected (SLSA L0) slsa_assessor · 90%