io.github.KevinRabun/judges

List files and directories in the workspace. Useful for exploring project structure before evaluating code.

Read the contents of a file in the workspace. Returns the file text, or an error if the file is too large or missing.

Evaluate code with progressive judge-by-judge reporting. Returns intermediate counts as each judge completes, useful for large files where full tribunal takes time.

Explain a Judges Panel finding in plain language. Provides OWASP/CWE references, risk context, and remediation guidance based on the rule ID and finding details.

Set the triage status of a tracked finding. Records the decision (accepted-risk, deferred, wont-fix, false-positive) with attribution and reason. Feeds back into calibration to reduce future false positives.

Get finding lifecycle statistics for the project: open, fixed, recurring, and triaged finding counts plus trends. Shows whether code quality is improving or declining over evaluation runs.

Analyze finding suppression patterns: FP rates by rule, suppression rates by judge, auto-suppress candidates, and actionable recommendations for tuning. Use this to understand which rules need calibration.

List findings that have been triaged (accepted-risk, deferred, wont-fix, false-positive). Optionally filter by triage status. Shows the triage decision, reason, and who made it.

Re-evaluate code with developer-provided context from a multi-turn conversation. Accepts disputed findings, accepted findings, and additional context to adjust the evaluation. This is the agentic feedback loop — the developer explains their intent and the tribunal re-evaluates with that context, applying auto-tune and confidence filtering.

Generate the boilerplate files to add a new judge to the Judges Panel. Creates the judge definition (with self-registration), evaluator skeleton, and tells you the one line to add to index.ts. Validates that the judge ID and rule prefix are unique.

Generate a starter plugin template for the Judges Panel. Creates a self-contained plugin file with custom rules, optional custom judges, and lifecycle hooks.

Evaluate code with the Judges Panel and automatically apply all available auto-fix patches. Returns the fixed code along with a summary of applied and remaining findings. Use this to fix security, performance, and quality issues in a single step.

List all available judges on the Agent Tribunal panel, including their areas of expertise and what they evaluate.

Run policy-aware tribunal evaluation with named policy profiles (startup, regulated, healthcare, fintech, public-sector), evidence calibration from runtime metrics, specialty-per-judge feedback, confidence scoring, and uncertainty reporting. Use this when code must meet specific compliance or vertical requirements.

Clone a public repository URL, run the full judges panel across source files, and generate a consolidated markdown report.

Run a 3-step app-builder workflow: tribunal review, plain-language risk translation, and prioritized remediation tasks with AI-fixable P0/P1 items.

Analyze a PACKAGE MANAGER manifest file (NOT infrastructure code) for supply-chain risks, version pinning issues, typosquatting indicators, and dependency hygiene. ONLY accepts: package.json, requirements.txt, Cargo.toml, go.mod, pom.xml, .csproj. Do NOT use this for Bicep, Terraform, ARM templates, CloudFormation, Dockerfiles, or any other infrastructure/deployment configuration — use evaluate_code or evaluate_code_single_judge for those.

Run the benchmark suite and check results against quality thresholds. Returns pass/fail with metric details including F1, precision, recall, and detection rate. Use in CI pipelines to prevent quality regressions.

Run the full benchmark suite and return a detailed dashboard with per-judge, per-category, and per-difficulty breakdowns. Includes precision, recall, F1, false positive rates, and individual case results. Use this to understand overall system quality and identify weak spots.

Evaluate multiple code files in a single call. Returns per-file verdicts with scores and findings, plus aggregate statistics.

Evaluate code and automatically generate fix patches for all findings that have auto-fix support. Returns the evaluation verdict alongside ready-to-apply patches. Use this for a single-step

Run a focused evaluation using only the specified judges. Use this after an initial full evaluation to re-check specific areas — for example, re-run only

Get the current evaluation session status — how many evaluations have been run, detected frameworks, verdict history per file, and stability indicators. Useful for understanding what the tribunal has already reviewed.

Record user feedback on a finding — mark it as a true positive (tp), false positive (fp), or won

Evaluate code changes from a git diff. Parses the unified diff from a git repository, identifies changed files and lines, and runs the full tribunal on each changed file — filtering findings to only those on changed lines. Supports both live git repos (provide repoPath + base ref) and pre-computed diffs (provide diffText).