← Back to leaderboardProject evidence record

Inspect AI

This free page contains the context needed to interpret the published static score. Deep review and runtime validation are separate evidence levels, not hidden score modifiers or certifications.

75Static score
out of 100

Current posture summary

The pinned Inspect AI snapshot scored 75 under the static AI baseline. All four required evidence providers completed with no unexpected scope gaps; the result describes repository-visible posture and does not test evaluation workloads or deployed runtime behavior.

Methodology: 2026-04-11.standards-static.v2

Repository: https://github.com/UKGovernmentBEIS/inspect_ai

Dimension evidence

  • Repo posture22
  • Agentic guardrails100
  • AI data exposure100
  • Observability & auditability100
  • Evidence readiness82

Strongest and weakest areas

Strongest: Agentic guardrails (100)

Weakest: Repo posture (22)

Evidence-backed strengths

  • All four required static evidence providers completed successfully.
  • No unexpected static controls were left unassessed, and repository evidence supported the assessed agentic, data-exposure, observability, and evidence-readiness controls.

Evidence-backed weaknesses

  • Build workflows reference third-party GitHub Actions by mutable version tags rather than immutable full commit SHAs.
  • The repository-posture dimension scored 22, lower than its other static dimensions in this snapshot.

Limits and next action

  • Twelve expected controls outside static reach—including prompt injection, authorization, isolation, exfiltration, resource-limit behavior, branch protection, and build provenance—were not assessed.
  • Two dependency advisories found in an Excalidraw documentation-tooling lockfile were not promoted into public project-vulnerability claims because runtime reachability and application impact were not established.
  • A scheduled model-data workflow's write permissions were suppressed as a finding because they are purpose-bound to creating a same-repository pull request and excess scope was not established.
  • Documentation, configuration, and code markers are not proof that controls remain effective in a deployed runtime.
  • Editorial adjudication was AI-assisted project-owner review, not independent certification or human ground truth.

Remediation guidance

  • Pin third-party GitHub Actions to reviewed full commit SHAs while retaining readable release tags in comments.
  • Assess documentation-tooling advisories separately with dependency-path and execution-reachability context before making vulnerability claims.
  • Use selective runtime testing to evaluate prompt-injection resistance, tool authorization, secret isolation, exfiltration boundaries, output handling, resource limits, and security telemetry.

No reviewed public case file is attached.

Public-safe finding summaries

medium

Build workflow dependencies use mutable references

The build workflow references actions/checkout@v7, astral-sh/ruff-action@v4.1.0, and astral-sh/setup-uv@v9.0.0 rather than immutable full commit SHAs. This is a static build-integrity weakness, not evidence of compromise.

.github/workflows/build.yml:29.github/workflows/build.yml:37.github/workflows/build.yml:59
Provenance

Source run: run_inspect-ai-pinned_55589fc8-fa77-4a45-b5ef-ce5263aa4904

Source schema: executive_summary.v1

SHA-256: 7e800681b4f19fc1cb1bd2991a4c5bc34225edb9dbf20c0248c6f046e0fbc32c

Independent commercial boundary

Need a fresh, private, or deeper assessment?

thethermark can expand scope, evidence, runtime validation, monitoring, and reporting. Payment never predetermines or changes the public result.

Request deeper assessment