Rules

Rules is an experimental interface. No implementation exists yet. The API described here reflects the current design and is subject to change as the protocol evolves.

Overview

Rules define declarative verification criteria that agent output must satisfy before being accepted. They are composable: rules can be evaluated individually or as a complete set against any content. Provider analogues include ESLint Rules, GitHub Checks, Vercel Deployment Checks, and OpenAI Guardrails.

Severity Levels

Severity	Meaning
`error`	The rule failure blocks acceptance. Output must not proceed.
`warning`	The rule failure is notable but does not block acceptance.
`info`	The rule result is informational only. No action is required.

TypeScript API

import type { RuleSeverity, RuleResult, Rule, Rules } from 'osprotocol/checks/rules'

RuleSeverity

type RuleSeverity = 'error' | 'warning' | 'info'

Indicates how a rule failure should be treated. An error blocks acceptance, a warning is surfaced without blocking, and info is purely observational.

RuleResult

interface RuleResult {
  ruleName: string
  passed: boolean
  severity: RuleSeverity
  message: string
  metadata?: Record<string, unknown>
}

The result returned after evaluating a single rule against content. passed indicates whether the rule was satisfied. message provides a human-readable explanation. metadata carries any structured diagnostic data the rule chooses to emit.

Rule

interface Rule {
  name: string
  description: string
  severity: RuleSeverity
  evaluate(content: unknown): Promise<RuleResult>
  metadata?: Record<string, unknown>
}

A single verifiable criterion. evaluate receives the content to check and returns a RuleResult. The severity on the Rule defines the default severity that should appear in results when the rule fails.

interface Rules {
  get(name: string): Promise<Rule | null>
  list(): Promise<Rule[]>
  evaluate(content: unknown): Promise<RuleResult[]>
}

A collection of rules. list enumerates all registered rules. get retrieves a specific rule by name. evaluate runs all rules against the provided content and returns a RuleResult for each one.

Usage Examples

Evaluate all rules against content

const results = await rules.evaluate(agentOutput)

for (const result of results) {
  if (!result.passed && result.severity === 'error') {
    throw new Error(`Rule failed: ${result.ruleName} — ${result.message}`)
  }
}

Get a specific rule by name

const rule = await rules.get('no-pii-in-output')

if (rule) {
  const result = await rule.evaluate(agentOutput)
  console.log(result.passed, result.message)
}

Define a custom rule

const noPiiRule: Rule = {
  name: 'no-pii-in-output',
  description: 'Ensures agent output does not contain personally identifiable information',
  severity: 'error',
  async evaluate(content: unknown): Promise<RuleResult> {
    const text = typeof content === 'string' ? content : JSON.stringify(content)
    const hasPii = /\b\d{3}-\d{2}-\d{4}\b/.test(text) // SSN pattern example
    return {
      ruleName: 'no-pii-in-output',
      passed: !hasPii,
      severity: 'error',
      message: hasPii ? 'Output contains potential PII' : 'No PII detected',
    }
  },
}

Integration

Rule results produced by Rules.evaluate feed into other parts of the checks and runs pipeline:

Judge — uses rule results alongside other signals to produce a quality verdict
Audit — records rule results for traceability and post-hoc review
Approval — an error-severity failure can gate a run and trigger a human approval step

Rules

On this page