Configure with natural language or YAML

The Settings tab on every agent page has two equal-weight surfaces:

An NL command bar at the top — type "require PR gate" or "alert #ops"
A YAML editor on the left — the canonical source-of-truth representation

They round-trip cleanly through the same AgentSettings Zod schema. P7 ("Composable via chat") and P8 ("Programmable in both directions") of the AI-native principles ADR require this: settings must be expressible as plain English to first-time users and as version-controlled YAML to power users, with no information loss between the two.

The NL command bar

Open /app/agents/[id]/settings and look at the bar at the top. Type a command and hit Enter — the parser returns a structured diff against your current settings, the UI renders the change inline, and one click commits it.

The seven Sprint 14 rules cover the high-frequency moves:

Phrasing	Effect
`require PR gate`	`policies.pr_gate_required = true`
`set SLO to 95%`, `slo 95`, `set slo to 0.95`	`policies.slo_floor = 0.95`
`alert #cx-alerts`, `add slack #cx-alerts`, `send alerts to #ops`	adds the channel to `integrations.slack.channels`
`remove slack #cx-alerts`, `stop alerting #ops`	removes the channel
`make alice@example.com editor`, `add @alice as editor`	adds to `access.editors`
`make demo`, `make public`, `make private`, `hide from team`	flips `agent.demo`
`rotate api keys`	rejected with a pointer to Workspace > API keys (per-workspace action, not per-agent)

Anything outside the seven rules returns:

ok: false
reason: "Couldn't parse intent"
suggestions:
  - Try: "require PR gate"
  - Try: "set SLO to 95%"
  - Try: "add slack #alerts"

The UI surfaces those three suggestions as buttons so first-time users learn supported phrasing.

The parser lives at lib/settings/nl-parser.ts. The LLM swap-in is marked at the top of parseNlSettings() — Sprint 15+ replaces the rule engine with an LLM caller (same return shape, same Zod-validated JSON) so the UI doesn't change.

Confidence on every parse

Every NL match carries a confidence, capped at 0.99 by P9 (no false certainty):

0.85 — unambiguous lexical match (one rule fires, intent fully expressible)
0.55 — ambiguous match (partial intent, channel name inferred, missing target)

The chip color on the inline diff preview tracks confidence — high (≥0.8) is green, med (0.6–0.8) is yellow, low (<0.6) is red. You can commit at any confidence; the bar shows the parsed reasoning so you can sanity-check before clicking apply.

Preview from the API

The NL surface is also an HTTP endpoint, so you can wire it into a Slack slash command or your own bot:

curl -s -X POST https://runtime-judgement-app.vercel.app/api/agents/$AGENT_ID/settings/preview-nl \
  -H "Authorization: Bearer $RJ_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"command":"alert #cx-alerts"}' | jq

Response:

{
  "ok": true,
  "diff": {
    "from": { "agent": { ... }, "integrations": { "slack": { "channels": [] } } },
    "to":   { "agent": { ... }, "integrations": { "slack": { "channels": ["#cx-alerts"] } } },
    "yamlDiff": "  agent:\n    id: ...\n+ integrations:\n+   slack:\n+     channels:\n+       - \"#cx-alerts\""
  },
  "confidence": 0.85,
  "reasoning": "Adds #cx-alerts to Slack alert channels."
}

This route only previews. To apply, PUT the resulting YAML to /api/agents/[id]/settings (the same endpoint the UI's apply button hits).

YAML round-trip

The YAML editor on the Settings page is the source-of-truth surface. The serializer is in lib/settings/yaml.ts and follows three rules:

Canonical key order: agent → policies → integrations → access → pipeline
2-space indentation, no line-wrap (lineWidth: 0)
Empty optional sub-objects are not emitted (a fresh agent's YAML is just an agent: block)

A populated settings YAML looks like:

agent:
  id: 01HZAGENT...
  name: cx-support
  color: indigo
  owner: ross@proveai.com
  demo: false

policies:
  pr_gate_required: true
  slo_floor: 0.95
  drift_alert_threshold: 0.1

integrations:
  github:
    repo: my-org/cx-support
    checks: true
  slack:
    channels:
      - "#cx-alerts"
      - "#rj-prod-incidents"
  cursor_mcp:
    enabled: true

access:
  owners:
    - ross@proveai.com
  editors:
    - alice@example.com
  viewers:
    - bob@example.com

pipeline:
  default: q72-k1

Programmatic helpers:

import {
  settingsToYaml,
  yamlToSettings,
  validateSettingsYaml,
} from "@/lib/settings/yaml"

const yaml = settingsToYaml(spec)         // AgentSettings → string
const back = yamlToSettings(yaml)         // string → AgentSettings (throws on invalid)
const check = validateSettingsYaml(yaml)  // string → { ok, spec | errors }

The round-trip property yamlToSettings(settingsToYaml(spec)) deep-equals spec for every spec the Zod schema accepts. Comments on top-level keys survive a parseSettingsDoc() round-trip; the lossy helpers above drop them by design (they go through Zod).

The same shape is also exposed for tests (lib/yaml/tests.ts) and suites (lib/yaml/suites.ts) — see the suite version in Build a snapshot suite for regression gating.

When to use which

NL — for changes you can describe in one sentence. Fastest path for the seven common moves. The chip and inline diff preview let you cancel before committing.
YAML — for changes you want in version control, for bulk edits, for anything outside the seven rules. The YAML editor's apply button calls the same PUT endpoint.

Both surfaces commit through the same write path. The audit log (P10, migration 0029) records which surface the change came from so the timeline shows e.g. "Ross set policies.slo_floor = 0.95 via NL command on 2026-05-26 17:42 CEST".

Pitfalls

hide from team is ambiguous

It flips agent.demo = false (matching "hide from anyone but owners"), confidence 0.55. If you meant something else (e.g. remove a specific user from access.viewers), use the explicit YAML edit instead — the parser will surface the ambiguity rather than guess.

Slack channel without #

add slack alerts adds #alerts (the parser normalises). If you actually meant a private channel name without #, edit the YAML directly — the NL surface always prefixes.

Idempotent commands return ok:true with no diff

alert #ops when #ops is already configured returns ok: true with from === to. The reasoning string explains: "Slack channel #ops is already configured — no change." This keeps the contract uniform — clients don't need a separate "no-op" branch.

Next steps

What's around the Settings tab: What's an AI-native agent page? — Settings is one of six sections; the other five follow the same NL + structured pattern
Apply settings from the terminal: Use the rj CLI — rj settings apply <file> lands Sprint 15+; today use curl PUT /api/agents/{id}/settings
Lock down regression behaviour: Build a snapshot suite for regression gating — the policies.pr_gate_required field this article configures is what the gate respects
The ADR that motivates bidirectional configuration: dev-docs/strategy/2026-05-26-adr-ai-native-principles.md (P7 + P8)

Configure with natural language or YAML

Configure with natural language or YAML

The NL command bar

Confidence on every parse

Preview from the API

YAML round-trip

When to use which

Pitfalls

Next steps

Related articles

Try it