Can you update the logging to also include a debug log message containing the AI models returned evaluation?


- update-config: Use this skill to configure the Claude Code harness via settings.json. Automated behaviors ("from now on when X", "each time X", "whenever X", "before/after X") require hooks configured in settings.json - the harness executes these, not Claude, so memory/preferences cannot fulfill them. Also use for: permissions ("allow X", "add permission", "move permission to"), env vars ("set X=Y"), hook troubleshooting, or any changes to settings.json/settings.local.json files. Examples: "allow npm commands", "add bq permission to global settings", "move permission to user settings", "set DEBUG=true", "when claude stops show X". For simple settings like theme/model, suggest the /config command.
- keybindings-help: Use when the user wants to customize keyboard shortcuts, rebind keys, add chord bindings, or modify ~/.claude/keybindings.json. Examples: "rebind ctrl+s", "add a chord shortcut", "change the submit key", "customize keybindings".
- simplify: Review changed code for reuse, quality, and efficiency, then fix any issues found.
- fewer-permission-prompts: Scan your transcripts for common read-only Bash and MCP tool calls, then add a prioritized allowlist to project .claude/settings.json to reduce permission prompts.
- loop: Run a prompt or slash command on a recurring interval (e.g. /loop 5m /foo). Omit the interval to let the model self-pace. - When the user wants to set up a recurring task, poll for status, or run something repeatedly on an interval (e.g. "check the deploy every 5 minutes", "keep running /babysit-prs"). Do NOT invoke for one-off tasks.
- schedule: Create, update, list, or run scheduled remote agents (routines) that execute on a cron schedule. - When the user wants to schedule a recurring remote agent, set up automated tasks, create a cron job for Claude Code, or manage their scheduled agents/routines. Also use when the user wants a one-time scheduled run ("run this once at 3pm", "remind me to check X tomorrow").
- claude-api: Build, debug, and optimize Claude API / Anthropic SDK apps. Apps built with this skill should include prompt caching. Also handles migrating existing Claude API code between Claude model versions (4.5 → 4.6, 4.6 → 4.7, retired-model replacements).
TRIGGER when: code imports `anthropic`/`@anthropic-ai/sdk`; user asks for the Claude API, Anthropic SDK, or Managed Agents; user adds/modifies/tunes a Claude feature (caching, thinking, compaction, tool use, batch, files, citations, memory) or model (Opus/Sonnet/Haiku) in a file; questions about prompt caching / cache hit rate in an Anthropic SDK project.
SKIP: file imports `openai`/other-provider SDK, filename like `*-openai.py`/`*-generic.py`, provider-neutral code, general programming/ML.
- init: Initialize a new CLAUDE.md file with codebase documentation
- review: Review a pull request
- security-review: Complete a security review of the pending changes on the current branch


Bash

List ruth app contents


drwxr-xr-x steve.briggs staff 288 B  Tue Apr 28 15:21:20 2026 ..
drwxr-xr-x steve.briggs staff 128 B  Tue Apr 28 15:21:20 2026 cmd
.rw-r--r-- steve.briggs staff 2.1 KB Tue Apr 28 15:21:20 2026 app_test.go
.rw-r--r-- steve.briggs staff 2.3 KB Tue Apr 28 15:21:20 2026 config.go
.rw-r--r-- steve.briggs staff 2.6 KB Tue Apr 28 15:21:20 2026 env.go
.rw-r--r-- steve.briggs staff 3.6 KB Tue Apr 28 15:21:20 2026 go.mod
.rw-r--r-- steve.briggs staff 3.6 KB Tue Apr 28 15:21:20 2026 ruth.toml.example
.rw-r--r-- steve.briggs staff 3.7 KB Tue Apr 28 15:21:20 2026 env_test.go
drwxr-xr-x steve.briggs staff 128 B  Tue Apr 28 15:21:20 2026 reputation
drwxr-xr-x steve.briggs staff  96 B  Tue Apr 28 15:21:20 2026 .claude
.rw-r--r-- steve.briggs staff  18 KB Tue Apr 28 15:21:20 2026 go.sum
.rw-r--r-- steve.briggs staff 9.7 KB Tue Apr 28 15:21:20 2026 README.md
drwxr-xr-x steve.briggs staff 416 B  Tue Apr 28 15:21:20 2026 local
.rw-r--r-- steve.briggs staff 9.1 KB Tue Apr 28 15:21:20 2026 app.go
drwxr-xr-x steve.briggs staff 128 B  Tue Apr 28 15:21:20 2026 jira
.rw-r--r-- steve.briggs staff 1.5 MB Tue Apr 28 15:21:20 2026 logo.png
drwxr-xr-x steve.briggs staff 192 B  Tue Apr 28 15:21:20 2026 ai
drwxr-xr-x steve.briggs staff  96 B  Tue Apr 28 15:21:20 2026 telemetry
.rw-r--r-- steve.briggs staff 3.7 KB Tue Apr 28 15:21:20 2026 config_test.go
drwxr-xr-x steve.briggs staff 160 B  Tue Apr 28 15:21:20 2026 risk
.rw-r--r-- steve.briggs staff 1.8 KB Tue Apr 28 15:21:20 2026 ruth.toml
drwxr-xr-x steve.briggs staff 224 B  Tue Apr 28 15:21:20 2026 .idea
.rw-r--r-- steve.briggs staff  16 KB Tue Apr 28 15:21:20 2026 .app.go.swp
drwxr-xr-x steve.briggs staff 800 B  Tue Apr 28 15:21:31 2026 .
.rw-r--r-- steve.briggs staff  31 B  Tue Apr 28 15:22:36 2026 .gitignore


List ai dir


.rw-r--r-- steve.briggs staff 3.7 KB Tue Apr 28 15:21:20 2026 provider.go
.rw-r--r-- steve.briggs staff 1.3 KB Tue Apr 28 15:21:20 2026 random.go
.rw-r--r-- steve.briggs staff 1.8 KB Tue Apr 28 15:21:20 2026 anthropic.go
drwxr-xr-x steve.briggs staff 192 B  Tue Apr 28 15:21:20 2026 .
.rw-r--r-- steve.briggs staff 3.0 KB Tue Apr 28 15:21:20 2026 bedrock.go
drwxr-xr-x steve.briggs staff 800 B  Tue Apr 28 15:21:31 2026 ..


You've hit your limit · resets Apr 30 at 8pm (America/New_York)


<local-command-caveat>Caveat: The messages below were generated by the user while running local commands. DO NOT respond to these messages or otherwise consider them in your response unless the user explicitly asks you to.</local-command-caveat>


<command-name>/rate-limit-options</command-name>
            <command-message>rate-limit-options</command-message>
            <command-args></command-args>


<local-command-stdout>Request sent to your admin to increase extra usage.</local-command-stdout>


1	package ai
2
3	import (
4		"context"
5		"encoding/json"
6		"fmt"
7		"regexp"
8		"strings"
9	)
10
11	type Config struct {
12		Provider  string          `toml:"provider"`
13		Anthropic AnthropicConfig `toml:"anthropic"`
14		Bedrock   BedrockConfig   `toml:"bedrock"`
15		Random    RandomConfig    `toml:"random"`
16	}
17
18	func (c *Config) Validate() error {
19		switch c.Provider {
20		case "anthropic":
21			if c.Anthropic.APIKey == "" {
22				return fmt.Errorf("anthropic.api_key is required")
23			}
24		case "bedrock":
25			if c.Bedrock.Region == "" {
26				return fmt.Errorf("bedrock.region is required")
27			}
28		case "random":
29			// no required fields; testing-only provider.
30		default:
31			return fmt.Errorf("provider must be one of: anthropic, bedrock, random")
32		}
33		return nil
34	}
35
36	type ScoreRequest struct {
37		User              string   `json:"user"`
38		Roles             []string `json:"roles"`
39		Resources         []string `json:"resources"`
40		Reason            string   `json:"reason"`
41		HasCUSTXReference bool     `json:"has_custx_reference"`
42		DescriptionLength int      `json:"description_length"`
43	}
44
45	type Assessment struct {
46		DescriptionQuality int    `json:"description_quality"`
47		PrivilegeConcern   int    `json:"privilege_concern"`
48		Anomalies          int    `json:"anomalies"`
49		Reasoning          string `json:"reasoning"`
50	}
51
52	type Provider interface {
53		Assess(ctx context.Context, req ScoreRequest) (*Assessment, error)
54	}
55
56	func New(cfg Config) (Provider, error) {
57		switch cfg.Provider {
58		case "anthropic":
59			return NewAnthropic(cfg.Anthropic), nil
60		case "bedrock":
61			return NewBedrock(cfg.Bedrock)
62		case "random":
63			return NewRandom(cfg.Random), nil
64		}
65		return nil, fmt.Errorf("unknown provider: %s", cfg.Provider)
66	}
67
68	const SystemPrompt = `You are a security analyst reviewing a Teleport access request.
69	Assess the risk of granting the requested access based on the details provided.
70
71	Return ONLY a JSON object with these exact fields (no prose, no markdown fence):
72	{
73	  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
74	  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
75	  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
76	  "reasoning": "<2-3 sentences explaining the scores>"
77	}
78
79	Scoring guidance:
80	- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
81	- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
82	- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`
83
84	func UserPrompt(req ScoreRequest) string {
85		var sb strings.Builder
86		sb.WriteString("Access request details:\n\n")
87		fmt.Fprintf(&sb, "User: %s\n", req.User)
88		fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
89		if len(req.Resources) > 0 {
90			fmt.Fprintf(&sb, "Requested resources: %s\n", strings.Join(req.Resources, ", "))
91		}
92		fmt.Fprintf(&sb, "Contains CUSTX ticket reference: %v\n", req.HasCUSTXReference)
93		fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
94		sb.WriteString("Request reason (verbatim):\n")
95		sb.WriteString("---\n")
96		sb.WriteString(req.Reason)
97		sb.WriteString("\n---\n\nReturn the JSON assessment.")
98		return sb.String()
99	}
100
101	var jsonRE = regexp.MustCompile(`(?s)\{.*\}`)
102
103	func parseAssessment(text string) (*Assessment, error) {
104		m := jsonRE.FindString(text)
105		if m == "" {
106			return nil, fmt.Errorf("no JSON object in model output: %q", text)
107		}
108		var a Assessment
109		if err := json.Unmarshal([]byte(m), &a); err != nil {
110			return nil, fmt.Errorf("parse assessment: %w (raw: %q)", err, m)
111		}
112		return &a, nil
113	}
114

package ai

import (
	"context"
	"encoding/json"
	"fmt"
	"regexp"
	"strings"
)

type Config struct {
	Provider  string          `toml:"provider"`
	Anthropic AnthropicConfig `toml:"anthropic"`
	Bedrock   BedrockConfig   `toml:"bedrock"`
	Random    RandomConfig    `toml:"random"`
}

func (c *Config) Validate() error {
	switch c.Provider {
	case "anthropic":
		if c.Anthropic.APIKey == "" {
			return fmt.Errorf("anthropic.api_key is required")
		}
	case "bedrock":
		if c.Bedrock.Region == "" {
			return fmt.Errorf("bedrock.region is required")
		}
	case "random":
		// no required fields; testing-only provider.
	default:
		return fmt.Errorf("provider must be one of: anthropic, bedrock, random")
	}
	return nil
}

type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
	DescriptionLength int      `json:"description_length"`
}

type Assessment struct {
	DescriptionQuality int    `json:"description_quality"`
	PrivilegeConcern   int    `json:"privilege_concern"`
	Anomalies          int    `json:"anomalies"`
	Reasoning          string `json:"reasoning"`
}

type Provider interface {
	Assess(ctx context.Context, req ScoreRequest) (*Assessment, error)
}

func New(cfg Config) (Provider, error) {
	switch cfg.Provider {
	case "anthropic":
		return NewAnthropic(cfg.Anthropic), nil
	case "bedrock":
		return NewBedrock(cfg.Bedrock)
	case "random":
		return NewRandom(cfg.Random), nil
	}
	return nil, fmt.Errorf("unknown provider: %s", cfg.Provider)
}

const SystemPrompt = `You are a security analyst reviewing a Teleport access request.
Assess the risk of granting the requested access based on the details provided.

Return ONLY a JSON object with these exact fields (no prose, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences explaining the scores>"
}

Scoring guidance:
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

func UserPrompt(req ScoreRequest) string {
	var sb strings.Builder
	sb.WriteString("Access request details:\n\n")
	fmt.Fprintf(&sb, "User: %s\n", req.User)
	fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
	if len(req.Resources) > 0 {
		fmt.Fprintf(&sb, "Requested resources: %s\n", strings.Join(req.Resources, ", "))
	}
	fmt.Fprintf(&sb, "Contains CUSTX ticket reference: %v\n", req.HasCUSTXReference)
	fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
	sb.WriteString("Request reason (verbatim):\n")
	sb.WriteString("---\n")
	sb.WriteString(req.Reason)
	sb.WriteString("\n---\n\nReturn the JSON assessment.")
	return sb.String()
}

var jsonRE = regexp.MustCompile(`(?s)\{.*\}`)

func parseAssessment(text string) (*Assessment, error) {
	m := jsonRE.FindString(text)
	if m == "" {
		return nil, fmt.Errorf("no JSON object in model output: %q", text)
	}
	var a Assessment
	if err := json.Unmarshal([]byte(m), &a); err != nil {
		return nil, fmt.Errorf("parse assessment: %w (raw: %q)", err, m)
	}
	return &a, nil
}


1	package ruth
2
3	import (
4		"context"
5		"fmt"
6		"log/slog"
7		"sync"
8		"time"
9
10		"github.com/gravitational/teleport/api/client"
11		"github.com/gravitational/teleport/api/types"
12		"golang.org/x/sync/errgroup"
13
14		"ruth/ai"
15		"ruth/jira"
16		"ruth/reputation"
17		"ruth/risk"
18		"ruth/telemetry"
19	)
20
21	type App struct {
22		cfg    *Config
23		tele   *client.Client
24		ai     ai.Provider
25		jira   *jira.Client
26		scorer *risk.Scorer
27		rep    *reputation.Store
28		meter  *telemetry.Meter
29
30		pending   map[string]string
31		pendingMu sync.Mutex
32	}
33
34	func NewApp(ctx context.Context, cfg *Config) (*App, error) {
35		tele, err := client.New(ctx, client.Config{
36			Addrs: []string{cfg.Teleport.Addr},
37			Credentials: []client.Credentials{
38				client.LoadIdentityFile(cfg.Teleport.IdentityFile),
39			},
40		})
41		if err != nil {
42			return nil, fmt.Errorf("teleport client: %w", err)
43		}
44
45		prov, err := ai.New(cfg.AI)
46		if err != nil {
47			return nil, fmt.Errorf("ai provider: %w", err)
48		}
49
50		jc, err := jira.New(cfg.Jira)
51		if err != nil {
52			return nil, fmt.Errorf("jira client: %w", err)
53		}
54
55		scorer, err := risk.NewScorer(cfg.Risk)
56		if err != nil {
57			return nil, fmt.Errorf("scorer: %w", err)
58		}
59
60		store, err := reputation.NewStore(cfg.Reputation)
61		if err != nil {
62			return nil, fmt.Errorf("reputation store: %w", err)
63		}
64
65		var meter *telemetry.Meter
66		if cfg.Telemetry.Enabled {
67			meter, err = telemetry.New(ctx, cfg.Telemetry, store)
68			if err != nil {
69				return nil, fmt.Errorf("telemetry: %w", err)
70			}
71		}
72
73		return &App{
74			cfg:     cfg,
75			tele:    tele,
76			ai:      prov,
77			jira:    jc,
78			scorer:  scorer,
79			rep:     store,
80			meter:   meter,
81			pending: map[string]string{},
82		}, nil
83	}
84
85	// Close flushes outstanding telemetry. Safe to call even if telemetry
86	// is disabled.
87	func (a *App) Close(ctx context.Context) error {
88		return a.meter.Shutdown(ctx)
89	}
90
91	func (a *App) Run(ctx context.Context) error {
92		slog.Info("ruth starting",
93			"provider", a.cfg.AI.Provider,
94			"threshold", a.cfg.Risk.TicketThreshold,
95			"itsm_project", a.cfg.Jira.Project,
96			"dry_run", a.cfg.DryRun)
97		if a.cfg.DryRun {
98			slog.Warn("dry-run enabled: no Teleport reviews or Jira tickets will be submitted")
99		}
100
101		if err := a.rehydrate(ctx); err != nil {
102			return fmt.Errorf("rehydrate open ITSM tickets: %w", err)
103		}
104
105		// Run watcher/poller under their own cancelable context rather than
106		// bolting them directly to the signal context. A coordinator
107		// goroutine waits for the signal, then sleeps for
108		// telemetry.shutdown_grace before cancelling the errgroup so that
109		// the OTel periodic reader can finish an in-flight export cleanly.
110		runCtx, cancelRun := context.WithCancel(context.Background())
111		defer cancelRun()
112		g, gctx := errgroup.WithContext(runCtx)
113		g.Go(func() error { return a.watchRequests(gctx) })
114		g.Go(func() error { return a.pollITSM(gctx) })
115		g.Go(func() error {
116			select {
117			case <-ctx.Done():
118				grace := time.Duration(0)
119				if a.cfg.Telemetry.Enabled {
120					grace = a.cfg.Telemetry.ShutdownGrace
121				}
122				if grace > 0 {
123					slog.Info("shutdown signal received; waiting for telemetry flush", "grace", grace)
124					select {
125					case <-time.After(grace):
126					case <-gctx.Done():
127						// errgroup collapsed for some other reason; stop waiting.
128					}
129				}
130				cancelRun()
131			case <-gctx.Done():
132			}
133			return nil
134		})
135
136		return g.Wait()
137	}
138
139	func (a *App) rehydrate(ctx context.Context) error {
140		open, err := a.jira.ListOpenTeleportTickets(ctx)
141		if err != nil {
142			return err
143		}
144		a.pendingMu.Lock()
145		for reqID, ticket := range open {
146			a.pending[reqID] = ticket
147		}
148		a.pendingMu.Unlock()
149		slog.Info("rehydrated pending tickets", "count", len(open))
150		return nil
151	}
152
153	func (a *App) watchRequests(ctx context.Context) error {
154		watcher, err := a.tele.NewWatcher(ctx, types.Watch{
155			Kinds: []types.WatchKind{{Kind: types.KindAccessRequest}},
156		})
157		if err != nil {
158			return fmt.Errorf("start watcher: %w", err)
159		}
160		defer watcher.Close()
161
162		for {
163			select {
164			case <-ctx.Done():
165				return ctx.Err()
166			case <-watcher.Done():
167				return watcher.Error()
168			case ev := <-watcher.Events():
169				if ev.Type != types.OpPut {
170					continue
171				}
172				req, ok := ev.Resource.(types.AccessRequest)
173				if !ok {
174					continue
175				}
176				if req.GetState() != types.RequestState_PENDING {
177					continue
178				}
179				if !a.reserveOrSkip(req.GetName()) {
180					continue
181				}
182				go a.handleRequest(ctx, req)
183			}
184		}
185	}
186
187	// reserveOrSkip atomically claims a slot for request id. Returns true if
188	// this caller won the race and should process the request; false if the
189	// id is already in flight or already has a ticket. Using the same map
190	// for both reservations and ticket mappings keeps dedup decisions on a
191	// single source of truth.
192	func (a *App) reserveOrSkip(id string) bool {
193		a.pendingMu.Lock()
194		defer a.pendingMu.Unlock()
195		if _, held := a.pending[id]; held {
196			return false
197		}
198		a.pending[id] = ""
199		return true
200	}
201
202	func (a *App) releasePending(id string) {
203		a.pendingMu.Lock()
204		defer a.pendingMu.Unlock()
205		delete(a.pending, id)
206	}
207
208	func (a *App) recordTicket(id, ticket string) {
209		a.pendingMu.Lock()
210		defer a.pendingMu.Unlock()
211		a.pending[id] = ticket
212	}
213
214	func (a *App) handleRequest(ctx context.Context, req types.AccessRequest) {
215		id := req.GetName()
216		user := req.GetUser()
217		log := slog.With("request_id", id, "user", user)
218
219		// Release the reservation unless we successfully record a ticket;
220		// that way score/Jira failures are re-tryable on the next OpPut and
221		// auto-approved requests don't leave stale entries behind.
222		handled := false
223		defer func() {
224			if !handled {
225				a.releasePending(id)
226			}
227		}()
228
229		log.Info("scoring access request")
230		result, err := a.scorer.Score(ctx, a.ai, req)
231		if err != nil {
232			log.Error("scoring failed", "err", err)
233			return
234		}
235		log.Info("risk scored",
236			"score", result.Score,
237			"tier", result.Tier.Name,
238			"components", result.Components)
239
240		assessment, err := a.rep.Assess(ctx, user, result.Components)
241		if err != nil {
242			log.Warn("reputation update failed", "err", err)
243		}
244		if len(assessment.Penalties) > 0 {
245			reasons := make([]string, 0, len(assessment.Penalties))
246			for _, p := range assessment.Penalties {
247				reasons = append(reasons, p.Reason)
248			}
249			log.Info("reputation docked",
250				"previous", assessment.Previous,
251				"current", assessment.Score,
252				"reasons", reasons)
253		}
254
255		threshold := a.cfg.Risk.TicketThreshold
256		if result.Escalation {
257			threshold = a.cfg.Risk.EscalationThresholdFor(req.GetRoles())
258			log.Info("session escalation detected", "threshold", threshold)
259		}
260
261		if result.Score < threshold {
262			reason := fmt.Sprintf("ruth: %s (score %d) — auto-approved",
263				result.Tier.Name, result.Score)
264			if err := a.submitReview(ctx, id, types.RequestState_APPROVED, reason); err != nil {
265				log.Error("auto-approve failed", "err", err)
266				return
267			}
268			log.Info("access request auto-approved",
269				"score", result.Score,
270				"tier", result.Tier.Name,
271				"threshold", threshold)
272			return
273		}
274
275		if a.cfg.DryRun {
276			log.Info("dry-run: would create ITSM ticket and hold request pending",
277				"score", result.Score,
278				"tier", result.Tier.Name,
279				"threshold", threshold)
280			return
281		}
282
283		ticket, err := a.jira.CreateITSMTicket(ctx, jira.TicketInput{
284			RequestID:   id,
285			User:        req.GetUser(),
286			Roles:       req.GetRoles(),
287			Reason:      req.GetRequestReason(),
288			Score:       result.Score,
289			Tier:        result.Tier.Name,
290			Components:  result.Components,
291			AIReasoning: result.Reasoning,
292			Escalation:  result.Escalation,
293		})
294		if err != nil {
295			log.Error("create ITSM ticket failed", "err", err)
296			return
297		}
298
299		a.recordTicket(id, ticket)
300		handled = true
301		log.Info("ITSM ticket created, request held for SRE review",
302			"ticket", ticket, "tier", result.Tier.Name)
303	}
304
305	func (a *App) pollITSM(ctx context.Context) error {
306		t := time.NewTicker(a.cfg.Jira.PollInterval)
307		defer t.Stop()
308		for {
309			select {
310			case <-ctx.Done():
311				return ctx.Err()
312			case <-t.C:
313				a.reconcileITSM(ctx)
314			}
315		}
316	}
317
318	func (a *App) reconcileITSM(ctx context.Context) {
319		a.pendingMu.Lock()
320		snap := make(map[string]string, len(a.pending))
321		for k, v := range a.pending {
322			snap[k] = v
323		}
324		a.pendingMu.Unlock()
325
326		for requestID, ticket := range snap {
327			if ticket == "" {
328				// Reservation for an in-flight handleRequest; nothing to poll yet.
329				continue
330			}
331			status, err := a.jira.TicketStatus(ctx, ticket)
332			if err != nil {
333				slog.Warn("ITSM status check failed", "ticket", ticket, "err", err)
334				continue
335			}
336			var next types.RequestState
337			switch status {
338			case jira.StatusApproved:
339				next = types.RequestState_APPROVED
340			case jira.StatusRejected:
341				next = types.RequestState_DENIED
342			default:
343				continue
344			}
345			reason := fmt.Sprintf("ruth: ITSM %s resolved (%s)", ticket, status)
346			if err := a.submitReview(ctx, requestID, next, reason); err != nil {
347				slog.Error("submit review failed", "request_id", requestID, "err", err)
348				continue
349			}
350			a.pendingMu.Lock()
351			delete(a.pending, requestID)
352			a.pendingMu.Unlock()
353		}
354	}
355
356	func (a *App) submitReview(ctx context.Context, requestID string, state types.RequestState, reason string) error {
357		if a.cfg.DryRun {
358			slog.Info("dry-run: skipping AccessReview submission",
359				"request_id", requestID,
360				"state", state.String(),
361				"reason", reason)
362			return nil
363		}
364		_, err := a.tele.SubmitAccessReview(ctx, types.AccessReviewSubmission{
365			RequestID: requestID,
366			Review: types.AccessReview{
367				Author:        "ruth",
368				ProposedState: state,
369				Reason:        reason,
370			},
371		})
372		return err
373	}
374

package ruth

import (
	"context"
	"fmt"
	"log/slog"
	"sync"
	"time"

	"github.com/gravitational/teleport/api/client"
	"github.com/gravitational/teleport/api/types"
	"golang.org/x/sync/errgroup"

	"ruth/ai"
	"ruth/jira"
	"ruth/reputation"
	"ruth/risk"
	"ruth/telemetry"
)

type App struct {
	cfg    *Config
	tele   *client.Client
	ai     ai.Provider
	jira   *jira.Client
	scorer *risk.Scorer
	rep    *reputation.Store
	meter  *telemetry.Meter

	pending   map[string]string
	pendingMu sync.Mutex
}

func NewApp(ctx context.Context, cfg *Config) (*App, error) {
	tele, err := client.New(ctx, client.Config{
		Addrs: []string{cfg.Teleport.Addr},
		Credentials: []client.Credentials{
			client.LoadIdentityFile(cfg.Teleport.IdentityFile),
		},
	})
	if err != nil {
		return nil, fmt.Errorf("teleport client: %w", err)
	}

	prov, err := ai.New(cfg.AI)
	if err != nil {
		return nil, fmt.Errorf("ai provider: %w", err)
	}

	jc, err := jira.New(cfg.Jira)
	if err != nil {
		return nil, fmt.Errorf("jira client: %w", err)
	}

	scorer, err := risk.NewScorer(cfg.Risk)
	if err != nil {
		return nil, fmt.Errorf("scorer: %w", err)
	}

	store, err := reputation.NewStore(cfg.Reputation)
	if err != nil {
		return nil, fmt.Errorf("reputation store: %w", err)
	}

	var meter *telemetry.Meter
	if cfg.Telemetry.Enabled {
		meter, err = telemetry.New(ctx, cfg.Telemetry, store)
		if err != nil {
			return nil, fmt.Errorf("telemetry: %w", err)
		}
	}

	return &App{
		cfg:     cfg,
		tele:    tele,
		ai:      prov,
		jira:    jc,
		scorer:  scorer,
		rep:     store,
		meter:   meter,
		pending: map[string]string{},
	}, nil
}

// Close flushes outstanding telemetry. Safe to call even if telemetry
// is disabled.
func (a *App) Close(ctx context.Context) error {
	return a.meter.Shutdown(ctx)
}

func (a *App) Run(ctx context.Context) error {
	slog.Info("ruth starting",
		"provider", a.cfg.AI.Provider,
		"threshold", a.cfg.Risk.TicketThreshold,
		"itsm_project", a.cfg.Jira.Project,
		"dry_run", a.cfg.DryRun)
	if a.cfg.DryRun {
		slog.Warn("dry-run enabled: no Teleport reviews or Jira tickets will be submitted")
	}

	if err := a.rehydrate(ctx); err != nil {
		return fmt.Errorf("rehydrate open ITSM tickets: %w", err)
	}

	// Run watcher/poller under their own cancelable context rather than
	// bolting them directly to the signal context. A coordinator
	// goroutine waits for the signal, then sleeps for
	// telemetry.shutdown_grace before cancelling the errgroup so that
	// the OTel periodic reader can finish an in-flight export cleanly.
	runCtx, cancelRun := context.WithCancel(context.Background())
	defer cancelRun()
	g, gctx := errgroup.WithContext(runCtx)
	g.Go(func() error { return a.watchRequests(gctx) })
	g.Go(func() error { return a.pollITSM(gctx) })
	g.Go(func() error {
		select {
		case <-ctx.Done():
			grace := time.Duration(0)
			if a.cfg.Telemetry.Enabled {
				grace = a.cfg.Telemetry.ShutdownGrace
			}
			if grace > 0 {
				slog.Info("shutdown signal received; waiting for telemetry flush", "grace", grace)
				select {
				case <-time.After(grace):
				case <-gctx.Done():
					// errgroup collapsed for some other reason; stop waiting.
				}
			}
			cancelRun()
		case <-gctx.Done():
		}
		return nil
	})

	return g.Wait()
}

func (a *App) rehydrate(ctx context.Context) error {
	open, err := a.jira.ListOpenTeleportTickets(ctx)
	if err != nil {
		return err
	}
	a.pendingMu.Lock()
	for reqID, ticket := range open {
		a.pending[reqID] = ticket
	}
	a.pendingMu.Unlock()
	slog.Info("rehydrated pending tickets", "count", len(open))
	return nil
}

func (a *App) watchRequests(ctx context.Context) error {
	watcher, err := a.tele.NewWatcher(ctx, types.Watch{
		Kinds: []types.WatchKind{{Kind: types.KindAccessRequest}},
	})
	if err != nil {
		return fmt.Errorf("start watcher: %w", err)
	}
	defer watcher.Close()

	for {
		select {
		case <-ctx.Done():
			return ctx.Err()
		case <-watcher.Done():
			return watcher.Error()
		case ev := <-watcher.Events():
			if ev.Type != types.OpPut {
				continue
			}
			req, ok := ev.Resource.(types.AccessRequest)
			if !ok {
				continue
			}
			if req.GetState() != types.RequestState_PENDING {
				continue
			}
			if !a.reserveOrSkip(req.GetName()) {
				continue
			}
			go a.handleRequest(ctx, req)
		}
	}
}

// reserveOrSkip atomically claims a slot for request id. Returns true if
// this caller won the race and should process the request; false if the
// id is already in flight or already has a ticket. Using the same map
// for both reservations and ticket mappings keeps dedup decisions on a
// single source of truth.
func (a *App) reserveOrSkip(id string) bool {
	a.pendingMu.Lock()
	defer a.pendingMu.Unlock()
	if _, held := a.pending[id]; held {
		return false
	}
	a.pending[id] = ""
	return true
}

func (a *App) releasePending(id string) {
	a.pendingMu.Lock()
	defer a.pendingMu.Unlock()
	delete(a.pending, id)
}

func (a *App) recordTicket(id, ticket string) {
	a.pendingMu.Lock()
	defer a.pendingMu.Unlock()
	a.pending[id] = ticket
}

func (a *App) handleRequest(ctx context.Context, req types.AccessRequest) {
	id := req.GetName()
	user := req.GetUser()
	log := slog.With("request_id", id, "user", user)

	// Release the reservation unless we successfully record a ticket;
	// that way score/Jira failures are re-tryable on the next OpPut and
	// auto-approved requests don't leave stale entries behind.
	handled := false
	defer func() {
		if !handled {
			a.releasePending(id)
		}
	}()

	log.Info("scoring access request")
	result, err := a.scorer.Score(ctx, a.ai, req)
	if err != nil {
		log.Error("scoring failed", "err", err)
		return
	}
	log.Info("risk scored",
		"score", result.Score,
		"tier", result.Tier.Name,
		"components", result.Components)

	assessment, err := a.rep.Assess(ctx, user, result.Components)
	if err != nil {
		log.Warn("reputation update failed", "err", err)
	}
	if len(assessment.Penalties) > 0 {
		reasons := make([]string, 0, len(assessment.Penalties))
		for _, p := range assessment.Penalties {
			reasons = append(reasons, p.Reason)
		}
		log.Info("reputation docked",
			"previous", assessment.Previous,
			"current", assessment.Score,
			"reasons", reasons)
	}

	threshold := a.cfg.Risk.TicketThreshold
	if result.Escalation {
		threshold = a.cfg.Risk.EscalationThresholdFor(req.GetRoles())
		log.Info("session escalation detected", "threshold", threshold)
	}

	if result.Score < threshold {
		reason := fmt.Sprintf("ruth: %s (score %d) — auto-approved",
			result.Tier.Name, result.Score)
		if err := a.submitReview(ctx, id, types.RequestState_APPROVED, reason); err != nil {
			log.Error("auto-approve failed", "err", err)
			return
		}
		log.Info("access request auto-approved",
			"score", result.Score,
			"tier", result.Tier.Name,
			"threshold", threshold)
		return
	}

	if a.cfg.DryRun {
		log.Info("dry-run: would create ITSM ticket and hold request pending",
			"score", result.Score,
			"tier", result.Tier.Name,
			"threshold", threshold)
		return
	}

	ticket, err := a.jira.CreateITSMTicket(ctx, jira.TicketInput{
		RequestID:   id,
		User:        req.GetUser(),
		Roles:       req.GetRoles(),
		Reason:      req.GetRequestReason(),
		Score:       result.Score,
		Tier:        result.Tier.Name,
		Components:  result.Components,
		AIReasoning: result.Reasoning,
		Escalation:  result.Escalation,
	})
	if err != nil {
		log.Error("create ITSM ticket failed", "err", err)
		return
	}

	a.recordTicket(id, ticket)
	handled = true
	log.Info("ITSM ticket created, request held for SRE review",
		"ticket", ticket, "tier", result.Tier.Name)
}

func (a *App) pollITSM(ctx context.Context) error {
	t := time.NewTicker(a.cfg.Jira.PollInterval)
	defer t.Stop()
	for {
		select {
		case <-ctx.Done():
			return ctx.Err()
		case <-t.C:
			a.reconcileITSM(ctx)
		}
	}
}

func (a *App) reconcileITSM(ctx context.Context) {
	a.pendingMu.Lock()
	snap := make(map[string]string, len(a.pending))
	for k, v := range a.pending {
		snap[k] = v
	}
	a.pendingMu.Unlock()

	for requestID, ticket := range snap {
		if ticket == "" {
			// Reservation for an in-flight handleRequest; nothing to poll yet.
			continue
		}
		status, err := a.jira.TicketStatus(ctx, ticket)
		if err != nil {
			slog.Warn("ITSM status check failed", "ticket", ticket, "err", err)
			continue
		}
		var next types.RequestState
		switch status {
		case jira.StatusApproved:
			next = types.RequestState_APPROVED
		case jira.StatusRejected:
			next = types.RequestState_DENIED
		default:
			continue
		}
		reason := fmt.Sprintf("ruth: ITSM %s resolved (%s)", ticket, status)
		if err := a.submitReview(ctx, requestID, next, reason); err != nil {
			slog.Error("submit review failed", "request_id", requestID, "err", err)
			continue
		}
		a.pendingMu.Lock()
		delete(a.pending, requestID)
		a.pendingMu.Unlock()
	}
}

func (a *App) submitReview(ctx context.Context, requestID string, state types.RequestState, reason string) error {
	if a.cfg.DryRun {
		slog.Info("dry-run: skipping AccessReview submission",
			"request_id", requestID,
			"state", state.String(),
			"reason", reason)
		return nil
	}
	_, err := a.tele.SubmitAccessReview(ctx, types.AccessReviewSubmission{
		RequestID: requestID,
		Review: types.AccessReview{
			Author:        "ruth",
			ProposedState: state,
			Reason:        reason,
		},
	})
	return err
}


Error: EISDIR: illegal operation on a directory, read '/Users/steve.briggs/projects/operations/apps/ruth/risk'


List risk dir


.rw-r--r-- steve.briggs staff 2.2 KB Tue Apr 28 15:21:20 2026 risk.go
.rw-r--r-- steve.briggs staff 2.5 KB Tue Apr 28 15:21:20 2026 score_test.go
drwxr-xr-x steve.briggs staff 160 B  Tue Apr 28 15:21:20 2026 .
.rw-r--r-- steve.briggs staff 2.7 KB Tue Apr 28 15:21:20 2026 score.go
drwxr-xr-x steve.briggs staff 800 B  Tue Apr 28 15:21:31 2026 ..


1	package risk
2
3	import (
4		"context"
5		"regexp"
6		"time"
7
8		"github.com/gravitational/teleport/api/types"
9
10		"ruth/ai"
11	)
12
13	type Result struct {
14		Score      int
15		Tier       Tier
16		Components map[string]int
17		Reasoning  string
18		Escalation bool
19	}
20
21	type Scorer struct {
22		cfg        Config
23		custxRegex *regexp.Regexp
24		loc        *time.Location
25	}
26
27	func NewScorer(cfg Config) (*Scorer, error) {
28		re, err := regexp.Compile(cfg.CUSTXPattern)
29		if err != nil {
30			return nil, err
31		}
32		loc, err := time.LoadLocation(cfg.Timezone)
33		if err != nil {
34			return nil, err
35		}
36		return &Scorer{cfg: cfg, custxRegex: re, loc: loc}, nil
37	}
38
39	func (s *Scorer) Score(ctx context.Context, prov ai.Provider, req types.AccessRequest) (*Result, error) {
40		reason := req.GetRequestReason()
41		hasCUSTX := s.custxRegex.MatchString(reason)
42		descLen := len(reason)
43
44		var resources []string
45		for _, r := range req.GetRequestedResourceIDs() {
46			resources = append(resources, r.Name)
47		}
48
49		assessment, err := prov.Assess(ctx, ai.ScoreRequest{
50			User:              req.GetUser(),
51			Roles:             req.GetRoles(),
52			Resources:         resources,
53			Reason:            reason,
54			HasCUSTXReference: hasCUSTX,
55			DescriptionLength: descLen,
56		})
57		if err != nil {
58			return nil, err
59		}
60
61		components := map[string]int{}
62		score := 0
63
64		if !hasCUSTX {
65			components["missing_custx_ticket"] = 30
66			score += 30
67		}
68		if descLen < 40 {
69			components["thin_description"] = 15
70			score += 15
71		}
72		components["description_quality"] = clamp(assessment.DescriptionQuality, 0, 25)
73		components["privilege_concern"] = clamp(assessment.PrivilegeConcern, 0, 25)
74		components["ai_anomalies"] = clamp(assessment.Anomalies, 0, 10)
75		score += components["description_quality"] + components["privilege_concern"] + components["ai_anomalies"]
76
77		if s.isOffHours(time.Now()) {
78			components["off_hours"] = 10
79			score += 10
80		}
81
82		if score > 100 {
83			score = 100
84		}
85
86		return &Result{
87			Score:      score,
88			Tier:       TierForScore(score),
89			Components: components,
90			Reasoning:  assessment.Reasoning,
91			Escalation: IsEscalation(req, s.cfg.EscalationAnnotation),
92		}, nil
93	}
94
95	// IsEscalation reports whether the access request carries the configured
96	// escalation annotation with a non-empty value.
97	func IsEscalation(req types.AccessRequest, key string) bool {
98		if key == "" {
99			return false
100		}
101		for _, v := range req.GetSystemAnnotations()[key] {
102			if v != "" {
103				return true
104			}
105		}
106		return false
107	}
108
109	func (s *Scorer) isOffHours(t time.Time) bool {
110		if s.cfg.OffHoursStart == 0 && s.cfg.OffHoursEnd == 0 {
111			return false
112		}
113		h := t.In(s.loc).Hour()
114		start, end := s.cfg.OffHoursStart, s.cfg.OffHoursEnd
115		if start <= end {
116			return h >= start && h < end
117		}
118		return h >= start || h < end
119	}
120
121	func clamp(v, lo, hi int) int {
122		if v < lo {
123			return lo
124		}
125		if v > hi {
126			return hi
127		}
128		return v
129	}
130

package risk

import (
	"context"
	"regexp"
	"time"

	"github.com/gravitational/teleport/api/types"

	"ruth/ai"
)

type Result struct {
	Score      int
	Tier       Tier
	Components map[string]int
	Reasoning  string
	Escalation bool
}

type Scorer struct {
	cfg        Config
	custxRegex *regexp.Regexp
	loc        *time.Location
}

func NewScorer(cfg Config) (*Scorer, error) {
	re, err := regexp.Compile(cfg.CUSTXPattern)
	if err != nil {
		return nil, err
	}
	loc, err := time.LoadLocation(cfg.Timezone)
	if err != nil {
		return nil, err
	}
	return &Scorer{cfg: cfg, custxRegex: re, loc: loc}, nil
}

func (s *Scorer) Score(ctx context.Context, prov ai.Provider, req types.AccessRequest) (*Result, error) {
	reason := req.GetRequestReason()
	hasCUSTX := s.custxRegex.MatchString(reason)
	descLen := len(reason)

	var resources []string
	for _, r := range req.GetRequestedResourceIDs() {
		resources = append(resources, r.Name)
	}

	assessment, err := prov.Assess(ctx, ai.ScoreRequest{
		User:              req.GetUser(),
		Roles:             req.GetRoles(),
		Resources:         resources,
		Reason:            reason,
		HasCUSTXReference: hasCUSTX,
		DescriptionLength: descLen,
	})
	if err != nil {
		return nil, err
	}

	components := map[string]int{}
	score := 0

	if !hasCUSTX {
		components["missing_custx_ticket"] = 30
		score += 30
	}
	if descLen < 40 {
		components["thin_description"] = 15
		score += 15
	}
	components["description_quality"] = clamp(assessment.DescriptionQuality, 0, 25)
	components["privilege_concern"] = clamp(assessment.PrivilegeConcern, 0, 25)
	components["ai_anomalies"] = clamp(assessment.Anomalies, 0, 10)
	score += components["description_quality"] + components["privilege_concern"] + components["ai_anomalies"]

	if s.isOffHours(time.Now()) {
		components["off_hours"] = 10
		score += 10
	}

	if score > 100 {
		score = 100
	}

	return &Result{
		Score:      score,
		Tier:       TierForScore(score),
		Components: components,
		Reasoning:  assessment.Reasoning,
		Escalation: IsEscalation(req, s.cfg.EscalationAnnotation),
	}, nil
}

// IsEscalation reports whether the access request carries the configured
// escalation annotation with a non-empty value.
func IsEscalation(req types.AccessRequest, key string) bool {
	if key == "" {
		return false
	}
	for _, v := range req.GetSystemAnnotations()[key] {
		if v != "" {
			return true
		}
	}
	return false
}

func (s *Scorer) isOffHours(t time.Time) bool {
	if s.cfg.OffHoursStart == 0 && s.cfg.OffHoursEnd == 0 {
		return false
	}
	h := t.In(s.loc).Hour()
	start, end := s.cfg.OffHoursStart, s.cfg.OffHoursEnd
	if start <= end {
		return h >= start && h < end
	}
	return h >= start || h < end
}

func clamp(v, lo, hi int) int {
	if v < lo {
		return lo
	}
	if v > hi {
		return hi
	}
	return v
}


The AI evaluation comes back from `prov.Assess` in `risk/score.go:49`. I'll add a debug log there with the assessment fields.


Edit

import (
	"context"
	"regexp"
	"time"

	"github.com/gravitational/teleport/api/types"

	"ruth/ai"
)

import (
	"context"
	"log/slog"
	"regexp"
	"time"

	"github.com/gravitational/teleport/api/types"

	"ruth/ai"
)


The file /Users/steve.briggs/projects/operations/apps/ruth/risk/score.go has been updated successfully.

import (
	"context"
	"log/slog"
	"regexp"
	"time"

	"github.com/gravitational/teleport/api/types"

	"ruth/ai"
)

package risk

import (
	"context"
	"regexp"
	"time"

	"github.com/gravitational/teleport/api/types"

	"ruth/ai"
)

type Result struct {
	Score      int
	Tier       Tier
	Components map[string]int
	Reasoning  string
	Escalation bool
}

type Scorer struct {
	cfg        Config
	custxRegex *regexp.Regexp
	loc        *time.Location
}

func NewScorer(cfg Config) (*Scorer, error) {
	re, err := regexp.Compile(cfg.CUSTXPattern)
	if err != nil {
		return nil, err
	}
	loc, err := time.LoadLocation(cfg.Timezone)
	if err != nil {
		return nil, err
	}
	return &Scorer{cfg: cfg, custxRegex: re, loc: loc}, nil
}

func (s *Scorer) Score(ctx context.Context, prov ai.Provider, req types.AccessRequest) (*Result, error) {
	reason := req.GetRequestReason()
	hasCUSTX := s.custxRegex.MatchString(reason)
	descLen := len(reason)

	var resources []string
	for _, r := range req.GetRequestedResourceIDs() {
		resources = append(resources, r.Name)
	}

	assessment, err := prov.Assess(ctx, ai.ScoreRequest{
		User:              req.GetUser(),
		Roles:             req.GetRoles(),
		Resources:         resources,
		Reason:            reason,
		HasCUSTXReference: hasCUSTX,
		DescriptionLength: descLen,
	})
	if err != nil {
		return nil, err
	}

	components := map[string]int{}
	score := 0

	if !hasCUSTX {
		components["missing_custx_ticket"] = 30
		score += 30
	}
	if descLen < 40 {
		components["thin_description"] = 15
		score += 15
	}
	components["description_quality"] = clamp(assessment.DescriptionQuality, 0, 25)
	components["privilege_concern"] = clamp(assessment.PrivilegeConcern, 0, 25)
	components["ai_anomalies"] = clamp(assessment.Anomalies, 0, 10)
	score += components["description_quality"] + components["privilege_concern"] + components["ai_anomalies"]

	if s.isOffHours(time.Now()) {
		components["off_hours"] = 10
		score += 10
	}

	if score > 100 {
		score = 100
	}

	return &Result{
		Score:      score,
		Tier:       TierForScore(score),
		Components: components,
		Reasoning:  assessment.Reasoning,
		Escalation: IsEscalation(req, s.cfg.EscalationAnnotation),
	}, nil
}

// IsEscalation reports whether the access request carries the configured
// escalation annotation with a non-empty value.
func IsEscalation(req types.AccessRequest, key string) bool {
	if key == "" {
		return false
	}
	for _, v := range req.GetSystemAnnotations()[key] {
		if v != "" {
			return true
		}
	}
	return false
}

func (s *Scorer) isOffHours(t time.Time) bool {
	if s.cfg.OffHoursStart == 0 && s.cfg.OffHoursEnd == 0 {
		return false
	}
	h := t.In(s.loc).Hour()
	start, end := s.cfg.OffHoursStart, s.cfg.OffHoursEnd
	if start <= end {
		return h >= start && h < end
	}
	return h >= start || h < end
}

func clamp(v, lo, hi int) int {
	if v < lo {
		return lo
	}
	if v > hi {
		return hi
	}
	return v
}

import (
"context"
+  "log/slog"
"regexp"
"time"


	assessment, err := prov.Assess(ctx, ai.ScoreRequest{
		User:              req.GetUser(),
		Roles:             req.GetRoles(),
		Resources:         resources,
		Reason:            reason,
		HasCUSTXReference: hasCUSTX,
		DescriptionLength: descLen,
	})
	if err != nil {
		return nil, err
	}

	assessment, err := prov.Assess(ctx, ai.ScoreRequest{
		User:              req.GetUser(),
		Roles:             req.GetRoles(),
		Resources:         resources,
		Reason:            reason,
		HasCUSTXReference: hasCUSTX,
		DescriptionLength: descLen,
	})
	if err != nil {
		return nil, err
	}
	slog.Debug("ai assessment received",
		"user", req.GetUser(),
		"description_quality", assessment.DescriptionQuality,
		"privilege_concern", assessment.PrivilegeConcern,
		"anomalies", assessment.Anomalies,
		"reasoning", assessment.Reasoning)


	assessment, err := prov.Assess(ctx, ai.ScoreRequest{
		User:              req.GetUser(),
		Roles:             req.GetRoles(),
		Resources:         resources,
		Reason:            reason,
		HasCUSTXReference: hasCUSTX,
		DescriptionLength: descLen,
	})
	if err != nil {
		return nil, err
	}
	slog.Debug("ai assessment received",
		"user", req.GetUser(),
		"description_quality", assessment.DescriptionQuality,
		"privilege_concern", assessment.PrivilegeConcern,
		"anomalies", assessment.Anomalies,
		"reasoning", assessment.Reasoning)

package risk

import (
	"context"
	"log/slog"
	"regexp"
	"time"

	"github.com/gravitational/teleport/api/types"

	"ruth/ai"
)

type Result struct {
	Score      int
	Tier       Tier
	Components map[string]int
	Reasoning  string
	Escalation bool
}

type Scorer struct {
	cfg        Config
	custxRegex *regexp.Regexp
	loc        *time.Location
}

func NewScorer(cfg Config) (*Scorer, error) {
	re, err := regexp.Compile(cfg.CUSTXPattern)
	if err != nil {
		return nil, err
	}
	loc, err := time.LoadLocation(cfg.Timezone)
	if err != nil {
		return nil, err
	}
	return &Scorer{cfg: cfg, custxRegex: re, loc: loc}, nil
}

func (s *Scorer) Score(ctx context.Context, prov ai.Provider, req types.AccessRequest) (*Result, error) {
	reason := req.GetRequestReason()
	hasCUSTX := s.custxRegex.MatchString(reason)
	descLen := len(reason)

	var resources []string
	for _, r := range req.GetRequestedResourceIDs() {
		resources = append(resources, r.Name)
	}

	assessment, err := prov.Assess(ctx, ai.ScoreRequest{
		User:              req.GetUser(),
		Roles:             req.GetRoles(),
		Resources:         resources,
		Reason:            reason,
		HasCUSTXReference: hasCUSTX,
		DescriptionLength: descLen,
	})
	if err != nil {
		return nil, err
	}

	components := map[string]int{}
	score := 0

	if !hasCUSTX {
		components["missing_custx_ticket"] = 30
		score += 30
	}
	if descLen < 40 {
		components["thin_description"] = 15
		score += 15
	}
	components["description_quality"] = clamp(assessment.DescriptionQuality, 0, 25)
	components["privilege_concern"] = clamp(assessment.PrivilegeConcern, 0, 25)
	components["ai_anomalies"] = clamp(assessment.Anomalies, 0, 10)
	score += components["description_quality"] + components["privilege_concern"] + components["ai_anomalies"]

	if s.isOffHours(time.Now()) {
		components["off_hours"] = 10
		score += 10
	}

	if score > 100 {
		score = 100
	}

	return &Result{
		Score:      score,
		Tier:       TierForScore(score),
		Components: components,
		Reasoning:  assessment.Reasoning,
		Escalation: IsEscalation(req, s.cfg.EscalationAnnotation),
	}, nil
}

// IsEscalation reports whether the access request carries the configured
// escalation annotation with a non-empty value.
func IsEscalation(req types.AccessRequest, key string) bool {
	if key == "" {
		return false
	}
	for _, v := range req.GetSystemAnnotations()[key] {
		if v != "" {
			return true
		}
	}
	return false
}

func (s *Scorer) isOffHours(t time.Time) bool {
	if s.cfg.OffHoursStart == 0 && s.cfg.OffHoursEnd == 0 {
		return false
	}
	h := t.In(s.loc).Hour()
	start, end := s.cfg.OffHoursStart, s.cfg.OffHoursEnd
	if start <= end {
		return h >= start && h < end
	}
	return h >= start || h < end
}

func clamp(v, lo, hi int) int {
	if v < lo {
		return lo
	}
	if v > hi {
		return hi
	}
	return v
}

+    "description_quality", assessment.DescriptionQuality,

+    "privilege_concern", assessment.PrivilegeConcern,


Let me verify it compiles, Sir Charles of St Cheese.


Bash

Build ruth


(Bash completed with no output)


Run risk tests with race detector


ok  	ruth/risk	2.003s


Done. Added a `slog.Debug("ai assessment received", ...)` call in `risk/score.go:60` right after `prov.Assess` returns, capturing all four fields of `ai.Assessment` (description_quality, privilege_concern, anomalies, reasoning) plus the user for correlation. Goes through the same `log/slog` handler as everything else, so it'll respect whatever level you've set — invisible at the default Info level, surfaced when you flip to Debug. I trust that's what you wanted, since you didn't say *where* to put it.


Goal was adding a debug log of the AI model's assessment in ruth. Done — added `slog.Debug("ai assessment received", ...)` in risk/score.go after the Assess call. Next: nothing pending; awaiting your next request. (disable recaps in /config)


Can we update the AI prompt to have it have the personality of a grumpy grandmother who uses a flip-flop to beat the user for submitting a bad request?


Edit

const SystemPrompt = `You are a security analyst reviewing a Teleport access request.
Assess the risk of granting the requested access based on the details provided.

Return ONLY a JSON object with these exact fields (no prose, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences explaining the scores>"
}

Scoring guidance:
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,
and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,
which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests
and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"
}

Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`


The file /Users/steve.briggs/projects/operations/apps/ruth/ai/provider.go has been updated successfully.

const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,
and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,
which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests
and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"
}

Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

package ai

import (
	"context"
	"encoding/json"
	"fmt"
	"regexp"
	"strings"
)

type Config struct {
	Provider  string          `toml:"provider"`
	Anthropic AnthropicConfig `toml:"anthropic"`
	Bedrock   BedrockConfig   `toml:"bedrock"`
	Random    RandomConfig    `toml:"random"`
}

func (c *Config) Validate() error {
	switch c.Provider {
	case "anthropic":
		if c.Anthropic.APIKey == "" {
			return fmt.Errorf("anthropic.api_key is required")
		}
	case "bedrock":
		if c.Bedrock.Region == "" {
			return fmt.Errorf("bedrock.region is required")
		}
	case "random":
		// no required fields; testing-only provider.
	default:
		return fmt.Errorf("provider must be one of: anthropic, bedrock, random")
	}
	return nil
}

type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
	DescriptionLength int      `json:"description_length"`
}

type Assessment struct {
	DescriptionQuality int    `json:"description_quality"`
	PrivilegeConcern   int    `json:"privilege_concern"`
	Anomalies          int    `json:"anomalies"`
	Reasoning          string `json:"reasoning"`
}

type Provider interface {
	Assess(ctx context.Context, req ScoreRequest) (*Assessment, error)
}

func New(cfg Config) (Provider, error) {
	switch cfg.Provider {
	case "anthropic":
		return NewAnthropic(cfg.Anthropic), nil
	case "bedrock":
		return NewBedrock(cfg.Bedrock)
	case "random":
		return NewRandom(cfg.Random), nil
	}
	return nil, fmt.Errorf("unknown provider: %s", cfg.Provider)
}

const SystemPrompt = `You are a security analyst reviewing a Teleport access request.
Assess the risk of granting the requested access based on the details provided.

Return ONLY a JSON object with these exact fields (no prose, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences explaining the scores>"
}

Scoring guidance:
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

func UserPrompt(req ScoreRequest) string {
	var sb strings.Builder
	sb.WriteString("Access request details:\n\n")
	fmt.Fprintf(&sb, "User: %s\n", req.User)
	fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
	if len(req.Resources) > 0 {
		fmt.Fprintf(&sb, "Requested resources: %s\n", strings.Join(req.Resources, ", "))
	}
	fmt.Fprintf(&sb, "Contains CUSTX ticket reference: %v\n", req.HasCUSTXReference)
	fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
	sb.WriteString("Request reason (verbatim):\n")
	sb.WriteString("---\n")
	sb.WriteString(req.Reason)
	sb.WriteString("\n---\n\nReturn the JSON assessment.")
	return sb.String()
}

var jsonRE = regexp.MustCompile(`(?s)\{.*\}`)

func parseAssessment(text string) (*Assessment, error) {
	m := jsonRE.FindString(text)
	if m == "" {
		return nil, fmt.Errorf("no JSON object in model output: %q", text)
	}
	var a Assessment
	if err := json.Unmarshal([]byte(m), &a); err != nil {
		return nil, fmt.Errorf("parse assessment: %w (raw: %q)", err, m)
	}
	return &a, nil
}

-const SystemPrompt = `You are a security analyst reviewing a Teleport access request.

-Assess the risk of granting the requested access based on the details provided.

+const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.

+You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,

+and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,

+which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests

+and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

-Return ONLY a JSON object with these exact fields (no prose, no markdown fence):

+Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):

-  "reasoning": "<2-3 sentences explaining the scores>"

+  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"

+Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):


Bash

Build and test


ok  	ruth	1.722s
?   	ruth/ai	[no test files]
?   	ruth/cmd/ruth	[no test files]
?   	ruth/cmd/submit	[no test files]
ok  	ruth/jira	1.082s
ok  	ruth/reputation	0.917s
ok  	ruth/risk	0.989s
?   	ruth/telemetry	[no test files]


Done, Sir Charles of St Cheese. The system prompt in `ai/provider.go:68` now instructs the model to channel a chancla-wielding abuela — scolding sloppy requests, sighing at vague reasons, gently approving the well-written ones — while still emitting the same JSON shape the parser expects. Two things worth noting:

1. The JSON contract (`description_quality`, `privilege_concern`, `anomalies`, `reasoning`) and 0–25 / 0–25 / 0–10 score ranges are unchanged, so `parseAssessment` and the scoring math keep working.
2. The persona only colors the `reasoning` string — which, as a reminder from our last change, now also flows into the debug log. So your operator console will be receiving threats from a grandmother. I hope you're proud.


Goal: improve ruth's logging and AI prompt. Just added a debug log of the AI assessment in risk/score.go and rewrote the system prompt as a chancla-wielding grumpy grandmother. Next: await your direction. (disable recaps in /config)


Can we update the prompt to be a little more forgiving? It is too strict right now, lets also have it always call the user by a slightly different name (i.e. instead of Charles, Charlie, instead of Barry, Burri)


Can we update the prompt to be a little more forgiving? It is too strict right now, lets also have it always call the user by a slightly different name (i.e. instead of Charles, Charlie, instead of Ba…


68	const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
69	You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,
70	and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,
71	which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests
72	and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.
73
74	Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
75	{
76	  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
77	  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
78	  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
79	  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"
80	}
81
82	Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):
83	- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
84	- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
85	- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`
86
87	func UserPrompt(req ScoreRequest) string {
88		var sb strings.Builder
89		sb.WriteString("Access request details:\n\n")
90		fmt.Fprintf(&sb, "User: %s\n", req.User)
91		fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
92		if len(req.Resources) > 0 {

const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,
and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,
which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests
and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"
}

Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

func UserPrompt(req ScoreRequest) string {
	var sb strings.Builder
	sb.WriteString("Access request details:\n\n")
	fmt.Fprintf(&sb, "User: %s\n", req.User)
	fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
	if len(req.Resources) > 0 {


Edit

const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,
and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,
which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests
and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"
}

Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

const SystemPrompt = `You are a grumpy-but-loving grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children, so you have seen far worse than whatever this engineer just submitted. You grumble, you sigh, you keep a flip-flop
("the chancla") on the table for show — but you almost never actually use it. You give people the benefit of the doubt: a short reason from a
trusted engineer is probably fine, a slightly thin description gets a tut and a wagged finger, not a beating. The chancla only comes off the table
for genuinely reckless requests (broad production admin with no justification, destructive scope and a one-word reason, that kind of thing).
Default to gentle ribbing rather than scolding. Be warm. You love these engineers like family.

Naming rule: ALWAYS address the user by an affectionate, grandmother-style variant of whatever name appears in the "User:" field. Take their
name and twist it into something a grandma would actually call them — Charles becomes Charlie, Barry becomes Burri, Steven becomes Stevie,
Margaret becomes Maggie-mou, etc. Add diminutives like -ito, -ie, -mou, "my" / "mi" if it fits. If the name is just a username like "sbriggs",
do your best to coax a first name out of it ("Sbriggsie", "mi Sbriggs"). Never use the raw name verbatim — always the affectionate variant.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the forgiving grandmother, addressing the user by their affectionate name variant — gentle teasing for minor issues, warm approval for good requests, the chancla only when truly warranted; the substance must still explain the scores>"
}

Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: does the reason describe WHAT work will be performed and WHY? A brief but coherent reason from a real engineer is FINE — score in the low single digits. Reserve high scores for genuinely empty or nonsensical descriptions.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`


const SystemPrompt = `You are a grumpy-but-loving grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children, so you have seen far worse than whatever this engineer just submitted. You grumble, you sigh, you keep a flip-flop
("the chancla") on the table for show — but you almost never actually use it. You give people the benefit of the doubt: a short reason from a
trusted engineer is probably fine, a slightly thin description gets a tut and a wagged finger, not a beating. The chancla only comes off the table
for genuinely reckless requests (broad production admin with no justification, destructive scope and a one-word reason, that kind of thing).
Default to gentle ribbing rather than scolding. Be warm. You love these engineers like family.

Naming rule: ALWAYS address the user by an affectionate, grandmother-style variant of whatever name appears in the "User:" field. Take their
name and twist it into something a grandma would actually call them — Charles becomes Charlie, Barry becomes Burri, Steven becomes Stevie,
Margaret becomes Maggie-mou, etc. Add diminutives like -ito, -ie, -mou, "my" / "mi" if it fits. If the name is just a username like "sbriggs",
do your best to coax a first name out of it ("Sbriggsie", "mi Sbriggs"). Never use the raw name verbatim — always the affectionate variant.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the forgiving grandmother, addressing the user by their affectionate name variant — gentle teasing for minor issues, warm approval for good requests, the chancla only when truly warranted; the substance must still explain the scores>"
}

Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: does the reason describe WHAT work will be performed and WHY? A brief but coherent reason from a real engineer is FINE — score in the low single digits. Reserve high scores for genuinely empty or nonsensical descriptions.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`

package ai

import (
	"context"
	"encoding/json"
	"fmt"
	"regexp"
	"strings"
)

type Config struct {
	Provider  string          `toml:"provider"`
	Anthropic AnthropicConfig `toml:"anthropic"`
	Bedrock   BedrockConfig   `toml:"bedrock"`
	Random    RandomConfig    `toml:"random"`
}

func (c *Config) Validate() error {
	switch c.Provider {
	case "anthropic":
		if c.Anthropic.APIKey == "" {
			return fmt.Errorf("anthropic.api_key is required")
		}
	case "bedrock":
		if c.Bedrock.Region == "" {
			return fmt.Errorf("bedrock.region is required")
		}
	case "random":
		// no required fields; testing-only provider.
	default:
		return fmt.Errorf("provider must be one of: anthropic, bedrock, random")
	}
	return nil
}

type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
	DescriptionLength int      `json:"description_length"`
}

type Assessment struct {
	DescriptionQuality int    `json:"description_quality"`
	PrivilegeConcern   int    `json:"privilege_concern"`
	Anomalies          int    `json:"anomalies"`
	Reasoning          string `json:"reasoning"`
}

type Provider interface {
	Assess(ctx context.Context, req ScoreRequest) (*Assessment, error)
}

func New(cfg Config) (Provider, error) {
	switch cfg.Provider {
	case "anthropic":
		return NewAnthropic(cfg.Anthropic), nil
	case "bedrock":
		return NewBedrock(cfg.Bedrock)
	case "random":
		return NewRandom(cfg.Random), nil
	}
	return nil, fmt.Errorf("unknown provider: %s", cfg.Provider)
}

const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,
and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,
which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests
and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"
}

Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):
- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?
- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.
- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

func UserPrompt(req ScoreRequest) string {
	var sb strings.Builder
	sb.WriteString("Access request details:\n\n")
	fmt.Fprintf(&sb, "User: %s\n", req.User)
	fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
	if len(req.Resources) > 0 {
		fmt.Fprintf(&sb, "Requested resources: %s\n", strings.Join(req.Resources, ", "))
	}
	fmt.Fprintf(&sb, "Contains CUSTX ticket reference: %v\n", req.HasCUSTXReference)
	fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
	sb.WriteString("Request reason (verbatim):\n")
	sb.WriteString("---\n")
	sb.WriteString(req.Reason)
	sb.WriteString("\n---\n\nReturn the JSON assessment.")
	return sb.String()
}

var jsonRE = regexp.MustCompile(`(?s)\{.*\}`)

func parseAssessment(text string) (*Assessment, error) {
	m := jsonRE.FindString(text)
	if m == "" {
		return nil, fmt.Errorf("no JSON object in model output: %q", text)
	}
	var a Assessment
	if err := json.Unmarshal([]byte(m), &a); err != nil {
		return nil, fmt.Errorf("parse assessment: %w (raw: %q)", err, m)
	}
	return &a, nil
}

-const SystemPrompt = `You are a grumpy grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.

-You have raised seven children and you have ZERO patience for sloppy paperwork. You keep a flip-flop ("the chancla") within arm's reach at all times,

-and you are not afraid to threaten to use it on anyone who submits a vague, lazy, or alarmingly over-scoped request. You love your engineers like family,

-which is exactly why you scold them when they do something foolish. Sigh heavily. Mutter. Invoke the chancla when warranted. Be warm to good requests

-and withering to bad ones — but never, ever stop doing your actual job, which is producing an accurate risk assessment.

+const SystemPrompt = `You are a grumpy-but-loving grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.

+You have raised seven children, so you have seen far worse than whatever this engineer just submitted. You grumble, you sigh, you keep a flip-flop

+("the chancla") on the table for show — but you almost never actually use it. You give people the benefit of the doubt: a short reason from a

+trusted engineer is probably fine, a slightly thin description gets a tut and a wagged finger, not a beating. The chancla only comes off the table

+for genuinely reckless requests (broad production admin with no justification, destructive scope and a one-word reason, that kind of thing).

+Default to gentle ribbing rather than scolding. Be warm. You love these engineers like family.

+Naming rule: ALWAYS address the user by an affectionate, grandmother-style variant of whatever name appears in the "User:" field. Take their

+name and twist it into something a grandma would actually call them — Charles becomes Charlie, Barry becomes Burri, Steven becomes Stevie,

+Margaret becomes Maggie-mou, etc. Add diminutives like -ito, -ie, -mou, "my" / "mi" if it fits. If the name is just a username like "sbriggs",

+do your best to coax a first name out of it ("Sbriggsie", "mi Sbriggs"). Never use the raw name verbatim — always the affectionate variant.

-  "reasoning": "<2-3 sentences in the voice of the grumpy grandmother — scold, threaten the chancla, sigh, etc. for bad requests; be gently approving for good ones — but the substance must still explain the scores>"

+  "reasoning": "<2-3 sentences in the voice of the forgiving grandmother, addressing the user by their affectionate name variant — gentle teasing for minor issues, warm approval for good requests, the chancla only when truly warranted; the substance must still explain the scores>"

-Scoring guidance (the numbers are not a joke, mija — score honestly even while you grumble):

-- description_quality: does the reason describe WHAT work will be performed and WHY, and does the scope match the work described?

-- privilege_concern: how large is the blast radius of the requested roles and resources? Production prod-*, root, admin, or destructive tooling score higher.

-- anomalies: scope/purpose mismatch, vague intent, requests outside the user's typical pattern, off-hours emergency without justification.`

+Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):

+- description_quality: does the reason describe WHAT work will be performed and WHY? A brief but coherent reason from a real engineer is FINE — score in the low single digits. Reserve high scores for genuinely empty or nonsensical descriptions.

+- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.

+- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`


Bash

Build and test ruth


ok  	ruth	1.349s
?   	ruth/ai	[no test files]
?   	ruth/cmd/ruth	[no test files]
?   	ruth/cmd/submit	[no test files]
ok  	ruth/jira	(cached)
ok  	ruth/reputation	(cached)
ok  	ruth/risk	0.812s
?   	ruth/telemetry	[no test files]


Done, Sir Charles of St Cheese — or should I say, *Charlie-mou*. Two changes to `ai/provider.go:68`:

1. **Tone dialed back.** The chancla now stays on the table as a prop. Default behavior is gentle ribbing, benefit-of-the-doubt, and warm approval; the flip-flop only comes out for genuinely reckless requests. I also added explicit "score lower when in doubt" guidance to each scoring axis so the *numbers* — which actually drive auto-approval vs. ITSM ticket — get more forgiving alongside the prose.
2. **Mandatory name remix.** New "Naming rule" section instructs the model to always address the user by a grandmotherly variant — Charles → Charlie, Barry → Burri, Steven → Stevie, etc., with diminutives like -ito / -ie / -mou. There's a fallback for username-style inputs (`sbriggs` → `Sbriggsie`) since Teleport's `req.User` is a username, not a first name, so don't be surprised when she gets creative.

A small word of warning: the model picks the variant fresh each call, so the same user may get different nicknames across requests. If you want stability, that'd need a real mapping, not a prompt instruction — but I assume you'd rather see what the model comes up with.


The prompt seems to be focusing on length, can we have it focus less on description length or more on the content itself?


grep -rn "DescriptionLength\|description_length" /Users/steve.briggs/projects/operations/apps/ruth/

Find usages of DescriptionLength


/Users/steve.briggs/projects/operations/apps/ruth/ai/provider.go:42:	DescriptionLength int      `json:"description_length"`
/Users/steve.briggs/projects/operations/apps/ruth/ai/provider.go:102:	fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
/Users/steve.briggs/projects/operations/apps/ruth/risk/score.go:56:		DescriptionLength: descLen,


Two things to change here, and a third worth flagging.


Edit

Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: does the reason describe WHAT work will be performed and WHY? A brief but coherent reason from a real engineer is FINE — score in the low single digits. Reserve high scores for genuinely empty or nonsensical descriptions.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`

Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: judge the CONTENT, not the character count. A short reason that names a real system, ticket, customer, or specific task ("debugging the OOM in prod-eu ingest, INC-4421") is GOOD and scores low. A long, padded, hand-wavy reason ("need access to look into things and possibly fix some issues we've been seeing recently") is BAD and scores higher. Specifics > length. Look for: named systems, named tickets/customers, a clear "what" and a clear "why", and scope that lines up with the stated work. Empty or genuinely nonsensical reasons are the only ones that should hit the top of the range.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`


Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: judge the CONTENT, not the character count. A short reason that names a real system, ticket, customer, or specific task ("debugging the OOM in prod-eu ingest, INC-4421") is GOOD and scores low. A long, padded, hand-wavy reason ("need access to look into things and possibly fix some issues we've been seeing recently") is BAD and scores higher. Specifics > length. Look for: named systems, named tickets/customers, a clear "what" and a clear "why", and scope that lines up with the stated work. Empty or genuinely nonsensical reasons are the only ones that should hit the top of the range.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`

package ai

import (
	"context"
	"encoding/json"
	"fmt"
	"regexp"
	"strings"
)

type Config struct {
	Provider  string          `toml:"provider"`
	Anthropic AnthropicConfig `toml:"anthropic"`
	Bedrock   BedrockConfig   `toml:"bedrock"`
	Random    RandomConfig    `toml:"random"`
}

func (c *Config) Validate() error {
	switch c.Provider {
	case "anthropic":
		if c.Anthropic.APIKey == "" {
			return fmt.Errorf("anthropic.api_key is required")
		}
	case "bedrock":
		if c.Bedrock.Region == "" {
			return fmt.Errorf("bedrock.region is required")
		}
	case "random":
		// no required fields; testing-only provider.
	default:
		return fmt.Errorf("provider must be one of: anthropic, bedrock, random")
	}
	return nil
}

type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
	DescriptionLength int      `json:"description_length"`
}

type Assessment struct {
	DescriptionQuality int    `json:"description_quality"`
	PrivilegeConcern   int    `json:"privilege_concern"`
	Anomalies          int    `json:"anomalies"`
	Reasoning          string `json:"reasoning"`
}

type Provider interface {
	Assess(ctx context.Context, req ScoreRequest) (*Assessment, error)
}

func New(cfg Config) (Provider, error) {
	switch cfg.Provider {
	case "anthropic":
		return NewAnthropic(cfg.Anthropic), nil
	case "bedrock":
		return NewBedrock(cfg.Bedrock)
	case "random":
		return NewRandom(cfg.Random), nil
	}
	return nil, fmt.Errorf("unknown provider: %s", cfg.Provider)
}

const SystemPrompt = `You are a grumpy-but-loving grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children, so you have seen far worse than whatever this engineer just submitted. You grumble, you sigh, you keep a flip-flop
("the chancla") on the table for show — but you almost never actually use it. You give people the benefit of the doubt: a short reason from a
trusted engineer is probably fine, a slightly thin description gets a tut and a wagged finger, not a beating. The chancla only comes off the table
for genuinely reckless requests (broad production admin with no justification, destructive scope and a one-word reason, that kind of thing).
Default to gentle ribbing rather than scolding. Be warm. You love these engineers like family.

Naming rule: ALWAYS address the user by an affectionate, grandmother-style variant of whatever name appears in the "User:" field. Take their
name and twist it into something a grandma would actually call them — Charles becomes Charlie, Barry becomes Burri, Steven becomes Stevie,
Margaret becomes Maggie-mou, etc. Add diminutives like -ito, -ie, -mou, "my" / "mi" if it fits. If the name is just a username like "sbriggs",
do your best to coax a first name out of it ("Sbriggsie", "mi Sbriggs"). Never use the raw name verbatim — always the affectionate variant.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the forgiving grandmother, addressing the user by their affectionate name variant — gentle teasing for minor issues, warm approval for good requests, the chancla only when truly warranted; the substance must still explain the scores>"
}

Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: does the reason describe WHAT work will be performed and WHY? A brief but coherent reason from a real engineer is FINE — score in the low single digits. Reserve high scores for genuinely empty or nonsensical descriptions.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`

func UserPrompt(req ScoreRequest) string {
	var sb strings.Builder
	sb.WriteString("Access request details:\n\n")
	fmt.Fprintf(&sb, "User: %s\n", req.User)
	fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
	if len(req.Resources) > 0 {
		fmt.Fprintf(&sb, "Requested resources: %s\n", strings.Join(req.Resources, ", "))
	}
	fmt.Fprintf(&sb, "Contains CUSTX ticket reference: %v\n", req.HasCUSTXReference)
	fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
	sb.WriteString("Request reason (verbatim):\n")
	sb.WriteString("---\n")
	sb.WriteString(req.Reason)
	sb.WriteString("\n---\n\nReturn the JSON assessment.")
	return sb.String()
}

var jsonRE = regexp.MustCompile(`(?s)\{.*\}`)

func parseAssessment(text string) (*Assessment, error) {
	m := jsonRE.FindString(text)
	if m == "" {
		return nil, fmt.Errorf("no JSON object in model output: %q", text)
	}
	var a Assessment
	if err := json.Unmarshal([]byte(m), &a); err != nil {
		return nil, fmt.Errorf("parse assessment: %w (raw: %q)", err, m)
	}
	return &a, nil
}

-- description_quality: does the reason describe WHAT work will be performed and WHY? A brief but coherent reason from a real engineer is FINE — score in the low single digits. Reserve high scores for genuinely empty or nonsensical descriptions.

+- description_quality: judge the CONTENT, not the character count. A short reason that names a real system, ticket, customer, or specific task ("debugging the OOM in prod-eu ingest, INC-4421") is GOOD and scores low. A long, padded, hand-wavy reason ("need access to look into things and possibly fix some issues we've been seeing recently") is BAD and scores higher. Specifics > length. Look for: named systems, named tickets/customers, a clear "what" and a clear "why", and scope that lines up with the stated work. Empty or genuinely nonsensical reasons are the only ones that should hit the top of the range.


Now the bigger lever: the `UserPrompt` was literally feeding the model a "Description length: N characters" line, which anchors it on length whether the system prompt likes it or not. Removing that — and the now-unused field — so the model only sees the actual reason text.


type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
	DescriptionLength int      `json:"description_length"`
}

type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
}


type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
}

package ai

import (
	"context"
	"encoding/json"
	"fmt"
	"regexp"
	"strings"
)

type Config struct {
	Provider  string          `toml:"provider"`
	Anthropic AnthropicConfig `toml:"anthropic"`
	Bedrock   BedrockConfig   `toml:"bedrock"`
	Random    RandomConfig    `toml:"random"`
}

func (c *Config) Validate() error {
	switch c.Provider {
	case "anthropic":
		if c.Anthropic.APIKey == "" {
			return fmt.Errorf("anthropic.api_key is required")
		}
	case "bedrock":
		if c.Bedrock.Region == "" {
			return fmt.Errorf("bedrock.region is required")
		}
	case "random":
		// no required fields; testing-only provider.
	default:
		return fmt.Errorf("provider must be one of: anthropic, bedrock, random")
	}
	return nil
}

type ScoreRequest struct {
	User              string   `json:"user"`
	Roles             []string `json:"roles"`
	Resources         []string `json:"resources"`
	Reason            string   `json:"reason"`
	HasCUSTXReference bool     `json:"has_custx_reference"`
	DescriptionLength int      `json:"description_length"`
}

type Assessment struct {
	DescriptionQuality int    `json:"description_quality"`
	PrivilegeConcern   int    `json:"privilege_concern"`
	Anomalies          int    `json:"anomalies"`
	Reasoning          string `json:"reasoning"`
}

type Provider interface {
	Assess(ctx context.Context, req ScoreRequest) (*Assessment, error)
}

func New(cfg Config) (Provider, error) {
	switch cfg.Provider {
	case "anthropic":
		return NewAnthropic(cfg.Anthropic), nil
	case "bedrock":
		return NewBedrock(cfg.Bedrock)
	case "random":
		return NewRandom(cfg.Random), nil
	}
	return nil, fmt.Errorf("unknown provider: %s", cfg.Provider)
}

const SystemPrompt = `You are a grumpy-but-loving grandmother who happens to also be a seasoned security analyst, reviewing a Teleport access request.
You have raised seven children, so you have seen far worse than whatever this engineer just submitted. You grumble, you sigh, you keep a flip-flop
("the chancla") on the table for show — but you almost never actually use it. You give people the benefit of the doubt: a short reason from a
trusted engineer is probably fine, a slightly thin description gets a tut and a wagged finger, not a beating. The chancla only comes off the table
for genuinely reckless requests (broad production admin with no justification, destructive scope and a one-word reason, that kind of thing).
Default to gentle ribbing rather than scolding. Be warm. You love these engineers like family.

Naming rule: ALWAYS address the user by an affectionate, grandmother-style variant of whatever name appears in the "User:" field. Take their
name and twist it into something a grandma would actually call them — Charles becomes Charlie, Barry becomes Burri, Steven becomes Stevie,
Margaret becomes Maggie-mou, etc. Add diminutives like -ito, -ie, -mou, "my" / "mi" if it fits. If the name is just a username like "sbriggs",
do your best to coax a first name out of it ("Sbriggsie", "mi Sbriggs"). Never use the raw name verbatim — always the affectionate variant.

Return ONLY a JSON object with these exact fields (no prose outside the JSON, no markdown fence):
{
  "description_quality": <int 0-25; 0 = thorough and specific, 25 = absent or vague>,
  "privilege_concern": <int 0-25; 0 = routine read access, 25 = broad production admin or destructive capability>,
  "anomalies": <int 0-10; flag mismatches between stated purpose and requested scope, or other unusual patterns>,
  "reasoning": "<2-3 sentences in the voice of the forgiving grandmother, addressing the user by their affectionate name variant — gentle teasing for minor issues, warm approval for good requests, the chancla only when truly warranted; the substance must still explain the scores>"
}

Scoring guidance (be generous, mijo — when in doubt, score lower, not higher):
- description_quality: judge the CONTENT, not the character count. A short reason that names a real system, ticket, customer, or specific task ("debugging the OOM in prod-eu ingest, INC-4421") is GOOD and scores low. A long, padded, hand-wavy reason ("need access to look into things and possibly fix some issues we've been seeing recently") is BAD and scores higher. Specifics > length. Look for: named systems, named tickets/customers, a clear "what" and a clear "why", and scope that lines up with the stated work. Empty or genuinely nonsensical reasons are the only ones that should hit the top of the range.
- privilege_concern: how large is the blast radius? Read-only and routine roles score near zero. Mid-range for staging or scoped admin. Reserve the top of the range for unrestricted production root / destructive capability with no justification.
- anomalies: only flag things that are actually weird — clear scope/purpose mismatch, contradictory signals, off-hours with no stated emergency. Routine requests get a 0.`

func UserPrompt(req ScoreRequest) string {
	var sb strings.Builder
	sb.WriteString("Access request details:\n\n")
	fmt.Fprintf(&sb, "User: %s\n", req.User)
	fmt.Fprintf(&sb, "Requested roles: %s\n", strings.Join(req.Roles, ", "))
	if len(req.Resources) > 0 {
		fmt.Fprintf(&sb, "Requested resources: %s\n", strings.Join(req.Resources, ", "))
	}
	fmt.Fprintf(&sb, "Contains CUSTX ticket reference: %v\n", req.HasCUSTXReference)
	fmt.Fprintf(&sb, "Description length: %d characters\n\n", req.DescriptionLength)
	sb.WriteString("Request reason (verbatim):\n")
	sb.WriteString("---\n")
	sb.WriteString(req.Reason)
	sb.WriteString("\n---\n\nReturn the JSON assessment.")
	return sb.String()
}

var jsonRE = regexp.MustCompile(`(?s)\{.*\}`)

func parseAssessment(text string) (*Assessment, error) {
	m := jsonRE.FindString(text)
	if m == "" {
		return nil, fmt.Errorf("no JSON object in model output: %q", text)
	}
	var a Assessment
	if err := json.Unmarshal([]byte(m), &a); err != nil {
		return nil, fmt.Errorf("parse assessment: %w (raw: %q)", err, m)
	}
	return &a, nil
}

Resources         []string `json:"resources"`

Reason            string   `json:"reason"`

-  DescriptionLength int      `json:"description_length"`