You are implementing Task A8: add the `redact_emails_metrics` remap transform to the Vector helm values template, plus a VRL unit test.

## What's already in place

- Working directory: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector`
- `templates/values.yaml.tpl` has sources, drop_audit, redact_emails_logs, redact_emails_traces, sinks. The `datadog_metrics` sink currently reads directly from `otlp_in.metrics`.
- `tests/vrl/{drop_audit,redact_emails_logs,redact_emails_traces}/` are all passing.
- `vector` 0.55.0 CLI installed at `/opt/homebrew/bin/vector`. `jq` installed.

## Vector 0.55 VRL lessons (apply proactively)

1. **Path access is infallible.** Use `if is_string(.path) { string!(.path) } else { "" }` instead of `.path ?? "x"`.
2. **Local-variable type narrowing required for `map_values` lambdas:**
   ```vrl
   if is_object(.attributes) {
     attrs, _err = object(.attributes)
     if is_object(attrs) {
       .attributes = map_values(object!(attrs)) -> |v| {
         if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
       }
     }
   }
   ```
3. **Runner pattern:** `vector vrl -p program.vrl -i inputs.ndjson -q -o > .actual.ndjson`, diff via `jq -c .`.
4. **Expected outputs may need alphabetical key order** to match Vector's emitted output (jq -c normalizes whitespace but compares content as-given). If diff fails on key-order alone, reorder keys in `expected.ndjson` to alphabetical.

## Task Description

### Task A8: redact_emails_metrics remap + VRL unit test

**Files:**
- Modify: `templates/values.yaml.tpl`
- Create: `tests/vrl/redact_emails_metrics/program.vrl`
- Create: `tests/vrl/redact_emails_metrics/inputs.ndjson`
- Create: `tests/vrl/redact_emails_metrics/expected.ndjson`
- Create: `tests/vrl/redact_emails_metrics/run.sh`

For OTLP metrics, redaction targets `.attributes` (metric attributes / labels). Some metric shapes may also have `.resource.attributes` for resource-level — walk both for safety, mirroring traces.

- [ ] **Step 1: VRL program** at `tests/vrl/redact_emails_metrics/program.vrl`:

```vrl
email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'

if is_object(.attributes) {
  attrs, _err = object(.attributes)
  if is_object(attrs) {
    .attributes = map_values(object!(attrs)) -> |v| {
      if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
    }
  }
}

if is_object(.resource) {
  res, _err = object(.resource)
  if is_object(res) && is_object(res.attributes) {
    rattrs, _err2 = object(res.attributes)
    if is_object(rattrs) {
      .resource.attributes = map_values(object!(rattrs)) -> |v| {
        if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
      }
    }
  }
}

.
```

- [ ] **Step 2: Inputs** at `tests/vrl/redact_emails_metrics/inputs.ndjson`:

```json
{"name": "request.count", "value": 42, "attributes": {"submitted_by": "ops@x.com"}}
{"name": "cpu.usage", "value": 0.5, "attributes": {"host": "node-1"}}
```

- [ ] **Step 3: Expected outputs** at `tests/vrl/redact_emails_metrics/expected.ndjson`:

```json
{"name": "request.count", "value": 42, "attributes": {"submitted_by": "[REDACTED-EMAIL]"}}
{"name": "cpu.usage", "value": 0.5, "attributes": {"host": "node-1"}}
```

If the runner fails due to key-order mismatch, reorder these expected lines to alphabetical key order — Vector emits keys sorted, and jq's diff compares verbatim content after whitespace normalization.

- [ ] **Step 4: Runner** at `tests/vrl/redact_emails_metrics/run.sh`:

```bash
#!/usr/bin/env bash
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ACTUAL="$DIR/.actual.ndjson"
vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o > "$ACTUAL"
diff -u <(jq -c . "$DIR/expected.ndjson") <(jq -c . "$ACTUAL")
echo "redact_emails_metrics VRL OK"
```

`chmod +x tests/vrl/redact_emails_metrics/run.sh`.

- [ ] **Step 5: Run, expect PASS.**

```bash
./tests/vrl/redact_emails_metrics/run.sh
```

- [ ] **Step 6: Add transform to `templates/values.yaml.tpl`.** Append into the existing `transforms:` map (after `redact_emails_traces:`):

```yaml
    redact_emails_metrics:
      type: remap
      inputs:
        - otlp_in.metrics
      source: |
        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
        if is_object(.attributes) {
          attrs, _err = object(.attributes)
          if is_object(attrs) {
            .attributes = map_values(object!(attrs)) -> |v| {
              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
            }
          }
        }
        if is_object(.resource) {
          res, _err = object(.resource)
          if is_object(res) && is_object(res.attributes) {
            rattrs, _err2 = object(res.attributes)
            if is_object(rattrs) {
              .resource.attributes = map_values(object!(rattrs)) -> |v| {
                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
              }
            }
          }
        }
```

- [ ] **Step 7: Update metrics sink:**

```yaml
    datadog_metrics:
      type: datadog_metrics
      inputs:
        - redact_emails_metrics   # was: otlp_in.metrics
```

- [ ] **Step 8: Run `terraform fmt -check -recursive`** — expected exit 0.

## Constraints

- **DO NOT run any git commands.**
- **DO NOT** push, tag, open MRs.
- **DO NOT** run `terraform apply`, `kubectl`, `helm install`, etc.
- **DO NOT** create snapshot-test infrastructure.
- VRL test execution via `vector` CLI is required.

## Spec / plan references

- Spec: `/Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-05-vector-redaction-design.md` §4.3
- Plan: `/Users/steve.briggs/projects/IAC/docs/superpowers/plans/2026-05-05-vector-redaction-pipeline.md` Task A8

## Report Format

- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT
- VRL test result
- Template changes
- `terraform fmt` result
- Self-review findings


I'll start by reading the existing files to understand the current state before making changes.


Bash

List existing VRL test directories


1	role: Aggregator
2	replicas: ${replicas}
3	podAnnotations:
4	  reloader.stakater.com/auto: "true"
5	service:
6	  enabled: true
7	  type: ClusterIP
8	  ports:
9	    - name: otlp-grpc
10	      port: 4317
11	      protocol: TCP
12	    - name: otlp-http
13	      port: 4318
14	      protocol: TCP
15	env:
16	  - name: DATADOG_API_KEY
17	    valueFrom:
18	      secretKeyRef:
19	        name: vector-datadog
20	        key: api_key
21	resources:
22	  requests:
23	    cpu: ${requests_cpu}
24	    memory: ${requests_memory}
25	  limits:
26	    cpu: "${limits_cpu}"
27	    memory: ${limits_memory}
28	customConfig:
29	  data_dir: /vector-data-dir
30	  api:
31	    enabled: true
32	    address: 127.0.0.1:8686
33	  sources:
34	    otlp_in:
35	      type: opentelemetry
36	      grpc:
37	        address: 0.0.0.0:4317
38	      http:
39	        address: 0.0.0.0:4318
40	  transforms:
41	    drop_audit:
42	      type: filter
43	      inputs:
44	        - otlp_in.logs
45	      condition: |
46	        msg = if is_string(.message) { string!(.message) } else { "" }
47	        !match(msg, r'Audit\s-\s.+') && !match(msg, r'"level":"audit"')
48	    redact_emails_logs:
49	      type: remap
50	      inputs:
51	        - drop_audit
52	      source: |
53	        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
54	        if is_string(.message) {
55	          .message = replace(string!(.message), email_re, "${email_redaction_replacement}")
56	        }
57	        if is_object(.attributes) {
58	          attrs, _err = object(.attributes)
59	          if is_object(attrs) {
60	            .attributes = map_values(object!(attrs)) -> |v| {
61	              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
62	            }
63	          }
64	        }
65	    redact_emails_traces:
66	      type: remap
67	      inputs:
68	        - otlp_in.traces
69	      source: |
70	        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
71	        if is_object(.attributes) {
72	          attrs, _err = object(.attributes)
73	          if is_object(attrs) {
74	            .attributes = map_values(object!(attrs)) -> |v| {
75	              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
76	            }
77	          }
78	        }
79	        if is_object(.resource) {
80	          res, _err = object(.resource)
81	          if is_object(res) && is_object(res.attributes) {
82	            rattrs, _err2 = object(res.attributes)
83	            if is_object(rattrs) {
84	              .resource.attributes = map_values(object!(rattrs)) -> |v| {
85	                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
86	              }
87	            }
88	          }
89	        }
90	  sinks:
91	    datadog_logs:
92	      type: datadog_logs
93	      inputs:
94	        - redact_emails_logs
95	      default_api_key: $${DATADOG_API_KEY}
96	      site: ${datadog_site}
97	    datadog_traces:
98	      type: datadog_traces
99	      inputs:
100	        - redact_emails_traces
101	      default_api_key: $${DATADOG_API_KEY}
102	      site: ${datadog_site}
103	    datadog_metrics:
104	      type: datadog_metrics
105	      inputs:
106	        - otlp_in.metrics
107	      default_api_key: $${DATADOG_API_KEY}
108	      site: ${datadog_site}
109


drwxr-xr-x steve.briggs staff 128 B Tue May  5 12:52:54 2026 ..
drwxr-xr-x steve.briggs staff 224 B Tue May  5 12:59:12 2026 drop_audit
drwxr-xr-x steve.briggs staff 224 B Tue May  5 13:08:23 2026 redact_emails_logs
drwxr-xr-x steve.briggs staff 160 B Tue May  5 13:14:54 2026 .
drwxr-xr-x steve.briggs staff 224 B Tue May  5 13:17:11 2026 redact_emails_traces


Good. Now I have a clear picture of the current state. Let me look at an existing test to understand the structure, then create all the new files.


Error: EISDIR: illegal operation on a directory, read '/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_traces'


List redact_emails_traces test files


drwxr-xr-x steve.briggs staff 160 B Tue May  5 13:14:54 2026 ..
.rw-r--r-- steve.briggs staff 648 B Tue May  5 13:15:17 2026 program.vrl
.rw-r--r-- steve.briggs staff 277 B Tue May  5 13:15:35 2026 inputs.ndjson
.rwxr-xr-x steve.briggs staff 292 B Tue May  5 13:15:58 2026 run.sh
.rw-r--r-- steve.briggs staff 290 B Tue May  5 13:17:11 2026 expected.ndjson
drwxr-xr-x steve.briggs staff 224 B Tue May  5 13:17:11 2026 .
.rw-r--r-- steve.briggs staff 306 B Tue May  5 13:17:17 2026 .actual.ndjson


Let me read the existing traces test files to understand the exact format used:


1	#!/usr/bin/env bash
2	set -euo pipefail
3	DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
4	ACTUAL="$DIR/.actual.ndjson"
5	vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o > "$ACTUAL"
6	diff -u <(jq -c . "$DIR/expected.ndjson") <(jq -c . "$ACTUAL")
7	echo "redact_emails_traces VRL OK"
8


1	{"attributes": {"http.status": 200, "user.email": "[REDACTED-EMAIL]"}, "name": "POST /signup", "resource": {"attributes": {"owner": "[REDACTED-EMAIL]", "service.name": "web"}}}
2	{"attributes": {"http.status": 200}, "name": "GET /health", "resource": {"attributes": {"service.name": "web"}}}
3


Now I can see that Vector emits keys in alphabetical order. Let me create all the test files. First, let me figure out what key order Vector will produce for the metrics inputs:

The inputs have keys `name`, `value`, `attributes` — Vector will emit alphabetically: `attributes`, `name`, `value`. Let me create all the files now.


Create redact_emails_metrics test directory


(Bash completed with no output)


Write

email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'

if is_object(.attributes) {
  attrs, _err = object(.attributes)
  if is_object(attrs) {
    .attributes = map_values(object!(attrs)) -> |v| {
      if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
    }
  }
}

if is_object(.resource) {
  res, _err = object(.resource)
  if is_object(res) && is_object(res.attributes) {
    rattrs, _err2 = object(res.attributes)
    if is_object(rattrs) {
      .resource.attributes = map_values(object!(rattrs)) -> |v| {
        if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
      }
    }
  }
}

.


{"name": "request.count", "value": 42, "attributes": {"submitted_by": "ops@x.com"}}
{"name": "cpu.usage", "value": 0.5, "attributes": {"host": "node-1"}}


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/program.vrl (file state is current in your context — no need to Read it back)


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/inputs.ndjson (file state is current in your context — no need to Read it back)


Now let me run the VRL program against the inputs first to see the actual output, so I can write the expected.ndjson with the correct key order:


Bash

DIR="/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics"
/opt/homebrew/bin/vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o 2>&1

Run VRL program to see actual output key order


2026-05-05T17:19:34.031507Z  INFO vector::app: Log level is enabled. level="info"
{ "attributes": { "submitted_by": "[REDACTED-EMAIL]" }, "name": "request.count", "value": 42 }
{ "attributes": { "host": "node-1" }, "name": "cpu.usage", "value": 0.5 }


Vector emits keys alphabetically: `attributes`, `name`, `value`. Now I'll write the expected.ndjson in the correct format (compact JSON as jq -c would produce it):


Write

{"attributes":{"submitted_by":"[REDACTED-EMAIL]"},"name":"request.count","value":42}
{"attributes":{"host":"node-1"},"name":"cpu.usage","value":0.5}


#!/usr/bin/env bash
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ACTUAL="$DIR/.actual.ndjson"
vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o > "$ACTUAL"
diff -u <(jq -c . "$DIR/expected.ndjson") <(jq -c . "$ACTUAL")
echo "redact_emails_metrics VRL OK"


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/expected.ndjson (file state is current in your context — no need to Read it back)


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/run.sh (file state is current in your context — no need to Read it back)


Bash

Make run.sh executable


Run the VRL unit test


2026-05-05T17:22:13.756015Z  INFO vector::app: Log level is enabled. level="info"
redact_emails_metrics VRL OK


Test passes. Now update `templates/values.yaml.tpl` — add the `redact_emails_metrics` transform and update the `datadog_metrics` sink input:


Edit

        if is_object(.resource) {
          res, _err = object(.resource)
          if is_object(res) && is_object(res.attributes) {
            rattrs, _err2 = object(res.attributes)
            if is_object(rattrs) {
              .resource.attributes = map_values(object!(rattrs)) -> |v| {
                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
              }
            }
          }
        }
  sinks:

        if is_object(.resource) {
          res, _err = object(.resource)
          if is_object(res) && is_object(res.attributes) {
            rattrs, _err2 = object(res.attributes)
            if is_object(rattrs) {
              .resource.attributes = map_values(object!(rattrs)) -> |v| {
                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
              }
            }
          }
        }
    redact_emails_metrics:
      type: remap
      inputs:
        - otlp_in.metrics
      source: |
        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
        if is_object(.attributes) {
          attrs, _err = object(.attributes)
          if is_object(attrs) {
            .attributes = map_values(object!(attrs)) -> |v| {
              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
            }
          }
        }
        if is_object(.resource) {
          res, _err = object(.resource)
          if is_object(res) && is_object(res.attributes) {
            rattrs, _err2 = object(res.attributes)
            if is_object(rattrs) {
              .resource.attributes = map_values(object!(rattrs)) -> |v| {
                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
              }
            }
          }
        }
  sinks:


The file /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/templates/values.yaml.tpl has been updated successfully. (file state is current in your context — no need to Read it back)


Now update the `datadog_metrics` sink to read from `redact_emails_metrics` instead of `otlp_in.metrics`:


    datadog_metrics:
      type: datadog_metrics
      inputs:
        - otlp_in.metrics

    datadog_metrics:
      type: datadog_metrics
      inputs:
        - redact_emails_metrics


Now run `terraform fmt -check -recursive`:


Bash

terraform fmt -check -recursive /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/; echo "exit: $?"

Run terraform fmt check


exit: 0


Let me do a final verification read of the updated template to confirm it looks correct:


88	          }
89	        }
90	    redact_emails_metrics:
91	      type: remap
92	      inputs:
93	        - otlp_in.metrics
94	      source: |
95	        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
96	        if is_object(.attributes) {
97	          attrs, _err = object(.attributes)
98	          if is_object(attrs) {
99	            .attributes = map_values(object!(attrs)) -> |v| {
100	              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
101	            }
102	          }
103	        }
104	        if is_object(.resource) {
105	          res, _err = object(.resource)
106	          if is_object(res) && is_object(res.attributes) {
107	            rattrs, _err2 = object(res.attributes)
108	            if is_object(rattrs) {
109	              .resource.attributes = map_values(object!(rattrs)) -> |v| {
110	                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
111	              }
112	            }
113	          }
114	        }
115	  sinks:
116	    datadog_logs:
117	      type: datadog_logs
118	      inputs:
119	        - redact_emails_logs
120	      default_api_key: $${DATADOG_API_KEY}
121	      site: ${datadog_site}
122	    datadog_traces:
123	      type: datadog_traces
124	      inputs:
125	        - redact_emails_traces
126	      default_api_key: $${DATADOG_API_KEY}
127	      site: ${datadog_site}
128	    datadog_metrics:
129	      type: datadog_metrics
130	      inputs:
131	        - redact_emails_metrics
132	      default_api_key: $${DATADOG_API_KEY}
133	      site: ${datadog_site}
134


Everything looks correct. Let me also run the existing tests to confirm nothing was broken:


for dir in /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/*/; do
  echo "--- running $(basename $dir) ---"
  "$dir/run.sh" 2>/dev/null | grep -v "^2026"
done

Run all VRL tests


--- running drop_audit ---
drop_audit VRL OK
--- running redact_emails_logs ---
redact_emails_logs VRL OK
--- running redact_emails_metrics ---
redact_emails_metrics VRL OK
--- running redact_emails_traces ---
redact_emails_traces VRL OK


All four tests pass. Here's the summary, Sir Charles of St Cheese:

---

**Status:** DONE

**VRL test result:** PASS — `redact_emails_metrics VRL OK`. All four VRL tests pass (drop_audit, redact_emails_logs, redact_emails_traces, redact_emails_metrics).

**Template changes:**
- Added `redact_emails_metrics` remap transform at line 90 of `templates/values.yaml.tpl`, inserted after `redact_emails_traces`. It reads from `otlp_in.metrics` and redacts emails from both `.attributes` and `.resource.attributes` using the `${email_redaction_replacement}` template variable, mirroring the traces transform structure.
- Updated `datadog_metrics` sink input from `otlp_in.metrics` → `redact_emails_metrics` (line 131).

**`terraform fmt` result:** Exit 0, no formatting issues.

**Self-review findings:**
- Expected output key order was validated empirically by running Vector before writing `expected.ndjson` — keys come out alphabetically (`attributes`, `name`, `value`), matching what the diff comparison requires.
- The `redact_emails_metrics` transform in the template correctly omits the trailing `.` statement (unlike the VRL test program) — that's fine as Vector remap sources don't need an explicit final expression.
- No drift from the traces transform pattern; the metrics transform is structurally identical, intentional given the same OTLP attribute shape.

**New files created:**
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/program.vrl`
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/inputs.ndjson`
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/expected.ndjson`
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_metrics/run.sh`