You are doing a final, holistic code review of the complete "DD Agent Logs via Vector (OPW Intake)" implementation. The change spans three terraform repos (additive only). No git commits — Sir Charles handles git.

## Spec
`/Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md`

## Plan
`/Users/steve.briggs/projects/IAC/docs/superpowers/plans/2026-05-06-datadog-agent-logs-via-vector.md`

## What changed (by repo)

### Vector module — `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/`
- `templates/values.yaml.tpl` — added port 8282 to `service.ports`; added `dd_agent_in` source under `customConfig.sources`; expanded `drop_audit.inputs` to multi-input.
- `netpol.tf` — added port 8282 to the `fromEntities = ["cluster"]` ingress rule.
- `outputs.tf` — added `service_endpoint_dd_agent` output.
- `tests/vrl/redact_emails_dd_agent_logs/` — new test directory with inputs.ndjson, expected.ndjson, program.vrl, run.sh (executable).
- `.gitlab-ci.yml` — added new test runner to the `vrl-tests` job.
- `README.md` — architecture, outputs, usage, testing all updated.

### DD Agent module — `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/`
- `variables.tf` — added `logs_intake_url` (string, default null) with URL-prefix validation.
- `main.tf` — passes `logs_intake_url = var.logs_intake_url` into the templatefile call.
- `values.yaml.tftpl` — added gated block (`%{ if logs_intake_url != null ~}...%{ endif ~}`) for two OPW env vars in `datadog.env`.
- `tests/render/` — new test directory: run.sh harness + expected-direct.yaml (130-line baseline) + expected-via-vector.yaml.
- `README.md` — added input row + new "OPW Logs Intake" section.

### Consumer — `/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/`
- `datadog_agent.tf` — bumped module ref `v1.1.2` → `v1.2.0`; added `logs_intake_url` ternary to module call; added new variable `vector_dd_agent_logs_enabled` (bool, nullable); added local that inherits from `local.vector_enabled`; added plan-time guard via `data "external"`.
- `vector.tf` — bumped module ref to `v1.1.0`, removed local-path source.
- `README.md` — added inputs table row (twice — friendly + auto-generated tables) and "DD Agent → Vector log routing (OPW)" section.

## Verification done

- All 5 VRL tests pass: `drop_audit`, `redact_emails_logs`, `redact_emails_traces`, `redact_emails_metrics`, `redact_emails_dd_agent_logs`.
- DD Agent render harness passes: rendering with `logs_intake_url = null` is byte-identical to the locked baseline; rendering with the URL produces the two OPW env vars.
- `terraform fmt -check` clean on all our touched .tf files. (Pre-existing fmt drift in `otel_collector.tf`, `sysdig_agent.tf`, `netpols.tf` — untouched by us.)
- `terraform validate` deferred for the consumer because module tags aren't published yet.

## Review rubric

Apply the standard requesting-code-review concerns. Plus the additions from the subagent-driven-development skill:
- Does each file have one clear responsibility with a well-defined interface?
- Are units decomposed so they can be understood and tested independently?
- Did this implementation create new files that are already large?

Specific things to evaluate:

1. **Spec compliance** — read the spec sections 3.1, 3.2, 4.1, 4.2, 4.3, 5, 6, 7. Confirm each requirement maps to actual code/file changes.
2. **Integration consistency** — the `vector_dd_agent_logs_enabled` flag in the consumer references `local.vector_enabled` (in vector.tf). Confirm the local exists. Confirm the ternary `local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null` short-circuits correctly when Vector is disabled (so `module.vector[0]` isn't dereferenced when count=0).
3. **Naming consistency** — same names across all three repos (e.g., `service_endpoint_dd_agent` in Vector outputs == consumer's wiring; `logs_intake_url` in DD Agent module == consumer's argument).
4. **Port consistency** — 8282 should appear consistently across Vector's service.ports + dd_agent_in.address + netpol.tf + service_endpoint_dd_agent output URL + DD Agent module README + consumer README.
5. **Version refs** — the Vector module README example uses `v1.1.0`; the consumer's vector.tf uses `v1.1.0`; the consumer's datadog_agent.tf uses `v1.2.0`; the DD Agent module README references `v1.2.0`. All consistent.
6. **Strip-newline directives** — the new `%{ if ... ~}` and `%{ endif ~}` in `values.yaml.tftpl` both have the `~` for whitespace stripping (proven by the byte-identical render assertion).
7. **Test rigor** — are the VRL fixtures meaningful (cover the redaction edge cases)? Is the render harness diff-based and reproducible?
8. **Backwards compatibility** — clusters that don't opt in (vector_enabled=false, or vector_dd_agent_logs_enabled explicitly false) should see zero behavior change. Verify by tracing the variable wiring.
9. **YAGNI** — anything overbuilt? The spec called out a single-bool variable (not a per-cluster-type map), confirm we didn't over-engineer.
10. **Pre-existing drift** — confirm that `netpols.tf` (in datadog-agent), `otel_collector.tf`, `sysdig_agent.tf` (in consumer) — files we did NOT touch — aren't part of our delta. Their formatting issues are pre-existing.

## Important context

This codebase has the convention "Sir Charles handles all git operations." The implementer subagents made file edits and ran validation but did NOT commit. Therefore:
- Some local repos are git, some are not (the vector module is NOT a git repo locally; the datadog-agent module IS).
- There are no SHAs to compare. Review the working-tree state.
- "Frequent commits" from the plan are advisory — the actual commit cadence is up to Sir Charles.

## Report format

**Strengths:**
- [bullets]

**Issues:**
- **Critical:** [must fix before tagging modules — none expected]
- **Important:** [should fix soon]
- **Minor:** [nits]

**Spec coverage gaps (if any):** [list]

**Assessment:** ✅ Ready to tag and ship / ⚠ Fix issues before tagging / ❌ Major rework needed


I'll do a thorough holistic review across all three repos. Let me start by reading the spec, plan, and all the changed files.


1	# Datadog Agent Logs via Vector (OPW Intake) — Design
2
3	**Date:** 2026-05-06
4	**Author:** Sir Charles of St Cheese (with Claudebob)
5	**Status:** Draft
6	**Depends on:** [2026-05-05 Vector + OpenTelemetry Redaction Pipeline](./2026-05-05-vector-redaction-design.md)
7
8	## 1. Goal
9
10	Apply Vector's existing email-redaction transform to the container logs the Datadog Agent collects today. Logs currently bypass Vector entirely (DD Agent ships direct to `datadoghq.com:443`); after this change, DD-Agent-collected logs traverse Vector before reaching Datadog SaaS.
11
12	**Out of scope:**
13	- Migrating the agent-side audit-drop processing rules (`processing-rules/{env}/rules.json`) into Vector. The agent rules continue to drop audit lines before transmission; Vector adds a defense-in-depth `drop_audit` filter on the new path but does not replace the agent rules.
14	- Routing DD Agent metrics or APM through Vector. Metrics and traces keep their existing OTel-based path.
15	- Any change to the OTel pipeline.
16
17	## 2. Approach
18
19	Use the **Datadog Observability Pipelines Worker (OPW)** intake protocol. Vector's native `datadog_agent` source is the same wire protocol Datadog's OPW product uses (Datadog acquired Vector; OPW is Vector with a curated config). Configuring the agent to ship to Vector is the standard, documented OPW deployment shape.
20
21	Two environment variables on the DD Agent — `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED=true` and `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL=<vector-url>` — redirect the agent's logs intake to Vector. No `logs_config.use_http` / `logs_no_ssl` / `logs_dd_url` block needed; the URL scheme implies plaintext for `http://` cluster-internal endpoints.
22
23	Reference: <https://docs.datadoghq.com/observability_pipelines/sources/datadog_agent/?tab=secretsmanagement>
24
25	## 3. Architecture
26
27	### 3.1 Pipeline (logs path)
28
29	**Before:**
30	```
31	DD Agent (DaemonSet)
32	  → [agent-side processing rules drop audits]
33	  → datadoghq.com:443
34	```
35
36	**After:**
37	```
38	DD Agent (DaemonSet)
39	  → [agent-side processing rules drop audits]
40	  → vector.vector.svc.cluster.local:8282  (OPW HTTP intake, no TLS)
41	  → Vector dd_agent_in source
42	  → drop_audit  (multi-input: otlp_in.logs + dd_agent_in.logs — defense-in-depth)
43	  → redact_emails_logs  (when enabled)
44	  → datadog_logs sink → datadoghq.com:443
45	```
46
47	Traces and metrics are unaffected — they keep their existing path (apps → otel-collector → vector → DD).
48
49	### 3.2 Module split
50
51	Three repos touched, additive only:
52
53	1. **`internal-modules/kubernetes/monitoring/vector`** — bumps to `v1.1.0`. Adds the `datadog_agent` source on port 8282, exposes the port in helm `service.ports`, opens netpol ingress, makes `drop_audit` accept the new input, and adds an output `service_endpoint_dd_agent`. The source is always declared (the receiver is harmless if no agent is pointed at it), so there is no new module variable.
54	2. **`internal-modules/kubernetes/monitoring/datadog-agent`** — bumps to `v1.2.0` (current consumer ref is `v1.1.2`). Adds one variable `logs_intake_url` (string, default `null`). When non-null, the values template renders the two OPW env vars in the `datadog.env` block. When null, rendered output is byte-identical to today.
55	3. **`eks-cluster-applications`** consumer — adds `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors the existing `vector_enabled` pattern). Wires the URL across modules.
56
57	## 4. Components
58
59	### 4.1 Vector module deltas
60
61	**`templates/values.yaml.tpl`:**
62
63	- `service.ports`: add `{ name: dd-agent, port: 8282, protocol: TCP }`.
64	- `customConfig.sources.dd_agent_in`:
65	  ```yaml
66	  dd_agent_in:
67	    type: datadog_agent
68	    address: 0.0.0.0:8282
69	    multiple_outputs: true
70	    disable_metrics: true
71	    disable_traces: true
72	    store_api_key: false
73	    acknowledgements:
74	      enabled: true
75	  ```
76	  - `multiple_outputs: true` — exposes the `.logs` named output for downstream transforms.
77	  - `disable_metrics`/`disable_traces: true` — DD Agent only forwards logs through this path; metrics/traces still come via OTel.
78	  - `store_api_key: false` — the Vector sink supplies its own API key from the existing ExternalSecret; storing the per-event key is unnecessary.
79	  - `acknowledgements.enabled: true` — DD Agent retries on transient Vector failures, no silent log loss.
80	- `customConfig.transforms.drop_audit.inputs`: change `["otlp_in.logs"]` → `["otlp_in.logs", "dd_agent_in.logs"]`.
81
82	**`netpol.tf`:** add `port = "8282"` to the existing `fromEntities = ["cluster"]` ingress rule's `toPorts.ports` list (alongside 4317/4318). DD Agent pods are cluster-internal, no new from-selector needed.
83
84	**`outputs.tf`:**
85	```hcl
86	output "service_endpoint_dd_agent" {
87	  description = "OPW-compatible Datadog Agent intake endpoint (http, cluster-internal)"
88	  value       = "http://vector.${var.namespace}.svc.cluster.local:8282"
89	}
90	```
91
92	**README:** note that Vector functions as an OPW logs intake on 8282; describe the `datadog_agent` source.
93
94	### 4.2 DD Agent module deltas
95
96	**`variables.tf`:**
97	```hcl
98	variable "logs_intake_url" {
99	  description = "OPW-compatible logs intake URL. When set, the agent ships logs to this endpoint via DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL. For cluster-internal Vector use module.vector.service_endpoint_dd_agent. When null (default), logs ship direct to Datadog SaaS."
100	  type        = string
101	  default     = null
102
103	  validation {
104	    condition     = var.logs_intake_url == null || can(regex("^https?://", var.logs_intake_url))
105	    error_message = "logs_intake_url must be a URL starting with http:// or https://, or null."
106	  }
107	}
108	```
109
110	**`values.yaml.tftpl`:** append to the existing `datadog.env` list, gated by `%{ if logs_intake_url != null ~}`:
111	```yaml
112	%{ if logs_intake_url != null ~}
113	    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED
114	      value: "true"
115	    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL
116	      value: "${logs_intake_url}"
117	%{ endif ~}
118	```
119
120	When `logs_intake_url` is null, rendered output is byte-identical to today.
121
122	**`main.tf`:** pass `logs_intake_url = var.logs_intake_url` into the `templatefile()` call's variables map.
123
124	**No netpol change.** The existing `egress: toEntities = ["cluster"]` catch-all already permits cluster-internal egress to Vector. The `*.datadoghq.com:443` FQDN rule still applies for metrics/traces and remains the path for logs when `logs_intake_url = null`.
125
126	**Agent-side processing rules unchanged.** `DD_LOGS_CONFIG_PROCESSING_RULES` continues to drop audit lines before transmission, regardless of destination.
127
128	**README:** document `logs_intake_url`.
129
130	### 4.3 Consumer (`eks-cluster-applications`) deltas
131
132	- New variable `vector_dd_agent_logs_enabled` (bool, nullable, default `null`). When `null`, the flag inherits from `local.vector_enabled` — i.e., defaults on wherever Vector is on. When set explicitly to `true`/`false`, it overrides. Single variable, no per-cluster-type map; the cluster-type granularity is already provided by `vector_enabled`'s map and there's nothing meaningful to do with this flag in clusters where Vector isn't deployed.
133	- Local:
134	  ```hcl
135	  locals {
136	    vector_dd_agent_logs_enabled = (
137	      var.vector_dd_agent_logs_enabled != null
138	      ? var.vector_dd_agent_logs_enabled
139	      : local.vector_enabled
140	    )
141	  }
142	  ```
143	- Local-precondition guard: if `local.vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.
144	- Wire on the existing `module.datadog_agent` call:
145	  ```hcl
146	  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
147	  ```
148	- Bump `module.datadog_agent` source ref to `v1.2.0` and `module.vector` source ref to `v1.1.0`.
149	- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).
150
151	## 5. Data Flow
152
153	Per pod, per log line, after cutover:
154
155	1. Container writes to stdout/stderr.
156	2. DD Agent's containerd integration tails the line, attaches Kubernetes metadata (namespace, pod, container, service, ddtags).
157	3. Agent applies `DD_LOGS_CONFIG_PROCESSING_RULES`. If the line matches an `exclude_at_match` audit pattern, **drop here** — never leaves the node.
158	4. Agent batches surviving lines and POSTs them to `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL` (= `http://vector.vector.svc.cluster.local:8282`) using the OPW HTTP intake protocol. The `DD-API-KEY` header is sent but ignored by Vector (`store_api_key: false`).
159	5. Service routes to one of the 3 Vector pods (round-robin via ClusterIP).
160	6. Vector's `dd_agent_in` source decodes the request, emits events on the `dd_agent_in.logs` named output.
161	7. `drop_audit` filter receives both `otlp_in.logs` and `dd_agent_in.logs`. Same regex check applied — defense-in-depth catches anything the agent rules would miss.
162	8. `redact_emails_logs` (when enabled) substitutes matching email patterns with `[REDACTED-EMAIL]` in `.message` and string-typed `.attributes` values.
163	9. `datadog_logs` sink batches and POSTs to `https://*.datadoghq.com:443` using the API key from the Vector ExternalSecret.
164	10. Acknowledgement flows back: sink → transforms → source → DD Agent. Agent retries on transient Vector failures.
165
166	**Cutover semantics:** when `vector_dd_agent_logs_enabled` flips false→true, terraform apply re-renders the DD Agent helm release with the two new env vars. The DaemonSet rolls (default RollingUpdate). During the rolling window, some agent pods ship to DD SaaS direct, others ship to Vector. Both paths converge to the same DD Logs index; no data loss, possible brief duplication if a single line spans the rollover. Acceptable.
167
168	## 6. Error Handling
169
170	### 6.1 Module input validation
171
172	- **DD Agent module — `logs_intake_url`:** validation regex `^https?://`, rejects empty string. Null is allowed (means: ship direct).
173	- **Consumer — `vector_dd_agent_logs_enabled`:** plan-time guard. If true but `local.vector_enabled = false`, fail with "vector_dd_agent_logs_enabled requires vector_enabled". Ensures the agent isn't pointed at a non-existent service.
174	- **Vector module — no new validation.** The `datadog_agent` source is harmless when no agent connects.
175
176	### 6.2 Misconfiguration paths
177
178	- Wrong port in URL (e.g., 8080 instead of 8282): DD Agent connection refused → agent buffers + retries → visible on `datadog.agent.logs.bytes_sent` flatlining and `datadog.agent.logs.errors_count` increasing.
179	- Vector source misconfigured (e.g., accidental `disable_metrics: false`): Vector startup config validation rejects at helm release time; pods crash-loop visibly via `kubectl logs`.
180	- Stale `module.vector` ref when bumping consumer: terraform plan diffs the new `service_endpoint_dd_agent` output; if module isn't bumped, the output doesn't exist and the consumer plan fails fast with "Unsupported attribute".
181
182	### 6.3 Runtime failure modes
183
184	- **Vector unhealthy / pod unreachable:** DD Agent retries via OPW protocol. If retries exhaust, agent buffers in its on-disk log buffer (existing behavior). Once Vector recovers, queued logs flush.
185	- **Vector backpressure (sink rejecting):** acknowledgements propagate back to DD Agent, slowing ingestion. No silent loss.
186	- **Email redact VRL panic on a malformed event:** Vector's `remap` transform drops the bad event by default. Visible on `vector_component_errors_total{component_id="redact_emails_logs"}`.
187	- **Bad rollout / Vector misbehaving:** flip `vector_dd_agent_logs_enabled = false` for the affected cluster, terraform apply, agent re-rolls back to direct-DD shipping in ~3-5 minutes.
188
189	### 6.4 Diagnostic surfaces
190
191	Use existing infrastructure:
192
193	- `vector_component_received_events_total{component_id="dd_agent_in"}` — ingest sanity.
194	- `vector_component_sent_events_total{component_id="datadog_logs"}` — sink shipping sanity.
195	- `vector_component_errors_total{component_id=~"dd_agent_in|drop_audit|redact_emails_logs|datadog_logs"}` — error rate per stage.
196	- DD Agent telemetry: `datadog.agent.logs.bytes_sent`, `datadog.agent.logs.errors_count`, `datadog.agent.logs.bytes_buffered`.
197
198	## 7. Testing
199
200	### 7.1 Unit-level (per module, in-repo)
201
202	1. **Vector module — VRL tests.** Add a 5th VRL test `tests/vrl/redact_emails_dd_agent_logs/` with input shape matching `datadog_agent` source events (top-level `.message` string with embedded email + DD agent metadata: `ddtags`, `service`, `hostname`). Same VRL program as the existing logs test. Confirms the redaction transform is source-agnostic.
203	2. **Vector module — `terraform fmt` / `validate`** at module root and `examples/basic/`. Standard.
204	3. **DD Agent module — template render check.** Two assertions via ad-hoc render:
205	   - `logs_intake_url = null` → output byte-identical to pre-change render (no churn).
206	   - `logs_intake_url = "http://vector.vector.svc.cluster.local:8282"` → both OPW env vars appear in `datadog.env`.
207	4. **Consumer — `terraform fmt` / `validate`.** Standard.
208
209	### 7.2 Integration verification (in-cluster, ad-hoc, after deploy)
210
211	1. Deploy a one-shot test pod that emits a UUID-tagged log line containing an email:
212	   ```bash
213	   kubectl run dd-redact-test-$RANDOM \
214	     --image=busybox --restart=Never --rm -it \
215	     -- sh -c 'echo "dd-e2e-$(cat /proc/sys/kernel/random/uuid): contact alice@example.com"; sleep 5'
216	   ```
217	2. Watch Vector's `dd_agent_in` ingest counter:
218	   ```bash
219	   kubectl exec -n vector deploy/vector -- curl -s http://localhost:8686/metrics \
220	     | grep 'vector_component_received_events_total{component_id="dd_agent_in"'
221	   ```
222	   Should tick up by ≥1.
223	3. Query Datadog for the UUID — expect the log to appear with `[REDACTED-EMAIL]` substituted for the address.
224	4. **Audit-drop verification:** emit an `Audit - test-<UUID>` line. Expect zero results in DD (dropped at the agent before transmission). Vector's `dd_agent_in` received counter should also NOT increment for this line.
225	5. **Rollback verification:** flip `vector_dd_agent_logs_enabled = false`, `terraform apply`, repeat step 1, expect log to appear in DD un-redacted (proving the agent went back to direct shipping). Then flip back.
226
227	### 7.3 Cross-cutting checks
228
229	- **DD Logs UI metadata preservation.** After cutover, verify `service`, `host`, `kubernetes.namespace`, `kubernetes.pod_name`, `ddtags` are still attached to logs in DD UI. The `datadog_agent` source preserves them; the `datadog_logs` sink forwards them; redaction transforms only touch `.message` and string `.attributes`. Verify via a known-shape log line.
230	- **Redaction does not corrupt JSON-structured logs.** A JSON line like `{"user":"alice@example.com","level":"info"}` should become `{"user":"[REDACTED-EMAIL]","level":"info"}` and remain parseable in DD.
231
232	### 7.4 Soak
233
234	- One sandbox/dev cluster: 1 day, full smoke suite + Vector self-metrics watch.
235	- One `mgmt` cluster: 3-7 day soak. Watch `vector_component_errors_total{component_id="dd_agent_in"}`, agent buffer depth (`datadog.agent.logs.bytes_buffered`), DD Logs ingest delta vs. baseline.
236	- Remaining `mgmt` + all `cps` clusters.
237	- `saas` clusters region-by-region.
238
239	## 8. Open Questions
240
241	1. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).
242	2. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.
243
244	## 9. Next Steps (after design approval)
245
246	1. Hand off to `writing-plans` skill to produce the implementation plan.
247	2. Plan should sequence: Vector module change → DD Agent module change → consumer wire-up → ad-hoc deploy verification.
248	3. Sir Charles handles git operations (tag `vector v1.1.0`, `datadog-agent v1.1.0`, MRs for all three repos) per established working pattern.
249


1	# Datadog Agent Logs via Vector (OPW Intake) Implementation Plan
2
3	> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
5	**Goal:** Route DD-Agent-collected container logs through Vector (using Datadog's OPW intake protocol) so the existing email-redaction VRL transform applies to them before they reach Datadog SaaS.
6
7	**Architecture:** Three additive module changes. Vector module adds a `datadog_agent` source on port 8282 and exposes a new output. DD Agent module adds a `logs_intake_url` variable that, when non-null, renders two OPW env vars (`DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED`, `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL`) into the agent's helm values. Consumer wires them via a new `vector_dd_agent_logs_enabled` flag that defaults to follow `local.vector_enabled`.
8
9	**Tech Stack:** Terraform (>= 1.3.0, < 2.0), Helm provider (~> 2.0), Kubernetes provider (~> 2.0), Vector (timberio/vector chart), Datadog Agent helm chart, VRL (Vector Remap Language), Cilium NetworkPolicy.
10
11	**Spec:** `docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md`
12
13	**Sir Charles handles all git operations** — branches, tags (`vector v1.1.0`, `datadog-agent v1.2.0`), and MRs. Each "Commit" step is something Sir Charles will run himself or skip; the plan still calls them out so the work is broken cleanly.
14
15	---
16
17	## File Structure
18
19	### Vector module (`/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector`)
20
21	| Path | Action | Responsibility |
22	|---|---|---|
23	| `templates/values.yaml.tpl` | Modify | Add 8282 port to service.ports; add `dd_agent_in` source block; expand `drop_audit.inputs` to include the new source |
24	| `netpol.tf` | Modify | Add port 8282 to existing `fromEntities = ["cluster"]` ingress rule |
25	| `outputs.tf` | Modify | Add `service_endpoint_dd_agent` output |
26	| `tests/vrl/redact_emails_dd_agent_logs/inputs.ndjson` | Create | DD-Agent-shape event fixtures (top-level message + ddtags/service/hostname metadata) |
27	| `tests/vrl/redact_emails_dd_agent_logs/expected.ndjson` | Create | Expected-output fixtures with emails redacted |
28	| `tests/vrl/redact_emails_dd_agent_logs/program.vrl` | Create | VRL transform (identical logic to `redact_emails_logs/program.vrl`, source-agnostic) |
29	| `tests/vrl/redact_emails_dd_agent_logs/run.sh` | Create | Shell runner mirroring sibling tests |
30	| `.gitlab-ci.yml` | Modify | Add the new VRL test to the `vrl-tests` job script list |
31	| `README.md` | Modify | Add architecture line about DD Agent intake; new output row; new port row |
32
33	### DD Agent module (`/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent`)
34
35	| Path | Action | Responsibility |
36	|---|---|---|
37	| `variables.tf` | Modify | Add `logs_intake_url` variable + URL-prefix validation |
38	| `values.yaml.tftpl` | Modify | Append two OPW env vars to `datadog.env`, gated by `%{ if logs_intake_url != null ~}` |
39	| `main.tf` | Modify | Pass `logs_intake_url = var.logs_intake_url` into the templatefile call |
40	| `tests/render/expected-direct.yaml` | Create | Expected render output for `logs_intake_url = null` (the "no churn" baseline) |
41	| `tests/render/expected-via-vector.yaml` | Create | Expected render output for `logs_intake_url = "http://vector.vector.svc.cluster.local:8282"` |
42	| `tests/render/run.sh` | Create | Renders the template under both inputs and diffs against expected outputs |
43	| `README.md` | Modify | Add `logs_intake_url` to the inputs table; one paragraph describing the OPW intake path |
44
45	### Consumer (`/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications`)
46
47	| Path | Action | Responsibility |
48	|---|---|---|
49	| `datadog_agent.tf` | Modify | Add `vector_dd_agent_logs_enabled` variable + local + plan-time guard; pass `logs_intake_url` into `module.datadog_agent`; bump module ref from `v1.1.2` → `v1.2.0` |
50	| `vector.tf` | Modify | Bump `module "vector"` source ref from `v1.0.0` → `v1.1.0` (replace the commented-out git ref + remove local-path source) |
51	| `README.md` | Modify | Add `vector_dd_agent_logs_enabled` row to inputs table; note dual-layer log filtering |
52
53	---
54
55	## Phase A: Vector module v1.1.0
56
57	### Task A1: Add VRL test fixture for DD-Agent-shape logs (failing)
58
59	**Files:**
60	- Create: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_dd_agent_logs/inputs.ndjson`
61	- Create: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_dd_agent_logs/expected.ndjson`
62	- Create: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_dd_agent_logs/program.vrl`
63	- Create: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_dd_agent_logs/run.sh`
64
65	- [ ] **Step 1: Write the input fixture**
66
67	Create `tests/vrl/redact_emails_dd_agent_logs/inputs.ndjson` with five DD-Agent-shape events. The `datadog_agent` source emits events with a top-level `.message` string plus DD-Agent-attached metadata (`ddtags`, `service`, `hostname`, `status`). The VRL transform should redact emails in `.message` and string-typed `.attributes` values, but leave metadata fields untouched.
68
69	```json
70	{"message": "user alice@example.com signed in", "ddtags": "env:prod,service:api", "service": "api", "hostname": "ip-10-0-1-23"}
71	{"message": "no email here", "ddtags": "env:prod", "service": "worker", "hostname": "ip-10-0-1-24"}
72	{"message": "multiple a@b.com and c.d+tag@sub.example.co.uk", "service": "api", "hostname": "ip-10-0-1-25"}
73	{"message": "ok", "attributes": {"user.email": "bob@x.io", "request.id": "abc-123"}, "ddtags": "env:staging", "service": "auth", "hostname": "ip-10-0-2-99"}
74	{"message": "json blob {\"email\":\"x@y.com\"}", "service": "api", "hostname": "ip-10-0-1-50"}
75	```
76
77	- [ ] **Step 2: Write the expected-output fixture**
78
79	Create `tests/vrl/redact_emails_dd_agent_logs/expected.ndjson`. Emails in `.message` and `.attributes.user.email` should become `[REDACTED-EMAIL]`. Metadata fields (`ddtags`, `service`, `hostname`) should pass through unchanged. Note: `jq` canonicalizes object key order — write attributes alphabetically.
80
81	```json
82	{"ddtags": "env:prod,service:api", "hostname": "ip-10-0-1-23", "message": "user [REDACTED-EMAIL] signed in", "service": "api"}
83	{"ddtags": "env:prod", "hostname": "ip-10-0-1-24", "message": "no email here", "service": "worker"}
84	{"hostname": "ip-10-0-1-25", "message": "multiple [REDACTED-EMAIL] and [REDACTED-EMAIL]", "service": "api"}
85	{"attributes": {"request.id": "abc-123", "user.email": "[REDACTED-EMAIL]"}, "ddtags": "env:staging", "hostname": "ip-10-0-2-99", "message": "ok", "service": "auth"}
86	{"hostname": "ip-10-0-1-50", "message": "json blob {\"email\":\"[REDACTED-EMAIL]\"}", "service": "api"}
87	```
88
89	- [ ] **Step 3: Write the VRL program**
90
91	Create `tests/vrl/redact_emails_dd_agent_logs/program.vrl`. Identical logic to the existing `redact_emails_logs/program.vrl` — the transform is source-agnostic, but having a dedicated test file makes per-source assertions explicit.
92
93	```vrl
94	email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
95
96	if is_string(.message) {
97	  .message = replace(string!(.message), email_re, "[REDACTED-EMAIL]")
98	}
99
100	if is_object(.attributes) {
101	  attrs, _err = object(.attributes)
102	  if is_object(attrs) {
103	    .attributes = map_values(attrs) -> |v| {
104	      if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
105	    }
106	  }
107	}
108	.
109	```
110
111	- [ ] **Step 4: Write the runner script**
112
113	Create `tests/vrl/redact_emails_dd_agent_logs/run.sh` (mirrors siblings), then make it executable:
114
115	```bash
116	#!/usr/bin/env bash
117	set -euo pipefail
118	DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
119	ACTUAL="$DIR/.actual.ndjson"
120	vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o > "$ACTUAL"
121	diff -u <(jq -c . "$DIR/expected.ndjson") <(jq -c . "$ACTUAL")
122	echo "redact_emails_dd_agent_logs VRL OK"
123	```
124
125	Then:
126	```bash
127	chmod +x /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_dd_agent_logs/run.sh
128	```
129
130	- [ ] **Step 5: Run the test (expect: passes immediately, since the transform is identical to the existing one)**
131
132	```bash
133	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
134	./tests/vrl/redact_emails_dd_agent_logs/run.sh
135	```
136
137	Expected: prints `redact_emails_dd_agent_logs VRL OK`.
138
139	If diff fails, check object-key alphabetization in `expected.ndjson` (jq canonicalizes — keys must be sorted).
140
141	- [ ] **Step 6: Commit**
142
143	```bash
144	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
145	git add tests/vrl/redact_emails_dd_agent_logs/
146	git commit -m "test: add VRL test for DD-Agent-shape log redaction"
147	```
148
149	---
150
151	### Task A2: Wire the new VRL test into CI
152
153	**Files:**
154	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/.gitlab-ci.yml`
155
156	- [ ] **Step 1: Add the new test invocation to the `vrl-tests` job**
157
158	Open `.gitlab-ci.yml`, locate the `vrl-tests:` job's `script:` list, and append a fifth line:
159
160	```yaml
161	vrl-tests:
162	  stage: test
163	  image: registry.ic-ops.net/cache-docker-hub/timberio/vector:latest-debian
164	  before_script:
165	    - apt-get update -qq && apt-get install -y -qq bash diffutils jq
166	  script:
167	    - ./tests/vrl/drop_audit/run.sh
168	    - ./tests/vrl/redact_emails_logs/run.sh
169	    - ./tests/vrl/redact_emails_traces/run.sh
170	    - ./tests/vrl/redact_emails_metrics/run.sh
171	    - ./tests/vrl/redact_emails_dd_agent_logs/run.sh
172	```
173
174	- [ ] **Step 2: Commit**
175
176	```bash
177	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
178	git add .gitlab-ci.yml
179	git commit -m "ci: run new DD-Agent-shape VRL test"
180	```
181
182	---
183
184	### Task A3: Add the `dd_agent_in` source to the helm values template
185
186	**Files:**
187	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/templates/values.yaml.tpl`
188
189	- [ ] **Step 1: Add port 8282 to the service**
190
191	Locate the `service.ports:` block (around lines 5-14 of `templates/values.yaml.tpl`). Append a third port entry:
192
193	```yaml
194	service:
195	  enabled: true
196	  type: ClusterIP
197	  ports:
198	    - name: otlp-grpc
199	      port: 4317
200	      protocol: TCP
201	    - name: otlp-http
202	      port: 4318
203	      protocol: TCP
204	    - name: dd-agent
205	      port: 8282
206	      protocol: TCP
207	```
208
209	- [ ] **Step 2: Add the `dd_agent_in` source under `customConfig.sources`**
210
211	Locate the `customConfig.sources:` block (around lines 59-65). Add a sibling source after `otlp_in`:
212
213	```yaml
214	  sources:
215	    otlp_in:
216	      type: opentelemetry
217	      grpc:
218	        address: 0.0.0.0:4317
219	      http:
220	        address: 0.0.0.0:4318
221	    dd_agent_in:
222	      type: datadog_agent
223	      address: 0.0.0.0:8282
224	      multiple_outputs: true
225	      disable_metrics: true
226	      disable_traces: true
227	      store_api_key: false
228	      acknowledgements:
229	        enabled: true
230	```
231
232	`multiple_outputs: true` is required so downstream transforms can reference `dd_agent_in.logs` (the named-output suffix).
233
234	- [ ] **Step 3: Update `drop_audit.inputs` to be multi-input**
235
236	Locate the `drop_audit:` transform (around lines 67-73). Change its `inputs` list:
237
238	```yaml
239	    drop_audit:
240	      type: filter
241	      inputs:
242	        - otlp_in.logs
243	        - dd_agent_in.logs
244	      condition: |
245	        msg = if is_string(.message) { string!(.message) } else { "" }
246	        ${audit_drop_condition}
247	```
248
249	- [ ] **Step 4: Validate the example renders**
250
251	The template change is gated only by template variables already present in the consumer; no new template inputs added. Run validation:
252
253	```bash
254	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/examples/basic
255	terraform init -backend=false
256	terraform validate
257	```
258
259	Expected: `Success! The configuration is valid.`
260
261	If validate fails, check YAML indentation in the new source block (it must match the 4-space indent of the surrounding map).
262
263	- [ ] **Step 5: Commit**
264
265	```bash
266	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
267	git add templates/values.yaml.tpl
268	git commit -m "feat: add datadog_agent OPW source on port 8282"
269	```
270
271	---
272
273	### Task A4: Open netpol ingress for port 8282
274
275	**Files:**
276	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/netpol.tf`
277
278	- [ ] **Step 1: Add port 8282 to the existing cluster-from ingress rule**
279
280	Open `netpol.tf`. Locate the first ingress rule (the one with `fromEntities = ["cluster"]` and the OTLP ports list, around lines 12-21). Add a third port entry:
281
282	```hcl
283	      {
284	        # OTLP from any cluster-internal sender (primarily otel-collector).
285	        # Port 8282 is the OPW-compatible Datadog Agent intake (datadog_agent source).
286	        fromEntities = ["cluster"]
287	        toPorts = [{
288	          ports = [
289	            { port = "4317" },
290	            { port = "4318" },
291	            { port = "8282" },
292	          ]
293	        }]
294	      },
295	```
296
297	Do not add 8282 to the `host` (kubelet) ingress rule — kubelet doesn't probe this port.
298
299	- [ ] **Step 2: Validate**
300
301	```bash
302	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
303	terraform fmt -check -recursive
304	cd examples/basic
305	terraform init -backend=false
306	terraform validate
307	```
308
309	Expected: both commands pass cleanly.
310
311	- [ ] **Step 3: Commit**
312
313	```bash
314	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
315	git add netpol.tf
316	git commit -m "feat: open netpol ingress for OPW intake on port 8282"
317	```
318
319	---
320
321	### Task A5: Expose the new endpoint as a module output
322
323	**Files:**
324	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/outputs.tf`
325
326	- [ ] **Step 1: Add `service_endpoint_dd_agent` output**
327
328	Open `outputs.tf` and append:
329
330	```hcl
331	output "service_endpoint_dd_agent" {
332	  description = "OPW-compatible Datadog Agent intake endpoint (http, cluster-internal). Pass to datadog-agent module's logs_intake_url to redirect agent logs through Vector."
333	  value       = "http://vector.${var.namespace}.svc.cluster.local:8282"
334	}
335	```
336
337	- [ ] **Step 2: Validate**
338
339	```bash
340	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
341	terraform fmt -check -recursive
342	cd examples/basic
343	terraform init -backend=false
344	terraform validate
345	```
346
347	Expected: pass.
348
349	- [ ] **Step 3: Commit**
350
351	```bash
352	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
353	git add outputs.tf
354	git commit -m "feat: expose service_endpoint_dd_agent output"
355	```
356
357	---
358
359	### Task A6: Update Vector module README
360
361	**Files:**
362	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/README.md`
363
364	- [ ] **Step 1: Update the architecture description**
365
366	In `README.md`, replace the existing architecture paragraph (lines 14-32) with:
367
368	```markdown
369	Vector sits between the OTel collector tier and Datadog, and additionally
370	serves as an OPW-compatible logs intake for the Datadog Agent:
371
372	```
373	apps → otel-lb → otel-collector ──OTLP/gRPC──▶ vector ──▶ Datadog SaaS
374	                                                  ▲
375	                                          DD Agent (logs)
376	```
377
378	Vector runs as a Deployment (default 3 replicas) fronted by a ClusterIP
379	Service. It exposes:
380
381	- OTLP/gRPC on port 4317 (from OTel collector)
382	- OTLP/HTTP on port 4318 (from OTel collector)
383	- Datadog Agent intake on port 8282 (from DD Agent, OPW protocol)
384
385	Three signal pipelines run in parallel:
386
387	- **logs** — `drop_audit` filter (multi-input: OTLP logs + DD Agent logs;
388	  drops lines matching audit patterns) → `redact_emails_logs` remap →
389	  `datadog_logs` sink.
390	- **traces** — `redact_emails_traces` remap → `datadog_traces` sink.
391	- **metrics** — `redact_emails_metrics` remap → `datadog_metrics` sink.
392
393	The Datadog API key is sourced from Vault via the existing `external-secrets`
394	operator. The Deployment is annotated with `reloader.stakater.com/auto: "true"`
395	so key rotation triggers a rolling restart automatically.
396	```
397
398	- [ ] **Step 2: Add the new output to the Outputs table**
399
400	Locate the Outputs table (around lines 59-63). Append:
401
402	```markdown
403	| `service_endpoint_dd_agent` | OPW-compatible Datadog Agent intake endpoint (`http://vector.<namespace>.svc.cluster.local:8282`) |
404	```
405
406	- [ ] **Step 3: Update the example version pin**
407
408	Locate the Usage example (around line 73). Change the source ref from `?ref=v1.0.0` to `?ref=v1.1.0`.
409
410	- [ ] **Step 4: Add a usage hint for the DD Agent wire-up**
411
412	After the existing OTel-collector usage hint (around line 89-93), add:
413
414	```markdown
415	Point the Datadog Agent at Vector's OPW intake by passing the new endpoint to
416	the `datadog-agent` module (v1.2.0+):
417
418	```hcl
419	logs_intake_url = module.vector.service_endpoint_dd_agent
420	```
421	```
422
423	- [ ] **Step 5: Update the Testing section**
424
425	In the Testing section, add a fifth runner to the VRL test list (around line 109):
426
427	```sh
428	./tests/vrl/redact_emails_dd_agent_logs/run.sh
429	```
430
431	- [ ] **Step 6: Commit**
432
433	```bash
434	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
435	git add README.md
436	git commit -m "docs: document DD Agent OPW intake on port 8282"
437	```
438
439	---
440
441	### Task A7: Run all module tests + final validation
442
443	- [ ] **Step 1: Run all VRL tests**
444
445	```bash
446	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
447	./tests/vrl/drop_audit/run.sh
448	./tests/vrl/redact_emails_logs/run.sh
449	./tests/vrl/redact_emails_traces/run.sh
450	./tests/vrl/redact_emails_metrics/run.sh
451	./tests/vrl/redact_emails_dd_agent_logs/run.sh
452	```
453
454	Expected: each prints `<test_name> VRL OK` and exits 0.
455
456	- [ ] **Step 2: Terraform fmt + validate**
457
458	```bash
459	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector
460	terraform fmt -check -recursive
461	cd examples/basic
462	rm -rf .terraform .terraform.lock.hcl
463	terraform init -backend=false
464	terraform validate
465	```
466
467	Expected: fmt prints nothing, validate prints `Success!`.
468
469	- [ ] **Step 3: Sir Charles handles tagging**
470
471	Sir Charles tags `vector v1.1.0` and pushes (separate from this plan; not Claude's responsibility).
472
473	---
474
475	## Phase B: DD Agent module v1.2.0
476
477	### Task B1: Capture the current ("direct shipping") render as a baseline
478
479	**Files:**
480	- Create: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/tests/render/expected-direct.yaml`
481	- Create: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/tests/render/run.sh`
482
483	The point of this task is to lock in the current rendered output BEFORE making any changes. After Phase B's template change, rendering with `logs_intake_url = null` must produce byte-identical output.
484
485	- [ ] **Step 1: Create the test directory**
486
487	```bash
488	mkdir -p /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/tests/render
489	```
490
491	- [ ] **Step 2: Write a runner script that uses Terraform's templatefile() to render the template under fixed inputs**
492
493	Create `tests/render/run.sh`:
494
495	```bash
496	#!/usr/bin/env bash
497	# Renders values.yaml.tftpl under two input scenarios and diffs against the
498	# locked expected outputs. Catches accidental drift in the helm values output.
499	set -euo pipefail
500
501	DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
502	MODULE_DIR="$(cd "$DIR/../.." && pwd)"
503	TMP="$(mktemp -d)"
504	trap 'rm -rf "$TMP"' EXIT
505
506	render() {
507	  local intake_url="$1"
508	  local out="$2"
509
510	  # logs_intake_url is rendered as a literal "null" sentinel via jsonencode
511	  # when null; the template uses `!= null` to gate, so we pass the raw value
512	  # via terraform's tftpl_render harness.
513
514	  cat > "$TMP/main.tf" <<EOF
515	output "rendered" {
516	  value = templatefile("\${path.module}/values.yaml.tftpl", {
517	    cluster_name                    = "test-cluster"
518	    env                             = "nonprod"
519	    global_segment                  = "test"
520	    image_tag                       = "7.55.0"
521	    watch_namespace                 = "datadog"
522	    tags                            = ["env:test"]
523	    agents_priority_class_name      = "cloud-ops-system"
524	    apm_enabled                     = true
525	    container_exclude               = ""
526	    container_exclude_logs          = ""
527	    enable_mrds_monitoring          = false
528	    enable_nowrds_monitoring        = false
529	    enable_now_obfusicator_options  = false
530	    now_obfusicator_options         = { replace_digits = "true" }
531	    datadog_api_key_secret_name     = "datadog-api-key"
532	    cluster_agent_token_secret_name = "datadog-clusteragent-token"
533	    log_level                       = "ERROR"
534	    logs_intake_url                 = ${intake_url}
535	  })
536	}
537	EOF
538	  ln -sf "$MODULE_DIR/values.yaml.tftpl" "$TMP/values.yaml.tftpl"
539	  (cd "$TMP" && terraform init -backend=false -input=false >/dev/null && terraform apply -auto-approve -input=false >/dev/null)
540	  (cd "$TMP" && terraform output -raw rendered) > "$out"
541	  rm -rf "$TMP"/.terraform "$TMP"/terraform.tfstate*
542	}
543
544	render "null"                                                "$DIR/.actual-direct.yaml"
545	render '"http://vector.vector.svc.cluster.local:8282"'       "$DIR/.actual-via-vector.yaml"
546
547	diff -u "$DIR/expected-direct.yaml"     "$DIR/.actual-direct.yaml"
548	diff -u "$DIR/expected-via-vector.yaml" "$DIR/.actual-via-vector.yaml"
549
550	echo "datadog-agent values render OK"
551	```
552
553	Then make it executable:
554	```bash
555	chmod +x /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/tests/render/run.sh
556	```
557
558	- [ ] **Step 3: Generate the baseline `expected-direct.yaml` from the CURRENT (pre-change) template**
559
560	Before the template change lands, capture today's output as the locked baseline:
561
562	```bash
563	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
564	TMP="$(mktemp -d)"
565	ln -sf "$(pwd)/values.yaml.tftpl" "$TMP/values.yaml.tftpl"
566	cat > "$TMP/main.tf" <<'EOF'
567	output "rendered" {
568	  value = templatefile("${path.module}/values.yaml.tftpl", {
569	    cluster_name                    = "test-cluster"
570	    env                             = "nonprod"
571	    global_segment                  = "test"
572	    image_tag                       = "7.55.0"
573	    watch_namespace                 = "datadog"
574	    tags                            = ["env:test"]
575	    agents_priority_class_name      = "cloud-ops-system"
576	    apm_enabled                     = true
577	    container_exclude               = ""
578	    container_exclude_logs          = ""
579	    enable_mrds_monitoring          = false
580	    enable_nowrds_monitoring        = false
581	    enable_now_obfusicator_options  = false
582	    now_obfusicator_options         = { replace_digits = "true" }
583	    datadog_api_key_secret_name     = "datadog-api-key"
584	    cluster_agent_token_secret_name = "datadog-clusteragent-token"
585	    log_level                       = "ERROR"
586	  })
587	}
588	EOF
589	(cd "$TMP" && terraform init -backend=false -input=false >/dev/null && terraform apply -auto-approve -input=false >/dev/null)
590	(cd "$TMP" && terraform output -raw rendered) > tests/render/expected-direct.yaml
591	rm -rf "$TMP"
592	```
593
594	This produces `tests/render/expected-direct.yaml` containing the current rendered helm values (without the new template variable).
595
596	- [ ] **Step 4: Verify the baseline file is non-empty and looks right**
597
598	```bash
599	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
600	wc -l tests/render/expected-direct.yaml
601	head -5 tests/render/expected-direct.yaml
602	```
603
604	Expected: ~230 lines (the existing template is 234 lines), starts with `targetSystem: "linux"`.
605
606	- [ ] **Step 5: Commit the baseline + runner BEFORE the template change**
607
608	```bash
609	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
610	git add tests/render/run.sh tests/render/expected-direct.yaml
611	git commit -m "test: lock current values render as baseline before OPW change"
612	```
613
614	This commit is critical — it establishes the "before" state so the next task can verify the change is byte-identical when the new variable is null.
615
616	---
617
618	### Task B2: Add the `logs_intake_url` variable
619
620	**Files:**
621	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/variables.tf`
622
623	- [ ] **Step 1: Append the new variable to `variables.tf`**
624
625	Open `variables.tf` and append:
626
627	```hcl
628	variable "logs_intake_url" {
629	  description = "OPW-compatible logs intake URL. When set, the agent ships logs to this endpoint via DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL. For cluster-internal Vector use module.vector.service_endpoint_dd_agent. When null (default), logs ship direct to Datadog SaaS."
630	  type        = string
631	  default     = null
632
633	  validation {
634	    condition     = var.logs_intake_url == null || can(regex("^https?://", var.logs_intake_url))
635	    error_message = "logs_intake_url must be a URL starting with http:// or https://, or null."
636	  }
637	}
638	```
639
640	- [ ] **Step 2: Run terraform fmt**
641
642	```bash
643	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
644	terraform fmt variables.tf
645	```
646
647	Expected: no output (or it auto-formats and exits 0).
648
649	- [ ] **Step 3: Commit**
650
651	```bash
652	git add variables.tf
653	git commit -m "feat: add logs_intake_url variable for OPW redirection"
654	```
655
656	---
657
658	### Task B3: Wire the new variable into the templatefile call
659
660	**Files:**
661	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/main.tf`
662
663	- [ ] **Step 1: Add `logs_intake_url` to the templatefile arguments**
664
665	Open `main.tf`. Locate the `templatefile(...)` call inside `helm_release.this.values` (around lines 242-265). Add `logs_intake_url = var.logs_intake_url` to the arguments map. After the change, the templatefile block should look like:
666
667	```hcl
668	  values = [
669	    templatefile("${path.module}/values.yaml.tftpl", {
670	      cluster_name   = var.cluster_name
671	      env            = var.env == "staging" ? "nonprod" : var.env
672	      global_segment = var.global_segment
673	      image_tag      = var.image_tag
674	      watch_namespace          = var.create_namespace ? kubernetes_namespace.this[0].metadata[0].name : var.namespace
675	      tags                     = concat(["global_segment:${var.global_segment}"], var.datadog_tags)
676	      agents_priority_class_name = var.agents_priority_class_name
677
678	      apm_enabled = var.apm_enabled
679
680	      container_exclude = var.container_exclude
681	      container_exclude_logs = var.container_exclude_logs
682
683	      enable_mrds_monitoring   = var.enable_mrds_monitoring
684	      enable_nowrds_monitoring = var.enable_nowrds_monitoring
685	      enable_now_obfusicator_options = var.enable_now_obfusicator_options
686	      now_obfusicator_options = var.now_obfusicator_options
687
688	      datadog_api_key_secret_name = var.datadog_api_key_secret_name
689	      cluster_agent_token_secret_name = var.cluster_agent_token_secret_name
690	      log_level = var.log_level
691
692	      logs_intake_url = var.logs_intake_url
693	    })
694	  ]
695	```
696
697	- [ ] **Step 2: Validate (note: this will fail at apply time without a real provider, but `validate` should pass)**
698
699	```bash
700	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
701	terraform fmt -check
702	terraform init -backend=false -input=false 2>&1 | head -20 || true
703	terraform validate
704	```
705
706	Expected: `terraform validate` exits 0 with `Success!`. The `init` may complain about missing remote module sources — that's OK as long as `validate` passes.
707
708	- [ ] **Step 3: Commit**
709
710	```bash
711	git add main.tf
712	git commit -m "feat: pass logs_intake_url into helm values render"
713	```
714
715	---
716
717	### Task B4: Add the OPW env-var block to the values template
718
719	**Files:**
720	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/values.yaml.tftpl`
721
722	- [ ] **Step 1: Locate the `datadog.env` list**
723
724	In `values.yaml.tftpl`, find the `datadog.env:` block (lines 57-81). It currently ends with the `DD_APM_PROBABILISTIC_SAMPLER_SAMPLING_PERCENTAGE` entry on line 81.
725
726	- [ ] **Step 2: Append the gated OPW env vars**
727
728	After line 81 (the `value: 15` line for `DD_APM_PROBABILISTIC_SAMPLER_SAMPLING_PERCENTAGE`) and before `clusterAgent:` on line 82, insert:
729
730	```yaml
731	%{ if logs_intake_url != null ~}
732	    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED
733	      value: "true"
734	    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL
735	      value: "${logs_intake_url}"
736	%{ endif ~}
737	```
738
739	The 4-space indentation matches the surrounding `datadog.env` list items.
740
741	- [ ] **Step 3: Generate the "via-vector" expected render**
742
743	```bash
744	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
745	TMP="$(mktemp -d)"
746	ln -sf "$(pwd)/values.yaml.tftpl" "$TMP/values.yaml.tftpl"
747	cat > "$TMP/main.tf" <<'EOF'
748	output "rendered" {
749	  value = templatefile("${path.module}/values.yaml.tftpl", {
750	    cluster_name                    = "test-cluster"
751	    env                             = "nonprod"
752	    global_segment                  = "test"
753	    image_tag                       = "7.55.0"
754	    watch_namespace                 = "datadog"
755	    tags                            = ["env:test"]
756	    agents_priority_class_name      = "cloud-ops-system"
757	    apm_enabled                     = true
758	    container_exclude               = ""
759	    container_exclude_logs          = ""
760	    enable_mrds_monitoring          = false
761	    enable_nowrds_monitoring        = false
762	    enable_now_obfusicator_options  = false
763	    now_obfusicator_options         = { replace_digits = "true" }
764	    datadog_api_key_secret_name     = "datadog-api-key"
765	    cluster_agent_token_secret_name = "datadog-clusteragent-token"
766	    log_level                       = "ERROR"
767	    logs_intake_url                 = "http://vector.vector.svc.cluster.local:8282"
768	  })
769	}
770	EOF
771	(cd "$TMP" && terraform init -backend=false -input=false >/dev/null && terraform apply -auto-approve -input=false >/dev/null)
772	(cd "$TMP" && terraform output -raw rendered) > tests/render/expected-via-vector.yaml
773	rm -rf "$TMP"
774	```
775
776	- [ ] **Step 4: Verify the via-vector render contains the OPW env vars**
777
778	```bash
779	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
780	grep -A1 "DD_OBSERVABILITY_PIPELINES_WORKER" tests/render/expected-via-vector.yaml
781	```
782
783	Expected output:
784	```
785	    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED
786	      value: "true"
787	    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL
788	      value: "http://vector.vector.svc.cluster.local:8282"
789	```
790
791	- [ ] **Step 5: Run the render harness — both scenarios must pass**
792
793	```bash
794	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
795	./tests/render/run.sh
796	```
797
798	Expected: prints `datadog-agent values render OK`.
799
800	The "direct" diff must be **empty** (proving byte-identical when null). The "via-vector" diff must be empty (proving expected env vars present).
801
802	If the "direct" diff shows ANY change, your `%{ if ~}` directive isn't strip-newlining correctly — the template should produce zero output when `logs_intake_url == null`. Common cause: missing `~` on either delimiter.
803
804	- [ ] **Step 6: Commit**
805
806	```bash
807	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
808	git add values.yaml.tftpl tests/render/expected-via-vector.yaml
809	git commit -m "feat: render OPW env vars when logs_intake_url is set"
810	```
811
812	---
813
814	### Task B5: Update DD Agent README
815
816	**Files:**
817	- Modify: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/README.md`
818
819	- [ ] **Step 1: Add the `logs_intake_url` row to the Inputs table**
820
821	In `README.md`, locate the Inputs table (around lines 33-58). Find the row for `log_level` (alphabetical position) and insert immediately after it:
822
823	```markdown
824	| <a name="input_logs_intake_url"></a> [logs\_intake\_url](#input\_logs\_intake\_url) | OPW-compatible logs intake URL. When set, the agent ships logs to this endpoint via DD\_OBSERVABILITY\_PIPELINES\_WORKER\_LOGS\_URL. For cluster-internal Vector use module.vector.service\_endpoint\_dd\_agent. When null (default), logs ship direct to Datadog SaaS. | `string` | `null` | no |
825	```
826
827	- [ ] **Step 2: Add an "OPW Logs Intake" section after the Outputs table**
828
829	Append at the end of `README.md`:
830
831	```markdown
832
833	## OPW Logs Intake (optional)
834
835	By default, the agent ships logs directly to Datadog SaaS over HTTPS. To
836	redirect logs through a Datadog Observability Pipelines Worker (OPW), or
837	through a Vector deployment functioning as one, set `logs_intake_url`:
838
839	```hcl
840	module "datadog_agent" {
841	  # ...
842	  logs_intake_url = module.vector.service_endpoint_dd_agent
843	  # e.g. "http://vector.vector.svc.cluster.local:8282"
844	}
845	```
846
847	This sets two env vars on the agent containers:
848
849	- `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED=true`
850	- `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL=<logs_intake_url>`
851
852	Agent-side processing rules (`DD_LOGS_CONFIG_PROCESSING_RULES`) continue to
853	execute before transmission, regardless of destination. Metrics and APM
854	(traces) are unaffected and continue shipping direct to Datadog.
855
856	When `logs_intake_url` is `null`, the rendered helm values are byte-identical
857	to the v1.1.x line — there is no behavior change for clusters that do not opt in.
858
859	Reference: <https://docs.datadoghq.com/observability_pipelines/sources/datadog_agent/>
860	```
861
862	- [ ] **Step 3: Commit**
863
864	```bash
865	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
866	git add README.md
867	git commit -m "docs: document logs_intake_url for OPW redirection"
868	```
869
870	---
871
872	### Task B6: Final module validation
873
874	- [ ] **Step 1: Run the full render harness**
875
876	```bash
877	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
878	./tests/render/run.sh
879	```
880
881	Expected: `datadog-agent values render OK`.
882
883	- [ ] **Step 2: terraform fmt**
884
885	```bash
886	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
887	terraform fmt -check -recursive
888	```
889
890	Expected: no output, exit 0.
891
892	- [ ] **Step 3: terraform validate**
893
894	```bash
895	cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent
896	terraform init -backend=false -input=false 2>&1 | tail -5 || true
897	terraform validate
898	```
899
900	Expected: `Success!`
901
902	- [ ] **Step 4: Sir Charles handles tagging**
903
904	Sir Charles tags `datadog-agent v1.2.0` and pushes (separate from this plan).
905
906	---
907
908	## Phase C: Consumer (eks-cluster-applications) wire-up
909
910	### Task C1: Add the `vector_dd_agent_logs_enabled` flag
911
912	**Files:**
913	- Modify: `/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/datadog_agent.tf`
914
915	- [ ] **Step 1: Add the variable, local, and plan-time guard**
916
917	Open `datadog_agent.tf`. At the bottom of the file (after the existing `variable "datadog_agent_enabled"` block, around line 117), append:
918
919	```hcl
920	# ---------------------------------------------------------------------------------------------------------------------
921	# DD AGENT → VECTOR LOGS ROUTING (OPW)
922	# ---------------------------------------------------------------------------------------------------------------------
923	locals {
924	  vector_dd_agent_logs_enabled = (
925	    var.vector_dd_agent_logs_enabled != null
926	    ? var.vector_dd_agent_logs_enabled
927	    : local.vector_enabled
928	  )
929	}
930
931	variable "vector_dd_agent_logs_enabled" {
932	  description = "Override for routing DD Agent logs through Vector. When null (default), inherits from local.vector_enabled (i.e., on wherever Vector is on). Set to false to keep DD Agent shipping logs directly to DD SaaS even when Vector is deployed."
933	  type        = bool
934	  default     = null
935	}
936
937	# Plan-time guard: vector_dd_agent_logs_enabled cannot be true if vector itself is off.
938	# Mirrors the existing data "external" "vector_validate" pattern in vector.tf.
939	data "external" "vector_dd_agent_logs_validate" {
940	  count = local.vector_dd_agent_logs_enabled && !local.vector_enabled ? 1 : 0
941
942	  program = ["Error: vector_dd_agent_logs_enabled requires vector_enabled"]
943	}
944	```
945
946	- [ ] **Step 2: Run terraform fmt**
947
948	```bash
949	cd /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications
950	terraform fmt datadog_agent.tf
951	```
952
953	- [ ] **Step 3: Commit**
954
955	```bash
956	git add datadog_agent.tf
957	git commit -m "feat: add vector_dd_agent_logs_enabled flag with vector_enabled inheritance"
958	```
959
960	---
961
962	### Task C2: Bump the `module.datadog_agent` ref and pass `logs_intake_url`
963
964	**Files:**
965	- Modify: `/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/datadog_agent.tf`
966
967	- [ ] **Step 1: Bump the module source ref**
968
969	In `datadog_agent.tf`, line 14, change:
970
971	```hcl
972	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/datadog-agent?ref=v1.1.2"
973	```
974
975	to:
976
977	```hcl
978	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/datadog-agent?ref=v1.2.0"
979	```
980
981	- [ ] **Step 2: Add the `logs_intake_url` argument to the module call**
982
983	In the same `module "datadog_agent"` block (around lines 13-34), add `logs_intake_url` as the last argument before the closing `}`:
984
985	```hcl
986	module "datadog_agent" {
987	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/datadog-agent?ref=v1.2.0"
988	  count  = local.datadog_agent_enabled ? 1 : 0
989
990	  providers = {
991	    kubernetes = kubernetes.local-cluster
992	    helm       = helm.local-cluster
993	  }
994
995	  chart_version                  = var.datadog_agent_version
996	  image_tag                      = var.datadog_agent_image_tag
997	  create_namespace               = var.datadog_create_namespace
998	  cluster_name                   = var.cluster_name
999	  global_segment                 = var.global_segment
1000	  datadog_tags                   = var.datadog_tags
1001	  enable_nowrds_monitoring       = var.datadog_enable_nowrds_monitoring
1002	  enable_now_obfusicator_options = var.datadog_enable_now_obfusicator_options
1003	  enable_mrds_monitoring         = var.datadog_enable_mrds_monitoring
1004	  now_obfusicator_options        = var.datadog_now_obfusicator_options
1005	  tags                           = var.tags
1006	  env                            = var.env
1007
1008	  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
1009	}
1010	```
1011
1012	The ternary handles the `count = 0` case correctly: when `local.vector_dd_agent_logs_enabled = false`, the right-hand side evaluates to `null` without dereferencing `module.vector[0]`.
1013
1014	- [ ] **Step 3: Run terraform fmt**
1015
1016	```bash
1017	cd /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications
1018	terraform fmt datadog_agent.tf
1019	```
1020
1021	- [ ] **Step 4: Commit**
1022
1023	```bash
1024	git add datadog_agent.tf
1025	git commit -m "feat: route DD Agent logs through Vector when enabled"
1026	```
1027
1028	---
1029
1030	### Task C3: Bump `module.vector` ref to v1.1.0
1031
1032	**Files:**
1033	- Modify: `/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/vector.tf`
1034
1035	- [ ] **Step 1: Replace the local-path source with the tagged git ref**
1036
1037	In `vector.tf`, lines 14-15 currently read:
1038
1039	```hcl
1040	  #source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/vector.git?ref=v1.0.0"
1041	  source = "/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector"
1042	```
1043
1044	Change to:
1045
1046	```hcl
1047	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/vector.git?ref=v1.1.0"
1048	```
1049
1050	(Removes the local-path source and the commented-out v1.0.0 line; replaces with the v1.1.0 tagged git ref. The local-path source was only present during development, per the prior session note.)
1051
1052	- [ ] **Step 2: Run terraform fmt**
1053
1054	```bash
1055	cd /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications
1056	terraform fmt vector.tf
1057	```
1058
1059	- [ ] **Step 3: Commit**
1060
1061	```bash
1062	git add vector.tf
1063	git commit -m "chore: pin vector module to v1.1.0 (OPW intake)"
1064	```
1065
1066	---
1067
1068	### Task C4: Update consumer README
1069
1070	**Files:**
1071	- Modify: `/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/README.md`
1072
1073	- [ ] **Step 1: Read the existing inputs table**
1074
1075	```bash
1076	grep -n "vector_email_redaction_enabled\|vector_enabled\|input_vector" /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/README.md | head -30
1077	```
1078
1079	This locates the Vector-related inputs table rows so the new entry slots in alphabetically.
1080
1081	- [ ] **Step 2: Add the new input row**
1082
1083	In `README.md`, in the inputs table where Vector variables are listed, insert (alphabetically — between `vector_chart_version` and `vector_email_redaction_enabled` or wherever it sorts):
1084
1085	```markdown
1086	| <a name="input_vector_dd_agent_logs_enabled"></a> [vector\_dd\_agent\_logs\_enabled](#input\_vector\_dd\_agent\_logs\_enabled) | Override for routing DD Agent logs through Vector. When null (default), inherits from local.vector\_enabled (i.e., on wherever Vector is on). Set to false to keep DD Agent shipping logs directly to DD SaaS even when Vector is deployed. | `bool` | `null` | no |
1087	```
1088
1089	- [ ] **Step 3: Add a paragraph about dual-layer log filtering**
1090
1091	Find the existing Vector-related section in the README (added in the prior session per the spec) and append a new paragraph:
1092
1093	```markdown
1094
1095	### DD Agent → Vector log routing (OPW)
1096
1097	When `vector_dd_agent_logs_enabled` is true (defaults to `local.vector_enabled`),
1098	the Datadog Agent ships container logs to Vector via the OPW HTTP intake on
1099	`vector.vector.svc.cluster.local:8282`. Logs traverse two filtering layers
1100	before reaching Datadog SaaS:
1101
1102	1. **Agent-side** — `DD_LOGS_CONFIG_PROCESSING_RULES` (see
1103	   `datadog-agent/processing-rules/{env}/rules.json`) drops audit lines before
1104	   transmission. Authoritative for audit drops.
1105	2. **Vector** — `drop_audit` regex filter (defense-in-depth, same patterns) +
1106	   `redact_emails_logs` substitutes email addresses with `[REDACTED-EMAIL]`.
1107
1108	Metrics and APM (traces) are unaffected and continue their existing OTel-based
1109	path through Vector. To roll back logs to direct DD shipping for a single
1110	cluster while keeping Vector traces, set `vector_dd_agent_logs_enabled = false`
1111	on that cluster — Vector continues serving OTel traffic, agent re-rolls back to
1112	direct logs.
1113	```
1114
1115	- [ ] **Step 4: Commit**
1116
1117	```bash
1118	git add README.md
1119	git commit -m "docs: document vector_dd_agent_logs_enabled and dual-layer log filtering"
1120	```
1121
1122	---
1123
1124	### Task C5: Final consumer validation
1125
1126	- [ ] **Step 1: terraform fmt**
1127
1128	```bash
1129	cd /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications
1130	terraform fmt -check -recursive
1131	```
1132
1133	Expected: no output, exit 0.
1134
1135	- [ ] **Step 2: terraform validate (best-effort, may need cluster auth)**
1136
1137	```bash
1138	cd /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications
1139	terraform init -backend=false -input=false 2>&1 | tail -10 || true
1140	terraform validate
1141	```
1142
1143	Expected: `Success!`. If `init` cannot fetch the new module versions (because Sir Charles hasn't pushed the tags yet), `validate` will fail with module-not-found; that's expected pre-tag and Sir Charles will re-run after tagging.
1144
1145	- [ ] **Step 3: Note for Sir Charles**
1146
1147	Once `vector v1.1.0` and `datadog-agent v1.2.0` are tagged + pushed, re-run:
1148	```bash
1149	cd /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications
1150	rm -rf .terraform .terraform.lock.hcl
1151	terraform init -backend=false
1152	terraform validate
1153	```
1154
1155	To confirm clean resolution against the real tags.
1156
1157	---
1158
1159	## Phase D: Deploy verification (post-rollout, ad-hoc)
1160
1161	These steps run against a real cluster after the three repos are tagged, MRs merged, and `terraform apply` has rolled the change out. They are **not** committable steps — they are runbook entries.
1162
1163	### Task D1: Verify DD Agent ingests via OPW path (smoke)
1164
1165	- [ ] **Step 1: Confirm the new env vars are live on agent pods**
1166
1167	```bash
1168	kubectl exec -n datadog ds/immuta-dd -c agent -- env | grep DD_OBSERVABILITY_PIPELINES_WORKER
1169	```
1170
1171	Expected:
1172	```
1173	DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED=true
1174	DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL=http://vector.vector.svc.cluster.local:8282
1175	```
1176
1177	If empty: the helm release didn't roll, or `vector_dd_agent_logs_enabled` is false. Check `terraform plan` output and `kubectl describe ds/immuta-dd`.
1178
1179	- [ ] **Step 2: Confirm Vector is receiving DD Agent traffic**
1180
1181	```bash
1182	kubectl exec -n vector deploy/vector -- curl -s http://localhost:8686/metrics \
1183	  | grep 'vector_component_received_events_total{component_id="dd_agent_in"'
1184	```
1185
1186	Expected: a non-zero counter that increases over time.
1187
1188	If zero: check Vector pod logs for source-startup errors (`kubectl logs -n vector deploy/vector | grep dd_agent_in`).
1189
1190	### Task D2: End-to-end redaction verification
1191
1192	- [ ] **Step 1: Emit an email-tagged log line from a test pod**
1193
1194	```bash
1195	UUID="dd-redact-$(uuidgen)"
1196	kubectl run "${UUID}" --image=busybox --restart=Never --rm -it \
1197	  -- sh -c "echo '${UUID}: contact alice@example.com'; sleep 5"
1198	```
1199
1200	- [ ] **Step 2: Search Datadog Logs UI**
1201
1202	Query: `${UUID}` (the test marker). Within ~30s the matching log should appear.
1203
1204	Expected: log message reads `${UUID}: contact [REDACTED-EMAIL]` (not the original `alice@example.com`).
1205
1206	If unredacted: check `vector_component_errors_total{component_id="redact_emails_logs"}` and the email regex against the actual log shape.
1207
1208	- [ ] **Step 3: Verify metadata preservation**
1209
1210	In the same DD Logs UI result, verify these facets are populated:
1211	- `service` (should match the test pod's service tag, often the pod name)
1212	- `host` (the EC2 instance / node)
1213	- `kubernetes.namespace_name` (= `default`)
1214	- `kubernetes.pod_name` (= `${UUID}`)
1215
1216	If missing: the `datadog_agent` source's `multiple_outputs` may be dropping fields. Check Vector pod logs.
1217
1218	### Task D3: Audit-drop verification
1219
1220	- [ ] **Step 1: Emit an audit-shape log line**
1221
1222	```bash
1223	UUID2="dd-audit-$(uuidgen)"
1224	kubectl run "${UUID2}" --image=busybox --restart=Never --rm -it \
1225	  -- sh -c "echo 'Audit - test ${UUID2}'; sleep 5"
1226	```
1227
1228	- [ ] **Step 2: Confirm zero results in Datadog**
1229
1230	Query: `${UUID2}`. Expected: zero results (dropped at the agent before transmission to Vector).
1231
1232	- [ ] **Step 3: Confirm Vector did NOT receive it**
1233
1234	```bash
1235	kubectl exec -n vector deploy/vector -- curl -s http://localhost:8686/metrics \
1236	  | grep 'vector_component_received_events_total{component_id="dd_agent_in"'
1237	```
1238
1239	Note the value. Wait 30s, re-run. The counter should NOT have incremented for this audit line (since the agent dropped it before transmission).
1240
1241	If the counter ticked up: agent-side processing rules aren't matching the new audit pattern. Check `kubectl get cm -n datadog rules-config -o yaml`.
1242
1243	### Task D4: Rollback drill
1244
1245	- [ ] **Step 1: Flip the flag false on a non-prod cluster**
1246
1247	In the cluster's instance directory, set `vector_dd_agent_logs_enabled = false`. Run `terraform apply`.
1248
1249	- [ ] **Step 2: Confirm env vars removed from agent**
1250
1251	```bash
1252	kubectl exec -n datadog ds/immuta-dd -c agent -- env | grep DD_OBSERVABILITY_PIPELINES_WORKER || echo "no OPW env vars"
1253	```
1254
1255	Expected: prints `no OPW env vars`.
1256
1257	- [ ] **Step 3: Re-run the redaction smoke test from Task D2**
1258
1259	Expected: log appears in DD UI with the email address NOT redacted (proving the agent reverted to direct shipping).
1260
1261	- [ ] **Step 4: Flip the flag back on**
1262
1263	`vector_dd_agent_logs_enabled = null` (or `true`), `terraform apply`. Re-run Task D2 step 2 — redaction should be active again.
1264
1265	---
1266
1267	## Self-Review
1268
1269	**Spec coverage check** (each numbered subsection of the spec → mapped task):
1270
1271	| Spec section | Implementing task(s) |
1272	|---|---|
1273	| §3.1 Pipeline before/after | Phase A + B (mechanics), Phase C (gating) |
1274	| §3.2 Module split | Phase A (Vector), B (DD Agent), C (consumer) |
1275	| §4.1 Vector module deltas — `service.ports` | A3 step 1 |
1276	| §4.1 Vector module deltas — `dd_agent_in` source | A3 step 2 |
1277	| §4.1 Vector module deltas — `drop_audit.inputs` | A3 step 3 |
1278	| §4.1 netpol port 8282 | A4 |
1279	| §4.1 outputs | A5 |
1280	| §4.1 README | A6 |
1281	| §4.2 DD Agent module — `logs_intake_url` variable | B2 |
1282	| §4.2 DD Agent module — values template | B4 |
1283	| §4.2 DD Agent module — main.tf wiring | B3 |
1284	| §4.2 DD Agent module — README | B5 |
1285	| §4.3 Consumer — `vector_dd_agent_logs_enabled` flag | C1 |
1286	| §4.3 Consumer — module ref bumps | C2, C3 |
1287	| §4.3 Consumer — wire `logs_intake_url` | C2 |
1288	| §4.3 Consumer — README | C4 |
1289	| §6.1 Module input validation — `logs_intake_url` regex | B2 |
1290	| §6.1 Module input validation — consumer guard | C1 |
1291	| §7.1 Unit tests — VRL | A1 |
1292	| §7.1 Unit tests — DD Agent template render | B1, B4 |
1293	| §7.1 Unit tests — terraform fmt/validate | A7, B6, C5 |
1294	| §7.2 Integration verification | D1, D2, D3 |
1295	| §7.3 Cross-cutting — metadata preservation | D2 step 3 |
1296	| §7.3 Cross-cutting — JSON-shape redaction | (covered by A1 fixture line 5) |
1297	| §7.4 Soak | (Sir Charles's operational responsibility, not implementation) |
1298
1299	No gaps.
1300
1301	**Type/name consistency check:**
1302	- Variable name `logs_intake_url` consistent across DD Agent module + consumer wire-up.
1303	- Variable name `vector_dd_agent_logs_enabled` consistent across consumer var + local + guard.
1304	- Output name `service_endpoint_dd_agent` consistent across Vector module + consumer wire-up.
1305	- Port `8282` consistent across Vector service.ports + dd_agent_in.address + netpol + output URL + DD Agent docs/README.
1306	- Module versions: `vector v1.1.0` and `datadog-agent v1.2.0` consistent across spec + plan + consumer ref bumps.
1307


Now let me read the actual implementation files across all three repos.


Bash

List Vector module root


List Vector VRL test dirs


drwxr-xr-x steve.briggs staff 224 B  Tue May  5 12:32:56 2026 ..
.rw-r--r-- steve.briggs staff 242 B  Tue May  5 12:35:28 2026 versions.tf
.rw-r--r-- steve.briggs staff 136 B  Tue May  5 12:35:44 2026 .gitignore
drwxr-xr-x steve.briggs staff 128 B  Tue May  5 12:52:54 2026 tests
drwxr-xr-x steve.briggs staff  96 B  Tue May  5 13:46:52 2026 examples
drwxr-xr-x steve.briggs staff 256 B  Tue May  5 15:50:48 2026 .idea
.rw-r--r-- steve.briggs staff 2.9 KB Tue May  5 15:51:44 2026 variables.tf
.rw-r--r-- steve.briggs staff 2.8 KB Tue May  5 16:40:02 2026 main.tf
drwxr-xr-x steve.briggs staff 128 B  Tue May  5 16:40:10 2026 .terraform
.rw-r--r-- steve.briggs staff 2.2 KB Tue May  5 16:40:55 2026 .terraform.lock.hcl
drwxr-xr-x steve.briggs staff  96 B  Tue May  5 17:34:03 2026 .claude
.rw-r--r-- steve.briggs staff 798 B  Wed May  6 12:26:50 2026 .gitlab-ci.yml
drwxr-xr-x steve.briggs staff  96 B  Wed May  6 12:29:50 2026 templates
.rw-r--r-- steve.briggs staff 1.4 KB Wed May  6 12:38:51 2026 netpol.tf
.rw-r--r-- steve.briggs staff 748 B  Wed May  6 12:42:42 2026 outputs.tf
.rw-r--r-- steve.briggs staff 5.3 KB Wed May  6 12:45:41 2026 README.md
drwxr-xr-x steve.briggs staff 544 B  Wed May  6 12:45:41 2026 .


drwxr-xr-x steve.briggs staff 128 B Tue May  5 12:52:54 2026 ..
drwxr-xr-x steve.briggs staff 224 B Tue May  5 12:59:12 2026 drop_audit
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:20 2026 redact_emails_logs
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:24 2026 redact_emails_traces
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:28 2026 redact_emails_metrics
drwxr-xr-x steve.briggs staff 224 B Wed May  6 12:20:38 2026 .
drwxr-xr-x steve.briggs staff 224 B Wed May  6 12:21:34 2026 redact_emails_dd_agent_logs


List DD Agent module root


List DD Agent render test dir


.rw-r--r-- steve.briggs staff 494 B  Mon Feb  9 15:05:23 2026 output.tf
.rw-r--r-- steve.briggs staff 7.5 KB Mon Feb  9 16:48:12 2026 datadog-app.yaml
drwxr-xr-x steve.briggs staff 128 B  Thu Feb 12 17:09:38 2026 processing-rules
.rw-r--r-- steve.briggs staff 144 B  Thu Feb 26 11:23:08 2026 AGENTS.md
.rw-r--r-- steve.briggs staff  49 B  Thu Feb 26 12:26:57 2026 CLAUDE.MD
.rw-r--r-- steve.briggs staff 3.0 KB Thu Feb 26 12:43:41 2026 MEMORIES.md
.rw-r--r-- steve.briggs staff 2.0 KB Fri Mar  6 14:23:59 2026 netpols.tf
.rw-r--r-- steve.briggs staff  70 B  Wed Apr 15 19:26:08 2026 .gitignore
drwxr-xr-x steve.briggs staff 320 B  Wed Apr 22 12:42:10 2026 .idea
drwxr-xr-x steve.briggs staff 224 B  Tue May  5 12:32:56 2026 ..
drwxr-xr-x steve.briggs staff 448 B  Wed May  6 11:11:05 2026 .git
drwxr-xr-x steve.briggs staff  96 B  Wed May  6 12:49:11 2026 tests
.rw-r--r-- steve.briggs staff 3.8 KB Wed May  6 12:56:53 2026 variables.tf
.rw-r--r-- steve.briggs staff 9.5 KB Wed May  6 13:01:20 2026 main.tf
.rw-r--r-- steve.briggs staff 6.5 KB Wed May  6 13:05:12 2026 values.yaml.tftpl
.rw-r--r-- steve.briggs staff 7.5 KB Wed May  6 13:11:18 2026 README.md
drwxr-xr-x steve.briggs staff 544 B  Wed May  6 13:11:18 2026 .


drwxr-xr-x steve.briggs staff  96 B  Wed May  6 12:49:11 2026 ..
.rw-r--r-- steve.briggs staff 3.5 KB Wed May  6 12:49:26 2026 expected-direct.yaml
.rwxr-xr-x steve.briggs staff 2.0 KB Wed May  6 12:55:25 2026 run.sh
.rw-r--r-- steve.briggs staff 3.7 KB Wed May  6 13:06:08 2026 expected-via-vector.yaml
drwxr-xr-x steve.briggs staff 224 B  Wed May  6 13:06:52 2026 .
.rw-r--r-- steve.briggs staff 3.5 KB Wed May  6 13:12:05 2026 .actual-direct.yaml
.rw-r--r-- steve.briggs staff 3.7 KB Wed May  6 13:12:05 2026 .actual-via-vector.yaml


List consumer root


drwxr-xr-x steve.briggs staff 352 B  Fri Mar  6 12:13:52 2026 ..
.rw-r--r-- steve.briggs staff 3.0 KB Fri Mar  6 12:13:53 2026 .gitignore
.rw-r--r-- steve.briggs staff 4.0 KB Fri Mar  6 12:13:53 2026 application_deployment.tf
.rw-r--r-- steve.briggs staff 6.1 KB Fri Mar  6 12:13:53 2026 argocd_mgmt.tf
.rw-r--r-- steve.briggs staff 4.0 KB Fri Mar  6 12:13:53 2026 cluster_autoscaler.tf
.rw-r--r-- steve.briggs staff 2.7 KB Fri Mar  6 12:13:53 2026 cnpg_operator.tf
.rw-r--r-- steve.briggs staff 7.1 KB Fri Mar  6 12:13:53 2026 common.tf
.rw-r--r-- steve.briggs staff 2.2 KB Fri Mar  6 12:13:53 2026 coredns.tf
.rw-r--r-- steve.briggs staff 2.3 KB Fri Mar  6 12:13:53 2026 ebs_csi.tf
.rw-r--r-- steve.briggs staff 3.4 KB Fri Mar  6 12:13:53 2026 external_secrets.tf
.rw-r--r-- steve.briggs staff 1.9 KB Fri Mar  6 12:13:53 2026 kube_proxy.tf
.rw-r--r-- steve.briggs staff 5.4 KB Fri Mar  6 12:13:53 2026 kyverno.tf
.rw-r--r-- steve.briggs staff 2.5 KB Fri Mar  6 12:13:53 2026 metrics_server.tf
.rw-r--r-- steve.briggs staff 2.8 KB Fri Mar  6 12:13:53 2026 netpol.tf
.rw-r--r-- steve.briggs staff 2.7 KB Fri Mar  6 12:13:53 2026 reloader.tf
.rw-r--r-- steve.briggs staff 5.2 KB Fri Mar  6 12:13:53 2026 s3_csi.tf
.rw-r--r-- steve.briggs staff 3.3 KB Fri Mar  6 12:13:53 2026 vault_sidecar_injector.tf
drwxr-xr-x steve.briggs staff 256 B  Fri Mar  6 16:17:40 2026 .idea
.rw-r--r-- steve.briggs staff 9.1 KB Wed Apr  8 11:01:45 2026 cert_manager.tf
.rw-r--r-- steve.briggs staff  14 KB Wed Apr  8 11:01:45 2026 cilium.tf
.rw-r--r-- steve.briggs staff 6.6 KB Wed Apr  8 11:01:45 2026 external_dns.tf
.rw-r--r-- steve.briggs staff 6.7 KB Wed Apr  8 11:01:45 2026 teleport.tf
.rw-r--r-- steve.briggs staff  10 KB Tue May  5 12:42:37 2026 aws_loadbalancer.tf
.rw-r--r-- steve.briggs staff 6.7 KB Tue May  5 12:42:37 2026 external_networking.tf
.rw-r--r-- steve.briggs staff 3.6 KB Tue May  5 12:42:37 2026 fluxcd.tf
.rw-r--r-- steve.briggs staff 8.0 KB Tue May  5 12:42:37 2026 ingress_controller.tf
drwxr-xr-x steve.briggs staff 128 B  Tue May  5 14:52:45 2026 .terraform
.rw-r--r-- steve.briggs staff 7.6 KB Tue May  5 14:53:12 2026 .terraform.lock.hcl
drwxr-xr-x steve.briggs staff 512 B  Tue May  5 14:54:27 2026 .git
.rw-r--r-- steve.briggs staff 3.9 KB Tue May  5 15:23:22 2026 otel_collector.tf
.rw-r--r-- steve.briggs staff 4.7 KB Tue May  5 16:23:03 2026 sysdig_agent.tf
.rw-r--r-- steve.briggs staff 5.3 KB Wed May  6 13:14:21 2026 datadog_agent.tf
.rw-r--r-- steve.briggs staff 6.0 KB Wed May  6 13:14:49 2026 vector.tf
.rw-r--r-- steve.briggs staff  74 KB Wed May  6 13:15:43 2026 README.md
drwxr-xr-x steve.briggs staff 1.1 KB Wed May  6 13:15:43 2026 .
drwxr-xr-x steve.briggs staff 160 B  Wed May  6 13:17:14 2026 .claude


Now let me read all the critical files in parallel.


1	role: Aggregator
2	replicas: ${replicas}
3	podAnnotations:
4	  reloader.stakater.com/auto: "true"
5	service:
6	  enabled: true
7	  type: ClusterIP
8	  ports:
9	    - name: otlp-grpc
10	      port: 4317
11	      protocol: TCP
12	    - name: otlp-http
13	      port: 4318
14	      protocol: TCP
15	    - name: dd-agent
16	      port: 8282
17	      protocol: TCP
18	env:
19	  - name: DATADOG_API_KEY
20	    valueFrom:
21	      secretKeyRef:
22	        name: vector-datadog
23	        key: api_key
24	resources:
25	  requests:
26	    cpu: ${requests_cpu}
27	    memory: ${requests_memory}
28	  limits:
29	    cpu: "${limits_cpu}"
30	    memory: ${limits_memory}
31	podDisruptionBudget:
32	  enabled: true
33	  minAvailable: 2
34
35	topologySpreadConstraints:
36	  - maxSkew: 1
37	    topologyKey: topology.kubernetes.io/zone
38	    whenUnsatisfiable: ScheduleAnyway
39	    labelSelector:
40	      matchLabels:
41	        app.kubernetes.io/name: vector
42
43	affinity:
44	  podAntiAffinity:
45	    preferredDuringSchedulingIgnoredDuringExecution:
46	      - weight: 100
47	        podAffinityTerm:
48	          topologyKey: kubernetes.io/hostname
49	          labelSelector:
50	            matchLabels:
51	              app.kubernetes.io/name: vector
52
53	%{ if service_monitor_enabled ~}
54	serviceMonitor:
55	  enabled: true
56	%{ endif ~}
57	customConfig:
58	  data_dir: /vector-data-dir
59	  api:
60	    enabled: true
61	    address: 127.0.0.1:8686
62	  sources:
63	    otlp_in:
64	      type: opentelemetry
65	      grpc:
66	        address: 0.0.0.0:4317
67	      http:
68	        address: 0.0.0.0:4318
69	    dd_agent_in:
70	      type: datadog_agent
71	      address: 0.0.0.0:8282
72	      multiple_outputs: true
73	      disable_metrics: true
74	      disable_traces: true
75	      store_api_key: false
76	      acknowledgements:
77	        enabled: true
78	  transforms:
79	    drop_audit:
80	      type: filter
81	      inputs:
82	        - otlp_in.logs
83	        - dd_agent_in.logs
84	      condition: |
85	        msg = if is_string(.message) { string!(.message) } else { "" }
86	        ${audit_drop_condition}
87	%{ if email_redaction_enabled ~}
88	    redact_emails_logs:
89	      type: remap
90	      inputs:
91	        - drop_audit
92	      source: |
93	        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
94	        if is_string(.message) {
95	          .message = replace(string!(.message), email_re, "${email_redaction_replacement}")
96	        }
97	        if is_object(.attributes) {
98	          attrs, _err = object(.attributes)
99	          if is_object(attrs) {
100	            .attributes = map_values(attrs) -> |v| {
101	              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
102	            }
103	          }
104	        }
105	    redact_emails_traces:
106	      type: remap
107	      inputs:
108	        - otlp_in.traces
109	      source: |
110	        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
111	        if is_object(.attributes) {
112	          attrs, _err = object(.attributes)
113	          if is_object(attrs) {
114	            .attributes = map_values(attrs) -> |v| {
115	              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
116	            }
117	          }
118	        }
119	        if is_object(.resource) {
120	          res, _err = object(.resource)
121	          if is_object(res) && is_object(res.attributes) {
122	            rattrs, _err2 = object(res.attributes)
123	            if is_object(rattrs) {
124	              .resource.attributes = map_values(rattrs) -> |v| {
125	                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
126	              }
127	            }
128	          }
129	        }
130	    redact_emails_metrics:
131	      type: remap
132	      inputs:
133	        - otlp_in.metrics
134	      source: |
135	        email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
136	        if is_object(.attributes) {
137	          attrs, _err = object(.attributes)
138	          if is_object(attrs) {
139	            .attributes = map_values(attrs) -> |v| {
140	              if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
141	            }
142	          }
143	        }
144	        if is_object(.resource) {
145	          res, _err = object(.resource)
146	          if is_object(res) && is_object(res.attributes) {
147	            rattrs, _err2 = object(res.attributes)
148	            if is_object(rattrs) {
149	              .resource.attributes = map_values(rattrs) -> |v| {
150	                if is_string(v) { replace(string!(v), email_re, "${email_redaction_replacement}") } else { v }
151	              }
152	            }
153	          }
154	        }
155	%{ endif ~}
156	  sinks:
157	    datadog_logs:
158	      type: datadog_logs
159	      inputs:
160	        - ${email_redaction_enabled ? "redact_emails_logs" : "drop_audit"}
161	      default_api_key: $${DATADOG_API_KEY}
162	      site: ${datadog_site}
163	    datadog_traces:
164	      type: datadog_traces
165	      inputs:
166	        - ${email_redaction_enabled ? "redact_emails_traces" : "otlp_in.traces"}
167	      default_api_key: $${DATADOG_API_KEY}
168	      site: ${datadog_site}
169	    datadog_metrics:
170	      type: datadog_metrics
171	      inputs:
172	        - ${email_redaction_enabled ? "redact_emails_metrics" : "otlp_in.metrics"}
173	      default_api_key: $${DATADOG_API_KEY}
174	      site: ${datadog_site}
175


1	locals {
2	  netpols = {
3	    "vector" = local.vector_netpol
4	  }
5	  vector_netpol = {
6	    endpointSelector = {
7	      matchLabels = {
8	        "app.kubernetes.io/name" = "vector"
9	      }
10	    }
11	    ingress = [
12	      {
13	        # OTLP from any cluster-internal sender (primarily otel-collector).
14	        # Port 8282 is the OPW-compatible Datadog Agent intake (datadog_agent source).
15	        fromEntities = ["cluster"]
16	        toPorts = [{
17	          ports = [
18	            { port = "4317" },
19	            { port = "4318" },
20	            { port = "8282" },
21	          ]
22	        }]
23	      },
24	      {
25	        # Vector self-metrics scrape (Prometheus operator, Datadog Agent, etc.).
26	        fromEntities = ["cluster"]
27	        toPorts = [{
28	          ports = [{ port = "8686" }]
29	        }]
30	      },
31	      {
32	        # kubelet health probes from the node.
33	        fromEntities = ["host"]
34	        toPorts = [{
35	          ports = [
36	            { port = "4317" },
37	            { port = "4318" },
38	            { port = "8686" },
39	          ]
40	        }]
41	      },
42	      {
43	        fromEntities = ["kube-apiserver"]
44	      }
45	    ]
46	    egress = [
47	      {
48	        # Datadog SaaS HTTPS for logs/traces/metrics sinks.
49	        toEntities = ["world"]
50	        toPorts = [{
51	          ports = [{ port = "443" }]
52	        }]
53	      },
54	      {
55	        toEntities = ["kube-apiserver"]
56	      },
57	      {
58	        # DNS resolution + service discovery (CoreDNS lives here).
59	        toEntities = ["cluster"]
60	      }
61	    ]
62	  }
63	}
64


1	output "namespace" {
2	  description = "Namespace Vector is deployed into"
3	  value       = var.namespace
4	}
5
6	output "service_endpoint_grpc" {
7	  description = "Cluster-internal OTLP/gRPC endpoint to point upstream collectors at"
8	  value       = "http://vector.${var.namespace}.svc.cluster.local:4317"
9	}
10
11	output "service_endpoint_http" {
12	  description = "Cluster-internal OTLP/HTTP endpoint"
13	  value       = "http://vector.${var.namespace}.svc.cluster.local:4318"
14	}
15
16	output "service_endpoint_dd_agent" {
17	  description = "OPW-compatible Datadog Agent intake endpoint (http, cluster-internal). Pass to datadog-agent module's logs_intake_url to redirect agent logs through Vector."
18	  value       = "http://vector.${var.namespace}.svc.cluster.local:8282"
19	}
20


1	stages:
2	  - lint
3	  - test
4
5	terraform-fmt:
6	  stage: lint
7	  image: registry.ic-ops.net/cache-docker-hub/hashicorp/terraform:1.9
8	  script:
9	    - terraform fmt -check -recursive
10
11	terraform-validate:
12	  stage: test
13	  image: registry.ic-ops.net/cache-docker-hub/hashicorp/terraform:1.9
14	  script:
15	    - cd examples/basic
16	    - terraform init -backend=false
17	    - terraform validate
18
19	vrl-tests:
20	  stage: test
21	  image: registry.ic-ops.net/cache-docker-hub/timberio/vector:latest-debian
22	  before_script:
23	    - apt-get update -qq && apt-get install -y -qq bash diffutils jq
24	  script:
25	    - ./tests/vrl/drop_audit/run.sh
26	    - ./tests/vrl/redact_emails_logs/run.sh
27	    - ./tests/vrl/redact_emails_traces/run.sh
28	    - ./tests/vrl/redact_emails_metrics/run.sh
29	    - ./tests/vrl/redact_emails_dd_agent_logs/run.sh
30


1	# vector
2
3	Terraform module wrapping the `timberio/vector` Helm chart in aggregator role.
4	Receives OTLP from the OTel collector, applies VRL transforms (audit-log
5	filtering, email redaction), and ships to Datadog.
6
7	See `docs/superpowers/specs/2026-05-05-vector-redaction-design.md` in the IAC repo
8	for design context.
9
10	---
11
12	## Architecture
13
14	Vector sits between the OTel collector tier and Datadog, and additionally
15	serves as an OPW-compatible logs intake for the Datadog Agent:
16
17	```
18	apps → otel-lb → otel-collector ──OTLP/gRPC──▶ vector ──▶ Datadog SaaS
19	                                                  ▲
20	                                          DD Agent (logs)
21	```
22
23	Vector runs as a Deployment (default 3 replicas) fronted by a ClusterIP
24	Service. It exposes:
25
26	- OTLP/gRPC on port 4317 (from OTel collector)
27	- OTLP/HTTP on port 4318 (from OTel collector)
28	- Datadog Agent intake on port 8282 (from DD Agent, OPW protocol)
29
30	Three signal pipelines run in parallel:
31
32	- **logs** — `drop_audit` filter (multi-input: OTLP logs + DD Agent logs;
33	  drops lines matching audit patterns) → `redact_emails_logs` remap →
34	  `datadog_logs` sink.
35	- **traces** — `redact_emails_traces` remap → `datadog_traces` sink.
36	- **metrics** — `redact_emails_metrics` remap → `datadog_metrics` sink.
37
38	The Datadog API key is sourced from Vault via the existing `external-secrets`
39	operator. The Deployment is annotated with `reloader.stakater.com/auto: "true"`
40	so key rotation triggers a rolling restart automatically.
41
42	---
43
44	## Inputs
45
46	| Name | Type | Required | Default | Description |
47	|---|---|---|---|---|
48	| `chart_version` | `string` | yes | — | Vector Helm chart version pin |
49	| `image_tag_version` | `string` | yes | — | Vector image tag |
50	| `datadog_site` | `string` | yes | — | Datadog site (e.g. `datadoghq.com`, `us3.datadoghq.com`) |
51	| `datadog_api_key_vault_path` | `string` | yes | — | Vault path for the Datadog API key, consumed by ExternalSecret |
52	| `external_secrets_cluster_store_name` | `string` | yes | — | Name of the ClusterSecretStore created by the external-secrets module |
53	| `image` | `string` | no | `registry.ic-ops.net/cache-docker-hub/timberio/vector` | Vector image source |
54	| `replicas` | `number` | no | `3` | Number of Vector aggregator replicas |
55	| `namespace` | `string` | no | `"vector"` | Kubernetes namespace for Vector |
56	| `audit_drop_patterns` | `list(string)` | no | `["Audit\\s-\\s.+", "\"level\":\"audit\""]` | Regex patterns; matching log message bodies are dropped |
57	| `email_redaction_enabled` | `bool` | no | `true` | Toggle the email-redact VRL transform across all signals |
58	| `email_redaction_replacement` | `string` | no | `"[REDACTED-EMAIL]"` | String that replaces matched email addresses |
59	| `resources_requests` | `map(string)` | no | `{cpu="200m", memory="256Mi"}` | Pod resource requests |
60	| `resources_limits` | `map(string)` | no | `{cpu="2", memory="1Gi"}` | Pod resource limits |
61	| `service_monitor_enabled` | `bool` | no | `false` | Create a ServiceMonitor for Vector self-metrics (requires kube-prometheus-stack CRDs) |
62	| `additional_config` | `any` | no | `null` | Escape hatch — overwrites `customConfig` wholesale |
63
64	---
65
66	## Outputs
67
68	| Name | Description |
69	|---|---|
70	| `namespace` | Namespace Vector is deployed into |
71	| `service_endpoint_grpc` | Cluster-internal OTLP/gRPC endpoint (`http://vector.<namespace>.svc.cluster.local:4317`) |
72	| `service_endpoint_http` | Cluster-internal OTLP/HTTP endpoint (`http://vector.<namespace>.svc.cluster.local:4318`) |
73	| `service_endpoint_dd_agent` | OPW-compatible Datadog Agent intake endpoint (`http://vector.<namespace>.svc.cluster.local:8282`) |
74
75	---
76
77	## Usage
78
79	See [`examples/basic/`](examples/basic/) for a minimal working example.
80
81	```hcl
82	module "vector" {
83	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/vector.git?ref=v1.1.0"
84
85	  providers = {
86	    kubernetes = kubernetes.local-cluster
87	    helm       = helm.local-cluster
88	  }
89
90	  chart_version                        = "0.36.0"
91	  image_tag_version                    = "0.41.0-debian"
92	  datadog_site                         = "datadoghq.com"
93	  datadog_api_key_vault_path           = "secret/data/platform/datadog/api_key"
94	  external_secrets_cluster_store_name  = "vault-cluster-store"
95	}
96	```
97
98	Point the OTel collector at Vector's gRPC endpoint via the `downstream_endpoint`
99	variable in the `otel-collector` module (v1.1.0+):
100
101	```hcl
102	downstream_endpoint = module.vector.service_endpoint_grpc
103	```
104
105	Point the Datadog Agent at Vector's OPW intake by passing the new endpoint to
106	the `datadog-agent` module (v1.2.0+):
107
108	```hcl
109	logs_intake_url = module.vector.service_endpoint_dd_agent
110	```
111
112	---
113
114	## Testing
115
116	### VRL unit tests
117
118	Each transform has an input/expected fixture pair and a shell runner under
119	`tests/vrl/<transform-name>/`. Run all five locally (requires `vector` and `jq`
120	on PATH):
121
122	```sh
123	./tests/vrl/drop_audit/run.sh
124	./tests/vrl/redact_emails_logs/run.sh
125	./tests/vrl/redact_emails_traces/run.sh
126	./tests/vrl/redact_emails_metrics/run.sh
127	./tests/vrl/redact_emails_dd_agent_logs/run.sh
128	```
129
130	### Terraform
131
132	```sh
133	# format check
134	terraform fmt -check -recursive
135
136	# validate the example
137	cd examples/basic
138	terraform init -backend=false
139	terraform validate
140	```
141
142	CI runs all three checks automatically on every push (see `.gitlab-ci.yml`).
143


1	{"message": "user alice@example.com signed in", "ddtags": "env:prod,service:api", "service": "api", "hostname": "ip-10-0-1-23"}
2	{"message": "no email here", "ddtags": "env:prod", "service": "worker", "hostname": "ip-10-0-1-24"}
3	{"message": "multiple a@b.com and c.d+tag@sub.example.co.uk", "service": "api", "hostname": "ip-10-0-1-25"}
4	{"message": "ok", "attributes": {"user.email": "bob@x.io", "request.id": "abc-123"}, "ddtags": "env:staging", "service": "auth", "hostname": "ip-10-0-2-99"}
5	{"message": "json blob {\"email\":\"x@y.com\"}", "service": "api", "hostname": "ip-10-0-1-50"}
6