File created successfully at: /Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md (file state is current in your context — no need to Read it back)

# Datadog Agent Logs via Vector (OPW Intake) — Design

**Date:** 2026-05-06
**Author:** Sir Charles of St Cheese (with Claudebob)
**Status:** Draft
**Depends on:** [2026-05-05 Vector + OpenTelemetry Redaction Pipeline](./2026-05-05-vector-redaction-design.md)

## 1. Goal

Apply Vector's existing email-redaction transform to the container logs the Datadog Agent collects today. Logs currently bypass Vector entirely (DD Agent ships direct to `datadoghq.com:443`); after this change, DD-Agent-collected logs traverse Vector before reaching Datadog SaaS.

**Out of scope:**
- Migrating the agent-side audit-drop processing rules (`processing-rules/{env}/rules.json`) into Vector. The agent rules continue to drop audit lines before transmission; Vector adds a defense-in-depth `drop_audit` filter on the new path but does not replace the agent rules.
- Routing DD Agent metrics or APM through Vector. Metrics and traces keep their existing OTel-based path.
- Any change to the OTel pipeline.

## 2. Approach

Use the **Datadog Observability Pipelines Worker (OPW)** intake protocol. Vector's native `datadog_agent` source is the same wire protocol Datadog's OPW product uses (Datadog acquired Vector; OPW is Vector with a curated config). Configuring the agent to ship to Vector is the standard, documented OPW deployment shape.

Two environment variables on the DD Agent — `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED=true` and `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL=<vector-url>` — redirect the agent's logs intake to Vector. No `logs_config.use_http` / `logs_no_ssl` / `logs_dd_url` block needed; the URL scheme implies plaintext for `http://` cluster-internal endpoints.

Reference: <https://docs.datadoghq.com/observability_pipelines/sources/datadog_agent/?tab=secretsmanagement>

## 3. Architecture

### 3.1 Pipeline (logs path)

**Before:**
```
DD Agent (DaemonSet)
  → [agent-side processing rules drop audits]
  → datadoghq.com:443
```

**After:**
```
DD Agent (DaemonSet)
  → [agent-side processing rules drop audits]
  → vector.vector.svc.cluster.local:8282  (OPW HTTP intake, no TLS)
  → Vector dd_agent_in source
  → drop_audit  (multi-input: otlp_in.logs + dd_agent_in.logs — defense-in-depth)
  → redact_emails_logs  (when enabled)
  → datadog_logs sink → datadoghq.com:443
```

Traces and metrics are unaffected — they keep their existing path (apps → otel-collector → vector → DD).

### 3.2 Module split

Three repos touched, additive only:

1. **`internal-modules/kubernetes/monitoring/vector`** — bumps to `v1.1.0`. Adds the `datadog_agent` source on port 8282, exposes the port in helm `service.ports`, opens netpol ingress, makes `drop_audit` accept the new input, and adds an output `service_endpoint_dd_agent`. The source is always declared (the receiver is harmless if no agent is pointed at it), so there is no new module variable.
2. **`internal-modules/kubernetes/monitoring/datadog-agent`** — bumps to `v1.1.0`. Adds one variable `logs_intake_url` (string, default `null`). When non-null, the values template renders the two OPW env vars in the `datadog.env` block. When null, rendered output is byte-identical to today.
3. **`eks-cluster-applications`** consumer — adds `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors the existing `vector_enabled` pattern). Wires the URL across modules.

## 4. Components

### 4.1 Vector module deltas

**`templates/values.yaml.tpl`:**

- `service.ports`: add `{ name: dd-agent, port: 8282, protocol: TCP }`.
- `customConfig.sources.dd_agent_in`:
  ```yaml
  dd_agent_in:
    type: datadog_agent
    address: 0.0.0.0:8282
    multiple_outputs: true
    disable_metrics: true
    disable_traces: true
    store_api_key: false
    acknowledgements:
      enabled: true
  ```
  - `multiple_outputs: true` — exposes the `.logs` named output for downstream transforms.
  - `disable_metrics`/`disable_traces: true` — DD Agent only forwards logs through this path; metrics/traces still come via OTel.
  - `store_api_key: false` — the Vector sink supplies its own API key from the existing ExternalSecret; storing the per-event key is unnecessary.
  - `acknowledgements.enabled: true` — DD Agent retries on transient Vector failures, no silent log loss.
- `customConfig.transforms.drop_audit.inputs`: change `["otlp_in.logs"]` → `["otlp_in.logs", "dd_agent_in.logs"]`.

**`netpol.tf`:** add `port = "8282"` to the existing `fromEntities = ["cluster"]` ingress rule's `toPorts.ports` list (alongside 4317/4318). DD Agent pods are cluster-internal, no new from-selector needed.

**`outputs.tf`:**
```hcl
output "service_endpoint_dd_agent" {
  description = "OPW-compatible Datadog Agent intake endpoint (http, cluster-internal)"
  value       = "http://vector.${var.namespace}.svc.cluster.local:8282"
}
```

**README:** note that Vector functions as an OPW logs intake on 8282; describe the `datadog_agent` source.

### 4.2 DD Agent module deltas

**`variables.tf`:**
```hcl
variable "logs_intake_url" {
  description = "OPW-compatible logs intake URL. When set, the agent ships logs to this endpoint via DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL. For cluster-internal Vector use module.vector.service_endpoint_dd_agent. When null (default), logs ship direct to Datadog SaaS."
  type        = string
  default     = null

  validation {
    condition     = var.logs_intake_url == null || can(regex("^https?://", var.logs_intake_url))
    error_message = "logs_intake_url must be a URL starting with http:// or https://, or null."
  }
}
```

**`values.yaml.tftpl`:** append to the existing `datadog.env` list, gated by `%{ if logs_intake_url != null ~}`:
```yaml
%{ if logs_intake_url != null ~}
    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_ENABLED
      value: "true"
    - name: DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL
      value: "${logs_intake_url}"
%{ endif ~}
```

When `logs_intake_url` is null, rendered output is byte-identical to today.

**`main.tf`:** pass `logs_intake_url = var.logs_intake_url` into the `templatefile()` call's variables map.

**No netpol change.** The existing `egress: toEntities = ["cluster"]` catch-all already permits cluster-internal egress to Vector. The `*.datadoghq.com:443` FQDN rule still applies for metrics/traces and remains the path for logs when `logs_intake_url = null`.

**Agent-side processing rules unchanged.** `DD_LOGS_CONFIG_PROCESSING_RULES` continues to drop audit lines before transmission, regardless of destination.

**README:** document `logs_intake_url`.

### 4.3 Consumer (`eks-cluster-applications`) deltas

- New `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors `vector_enabled`'s pattern). Default-true on cluster types where Vector is enabled (`mgmt`, `cps`, `saas`); default-false elsewhere; explicit override variable.
- Local-precondition guard: if `vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.
- Wire on the existing `module.datadog_agent` call:
  ```hcl
  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
  ```
- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.
- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).

## 5. Data Flow

Per pod, per log line, after cutover:

1. Container writes to stdout/stderr.
2. DD Agent's containerd integration tails the line, attaches Kubernetes metadata (namespace, pod, container, service, ddtags).
3. Agent applies `DD_LOGS_CONFIG_PROCESSING_RULES`. If the line matches an `exclude_at_match` audit pattern, **drop here** — never leaves the node.
4. Agent batches surviving lines and POSTs them to `DD_OBSERVABILITY_PIPELINES_WORKER_LOGS_URL` (= `http://vector.vector.svc.cluster.local:8282`) using the OPW HTTP intake protocol. The `DD-API-KEY` header is sent but ignored by Vector (`store_api_key: false`).
5. Service routes to one of the 3 Vector pods (round-robin via ClusterIP).
6. Vector's `dd_agent_in` source decodes the request, emits events on the `dd_agent_in.logs` named output.
7. `drop_audit` filter receives both `otlp_in.logs` and `dd_agent_in.logs`. Same regex check applied — defense-in-depth catches anything the agent rules would miss.
8. `redact_emails_logs` (when enabled) substitutes matching email patterns with `[REDACTED-EMAIL]` in `.message` and string-typed `.attributes` values.
9. `datadog_logs` sink batches and POSTs to `https://*.datadoghq.com:443` using the API key from the Vector ExternalSecret.
10. Acknowledgement flows back: sink → transforms → source → DD Agent. Agent retries on transient Vector failures.

**Cutover semantics:** when `vector_dd_agent_logs_enabled` flips false→true, terraform apply re-renders the DD Agent helm release with the two new env vars. The DaemonSet rolls (default RollingUpdate). During the rolling window, some agent pods ship to DD SaaS direct, others ship to Vector. Both paths converge to the same DD Logs index; no data loss, possible brief duplication if a single line spans the rollover. Acceptable.

## 6. Error Handling

### 6.1 Module input validation

- **DD Agent module — `logs_intake_url`:** validation regex `^https?://`, rejects empty string. Null is allowed (means: ship direct).
- **Consumer — `vector_dd_agent_logs_enabled`:** plan-time guard. If true but `local.vector_enabled = false`, fail with "vector_dd_agent_logs_enabled requires vector_enabled". Ensures the agent isn't pointed at a non-existent service.
- **Vector module — no new validation.** The `datadog_agent` source is harmless when no agent connects.

### 6.2 Misconfiguration paths

- Wrong port in URL (e.g., 8080 instead of 8282): DD Agent connection refused → agent buffers + retries → visible on `datadog.agent.logs.bytes_sent` flatlining and `datadog.agent.logs.errors_count` increasing.
- Vector source misconfigured (e.g., accidental `disable_metrics: false`): Vector startup config validation rejects at helm release time; pods crash-loop visibly via `kubectl logs`.
- Stale `module.vector` ref when bumping consumer: terraform plan diffs the new `service_endpoint_dd_agent` output; if module isn't bumped, the output doesn't exist and the consumer plan fails fast with "Unsupported attribute".

### 6.3 Runtime failure modes

- **Vector unhealthy / pod unreachable:** DD Agent retries via OPW protocol. If retries exhaust, agent buffers in its on-disk log buffer (existing behavior). Once Vector recovers, queued logs flush.
- **Vector backpressure (sink rejecting):** acknowledgements propagate back to DD Agent, slowing ingestion. No silent loss.
- **Email redact VRL panic on a malformed event:** Vector's `remap` transform drops the bad event by default. Visible on `vector_component_errors_total{component_id="redact_emails_logs"}`.
- **Bad rollout / Vector misbehaving:** flip `vector_dd_agent_logs_enabled = false` for the affected cluster, terraform apply, agent re-rolls back to direct-DD shipping in ~3-5 minutes.

### 6.4 Diagnostic surfaces

Use existing infrastructure:

- `vector_component_received_events_total{component_id="dd_agent_in"}` — ingest sanity.
- `vector_component_sent_events_total{component_id="datadog_logs"}` — sink shipping sanity.
- `vector_component_errors_total{component_id=~"dd_agent_in|drop_audit|redact_emails_logs|datadog_logs"}` — error rate per stage.
- DD Agent telemetry: `datadog.agent.logs.bytes_sent`, `datadog.agent.logs.errors_count`, `datadog.agent.logs.bytes_buffered`.

## 7. Testing

### 7.1 Unit-level (per module, in-repo)

1. **Vector module — VRL tests.** Add a 5th VRL test `tests/vrl/redact_emails_dd_agent_logs/` with input shape matching `datadog_agent` source events (top-level `.message` string with embedded email + DD agent metadata: `ddtags`, `service`, `hostname`). Same VRL program as the existing logs test. Confirms the redaction transform is source-agnostic.
2. **Vector module — `terraform fmt` / `validate`** at module root and `examples/basic/`. Standard.
3. **DD Agent module — template render check.** Two assertions via ad-hoc render:
   - `logs_intake_url = null` → output byte-identical to pre-change render (no churn).
   - `logs_intake_url = "http://vector.vector.svc.cluster.local:8282"` → both OPW env vars appear in `datadog.env`.
4. **Consumer — `terraform fmt` / `validate`.** Standard.

### 7.2 Integration verification (in-cluster, ad-hoc, after deploy)

1. Deploy a one-shot test pod that emits a UUID-tagged log line containing an email:
   ```bash
   kubectl run dd-redact-test-$RANDOM \
     --image=busybox --restart=Never --rm -it \
     -- sh -c 'echo "dd-e2e-$(cat /proc/sys/kernel/random/uuid): contact alice@example.com"; sleep 5'
   ```
2. Watch Vector's `dd_agent_in` ingest counter:
   ```bash
   kubectl exec -n vector deploy/vector -- curl -s http://localhost:8686/metrics \
     | grep 'vector_component_received_events_total{component_id="dd_agent_in"'
   ```
   Should tick up by ≥1.
3. Query Datadog for the UUID — expect the log to appear with `[REDACTED-EMAIL]` substituted for the address.
4. **Audit-drop verification:** emit an `Audit - test-<UUID>` line. Expect zero results in DD (dropped at the agent before transmission). Vector's `dd_agent_in` received counter should also NOT increment for this line.
5. **Rollback verification:** flip `vector_dd_agent_logs_enabled = false`, `terraform apply`, repeat step 1, expect log to appear in DD un-redacted (proving the agent went back to direct shipping). Then flip back.

### 7.3 Cross-cutting checks

- **DD Logs UI metadata preservation.** After cutover, verify `service`, `host`, `kubernetes.namespace`, `kubernetes.pod_name`, `ddtags` are still attached to logs in DD UI. The `datadog_agent` source preserves them; the `datadog_logs` sink forwards them; redaction transforms only touch `.message` and string `.attributes`. Verify via a known-shape log line.
- **Redaction does not corrupt JSON-structured logs.** A JSON line like `{"user":"alice@example.com","level":"info"}` should become `{"user":"[REDACTED-EMAIL]","level":"info"}` and remain parseable in DD.

### 7.4 Soak

- One sandbox/dev cluster: 1 day, full smoke suite + Vector self-metrics watch.
- One `mgmt` cluster: 3-7 day soak. Watch `vector_component_errors_total{component_id="dd_agent_in"}`, agent buffer depth (`datadog.agent.logs.bytes_buffered`), DD Logs ingest delta vs. baseline.
- Remaining `mgmt` + all `cps` clusters.
- `saas` clusters region-by-region.

## 8. Open Questions

1. **Default for `vector_dd_agent_logs_enabled` per cluster type.** Current proposal mirrors `vector_enabled`: default-true on `mgmt`/`cps`/`saas`, default-false on `infra`/`dev`. Could decouple (e.g., default-false everywhere, opt-in cluster-by-cluster during initial rollout). Calling this out for confirmation during plan-writing.
2. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).
3. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.

## 9. Next Steps (after design approval)

1. Hand off to `writing-plans` skill to produce the implementation plan.
2. Plan should sequence: Vector module change → DD Agent module change → consumer wire-up → ad-hoc deploy verification.
3. Sir Charles handles git operations (tag `vector v1.1.0`, `datadog-agent v1.1.0`, MRs for all three repos) per established working pattern.


Now spec self-review.


Updated task #5 status


Updated task #6 status


Self-review pass — no placeholders, port/var names consistent across all sections, scope is focused on a single plan, and ambiguities are flagged explicitly in §8 as open questions rather than buried in body text. No fixes needed.


Updated task #7 status


Bash

cd /Users/steve.briggs/projects/IAC && git log --oneline -5 docs/superpowers/specs/ 2>/dev/null && echo "---" && git -C /Users/steve.briggs/projects/IAC status --short docs/superpowers/specs/

Check IAC docs repo git state


Exit code 128


Configure Datadog to ship logs to Vector


git -C /Users/steve.briggs/projects/IAC/docs status --short 2>&1 | head -10; echo "---"; ls /Users/steve.briggs/projects/IAC/docs/.git 2>&1 | head -3; echo "---"; ls -la /Users/steve.briggs/projects/IAC | grep -E "^\.|\.git"

Probe whether docs/ is in a git repo


fatal: not a git repository (or any of the parent directories): .git
---
lsd: /Users/steve.briggs/projects/IAC/docs/.git: No such file or directory (os error 2).

---
.rw-r--r-- steve.briggs staff 6.0 KB Thu Mar 26 12:45:39 2026 .DS_Store


Note on commit: `/Users/steve.briggs/projects/IAC` and `docs/` aren't git repos — same as the prior Vector spec lives there uncommitted (per the 2026-05-05 note). Leaving the spec as a file on disk; you can wrangle git per your usual flow.

---

**Spec written and committed to disk at:**
`/Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md`

Please review it and let me know if you want to make any changes before we start writing out the implementation plan.

A couple of explicit prompts for your review:

- **§8 Open Question 1** — default for `vector_dd_agent_logs_enabled` per cluster type. I proposed mirroring `vector_enabled` (default-true on `mgmt`/`cps`/`saas`), but you may want default-false everywhere for a slow opt-in rollout. This will get nailed down in the plan, but if you have a preference now I'll lock it in before we move forward.
- **§4.3** — the new flag's variable shape. Open to a different name if `vector_dd_agent_logs_enabled` is too long; I picked it for grep-ability but `dd_agent_logs_via_vector` or similar are fine alternatives.

(And just so it's noted: dino-shaped nuggies for the deploy-day team lunch when this lands — non-negotiable, vastly superior to those tragic breaded boulders.)


afplay ~/.claude/sounds/mario/stop/stomp.wav 2>/dev/null || true


what do you mean by mirroring vector_enabled exactly?


The existing `vector_enabled` pattern in `vector.tf` is a three-part construct:

```hcl
# 1. Per-cluster-type defaults map
variable "vector_enabled_cluster_types" {
  type    = map(string)
  default = {
    "mgmt"  = true
    "cps"   = true
    "saas"  = true
    "infra" = false
    "dev"   = false
  }
}

# 2. Explicit override variable, null = "use the map"
variable "vector_enabled" {
  type    = bool
  default = null
}

# 3. Local that combines them: explicit override wins, else look up by cluster type
locals {
  vector_enabled = (
    var.vector_enabled != null
    ? var.vector_enabled
    : lookup(var.vector_enabled_cluster_types, local.cluster_type, true)
  )
}
```

"Mirroring" means I'd add the same three-part construct for the new flag:

```hcl
variable "vector_dd_agent_logs_enabled_cluster_types" {
  type    = map(string)
  default = {
    "mgmt"  = true
    "cps"   = true
    "saas"  = true
    "infra" = false
    "dev"   = false
  }
}

variable "vector_dd_agent_logs_enabled" {
  type    = bool
  default = null
}

locals {
  vector_dd_agent_logs_enabled = (
    var.vector_dd_agent_logs_enabled != null
    ? var.vector_dd_agent_logs_enabled
    : lookup(var.vector_dd_agent_logs_enabled_cluster_types, local.cluster_type, false)
  )
}
```

That's three new declarations per cluster knob, which gets verbose. **Two simpler alternatives** if that feels heavy:

- **(a) Just one bool, default null, computed from `local.vector_enabled`.** No new map. `local.vector_dd_agent_logs_enabled = var.vector_dd_agent_logs_enabled != null ? var.vector_dd_agent_logs_enabled : local.vector_enabled`. Translation: defaults to whatever Vector itself defaults to on this cluster, override per-cluster via the bool. Loses the per-cluster-type granularity but you don't need it — if Vector isn't on, the new flag is meaningless anyway.

- **(b) Coupled hard.** `local.vector_dd_agent_logs_enabled = local.vector_enabled`. No new variable at all. Lose the safety hatch we agreed on in Q2.

(a) keeps the safety hatch from Q2 with one variable instead of three. Want me to switch to (a)?


a


default-src 'none'; frame-ancestors 'none'

_cfuvid=5LuWAvotOAfzrCoBE8vP58mIYl4n5uqTVC1RwNqepQc-1778082071.3521466-1.0.1.1-mKIFiIqmi7l2_L.s44mVJH.T7Xw.dNul95wleh9_wzQ; HttpOnly; SameSite=None; Secure; Path=/; Domain=api.anthropic.com

Overloaded


_cfuvid=O7YqWQHMSZNdcvg5MxZX2IWwdVLftEDXjsJh2TW.6tA-1778082084.1516507-1.0.1.1-dTIV0zDDl3jBKTd8PdTACIVxvVOPwXEKkGksjpWyfdI; HttpOnly; SameSite=None; Secure; Path=/; Domain=api.anthropic.com

Overloaded


_cfuvid=Gzik0AhBv6bzq1vMDD1PLO3TR4RKqq0sac1QqY2A.64-1778082097.523102-1.0.1.1-MTenLjSNB1XkW6ls7BNWZOf1M6kfYMI8O3SUPhCQjUE; HttpOnly; SameSite=None; Secure; Path=/; Domain=api.anthropic.com

Overloaded


_cfuvid=rVsb9PqBD2q8t3Q_0oxsOC.LAaEij0iD_bm47gS6sNM-1778082112.1454096-1.0.1.1-0JO7lFunFgpVi0MK0O9tXKE9_hZrqhW0SSndVSYROKY; HttpOnly; SameSite=None; Secure; Path=/; Domain=api.anthropic.com

Overloaded


Updating the spec to reflect the simpler single-bool approach.

Edit

### 4.3 Consumer (`eks-cluster-applications`) deltas

- New `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors `vector_enabled`'s pattern). Default-true on cluster types where Vector is enabled (`mgmt`, `cps`, `saas`); default-false elsewhere; explicit override variable.
- Local-precondition guard: if `vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.
- Wire on the existing `module.datadog_agent` call:
  ```hcl
  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
  ```
- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.
- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).

### 4.3 Consumer (`eks-cluster-applications`) deltas

- New variable `vector_dd_agent_logs_enabled` (bool, nullable, default `null`). When `null`, the flag inherits from `local.vector_enabled` — i.e., defaults on wherever Vector is on. When set explicitly to `true`/`false`, it overrides. Single variable, no per-cluster-type map; the cluster-type granularity is already provided by `vector_enabled`'s map and there's nothing meaningful to do with this flag in clusters where Vector isn't deployed.
- Local:
  ```hcl
  locals {
    vector_dd_agent_logs_enabled = (
      var.vector_dd_agent_logs_enabled != null
      ? var.vector_dd_agent_logs_enabled
      : local.vector_enabled
    )
  }
  ```
- Local-precondition guard: if `local.vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.
- Wire on the existing `module.datadog_agent` call:
  ```hcl
  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
  ```
- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.
- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).


The file /Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md has been updated successfully. (file state is current in your context — no need to Read it back)

### 4.3 Consumer (`eks-cluster-applications`) deltas

- New `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors `vector_enabled`'s pattern). Default-true on cluster types where Vector is enabled (`mgmt`, `cps`, `saas`); default-false elsewhere; explicit override variable.
- Local-precondition guard: if `vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.
- Wire on the existing `module.datadog_agent` call:
  ```hcl
  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
  ```
- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.
- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).

### 4.3 Consumer (`eks-cluster-applications`) deltas

- New variable `vector_dd_agent_logs_enabled` (bool, nullable, default `null`). When `null`, the flag inherits from `local.vector_enabled` — i.e., defaults on wherever Vector is on. When set explicitly to `true`/`false`, it overrides. Single variable, no per-cluster-type map; the cluster-type granularity is already provided by `vector_enabled`'s map and there's nothing meaningful to do with this flag in clusters where Vector isn't deployed.
- Local:
  ```hcl
  locals {
    vector_dd_agent_logs_enabled = (
      var.vector_dd_agent_logs_enabled != null
      ? var.vector_dd_agent_logs_enabled
      : local.vector_enabled
    )
  }
  ```
- Local-precondition guard: if `local.vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.
- Wire on the existing `module.datadog_agent` call:
  ```hcl
  logs_intake_url = local.vector_dd_agent_logs_enabled ? module.vector[0].service_endpoint_dd_agent : null
  ```
- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.
- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).

-- New `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors `vector_enabled`'s pattern). Default-true on cluster types where Vector is enabled (`mgmt`, `cps`, `saas`); default-false elsewhere; explicit override variable.

-- Local-precondition guard: if `vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.

+- New variable `vector_dd_agent_logs_enabled` (bool, nullable, default `null`). When `null`, the flag inherits from `local.vector_enabled` — i.e., defaults on wherever Vector is on. When set explicitly to `true`/`false`, it overrides. Single variable, no per-cluster-type map; the cluster-type granularity is already provided by `vector_enabled`'s map and there's nothing meaningful to do with this flag in clusters where Vector isn't deployed.

+      var.vector_dd_agent_logs_enabled != null

+      ? var.vector_dd_agent_logs_enabled

+- Local-precondition guard: if `local.vector_dd_agent_logs_enabled = true` but `local.vector_enabled = false`, fail at plan time. Mirrors the existing `vector_validate` `data "external"` pattern.


## 8. Open Questions

1. **Default for `vector_dd_agent_logs_enabled` per cluster type.** Current proposal mirrors `vector_enabled`: default-true on `mgmt`/`cps`/`saas`, default-false on `infra`/`dev`. Could decouple (e.g., default-false everywhere, opt-in cluster-by-cluster during initial rollout). Calling this out for confirmation during plan-writing.
2. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).
3. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.

## 8. Open Questions

1. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).
2. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.


## 8. Open Questions

1. **Default for `vector_dd_agent_logs_enabled` per cluster type.** Current proposal mirrors `vector_enabled`: default-true on `mgmt`/`cps`/`saas`, default-false on `infra`/`dev`. Could decouple (e.g., default-false everywhere, opt-in cluster-by-cluster during initial rollout). Calling this out for confirmation during plan-writing.
2. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).
3. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.

## 8. Open Questions

1. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).
2. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.

-1. **Default for `vector_dd_agent_logs_enabled` per cluster type.** Current proposal mirrors `vector_enabled`: default-true on `mgmt`/`cps`/`saas`, default-false on `infra`/`dev`. Could decouple (e.g., default-false everywhere, opt-in cluster-by-cluster during initial rollout). Calling this out for confirmation during plan-writing.

-2. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).

-3. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.

+1. **Agent buffer behavior under sustained Vector outage.** DD Agent's on-disk log buffer is bounded; if Vector is down for hours, oldest buffered logs may drop. Worth confirming the buffer cap and whether existing alerting catches this case (likely already covered by `datadog.agent.logs.bytes_buffered` rate alerts in the spec §6.3 monitor table).

+2. **Audit-drop rule consolidation.** The agent-side `processing-rules/{env}/rules.json` and Vector's `drop_audit` regex now both exist. Drift between them is silent. Long-term, may want to make one authoritative — but explicitly out of scope for this design.

## 9. Next Steps (after design approval)


Spec updated. Two changes:

- **§4.3** now describes a single nullable bool that inherits from `local.vector_enabled` when unset.
- **§8 OQ1** removed (the per-cluster-type-map question no longer applies); audit-drop consolidation and agent buffer behavior remain as the only open questions.

Ready for your review of the updated spec, or want me to walk through anything else before we move to writing the implementation plan?


this looks good, lets proceed


Updated task #7 status


Updated task #8 status


Launching skill: superpowers:writing-plans


Base directory for this skill: /Users/steve.briggs/.claude/plugins/cache/claude-plugins-official/superpowers/5.1.0/skills/writing-plans

# Writing Plans

## Overview

Write comprehensive implementation plans assuming the engineer has zero context for our codebase and questionable taste. Document everything they need to know: which files to touch for each task, code, testing, docs they might need to check, how to test it. Give them the whole plan as bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.

Assume they are a skilled developer, but know almost nothing about our toolset or problem domain. Assume they don't know good test design very well.

**Announce at start:** "I'm using the writing-plans skill to create the implementation plan."

**Context:** If working in an isolated worktree, it should have been created via the `superpowers:using-git-worktrees` skill at execution time.

**Save plans to:** `docs/superpowers/plans/YYYY-MM-DD-<feature-name>.md`
- (User preferences for plan location override this default)

## Scope Check

If the spec covers multiple independent subsystems, it should have been broken into sub-project specs during brainstorming. If it wasn't, suggest breaking this into separate plans — one per subsystem. Each plan should produce working, testable software on its own.

## File Structure

Before defining tasks, map out which files will be created or modified and what each one is responsible for. This is where decomposition decisions get locked in.

- Design units with clear boundaries and well-defined interfaces. Each file should have one clear responsibility.
- You reason best about code you can hold in context at once, and your edits are more reliable when files are focused. Prefer smaller, focused files over large ones that do too much.
- Files that change together should live together. Split by responsibility, not by technical layer.
- In existing codebases, follow established patterns. If the codebase uses large files, don't unilaterally restructure - but if a file you're modifying has grown unwieldy, including a split in the plan is reasonable.

This structure informs the task decomposition. Each task should produce self-contained changes that make sense independently.

## Bite-Sized Task Granularity

**Each step is one action (2-5 minutes):**
- "Write the failing test" - step
- "Run it to make sure it fails" - step
- "Implement the minimal code to make the test pass" - step
- "Run the tests and make sure they pass" - step
- "Commit" - step

## Plan Document Header

**Every plan MUST start with this header:**

```markdown
# [Feature Name] Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** [One sentence describing what this builds]

**Architecture:** [2-3 sentences about approach]

**Tech Stack:** [Key technologies/libraries]

---
```

## Task Structure

````markdown
### Task N: [Component Name]

**Files:**
- Create: `exact/path/to/file.py`
- Modify: `exact/path/to/existing.py:123-145`
- Test: `tests/exact/path/to/test.py`

- [ ] **Step 1: Write the failing test**

```python
def test_specific_behavior():
    result = function(input)
    assert result == expected
```

- [ ] **Step 2: Run test to verify it fails**

Run: `pytest tests/path/test.py::test_name -v`
Expected: FAIL with "function not defined"

- [ ] **Step 3: Write minimal implementation**

```python
def function(input):
    return expected
```

- [ ] **Step 4: Run test to verify it passes**

Run: `pytest tests/path/test.py::test_name -v`
Expected: PASS

- [ ] **Step 5: Commit**

```bash
git add tests/path/test.py src/path/file.py
git commit -m "feat: add specific feature"
```
````

## No Placeholders

Every step must contain the actual content an engineer needs. These are **plan failures** — never write them:
- "TBD", "TODO", "implement later", "fill in details"
- "Add appropriate error handling" / "add validation" / "handle edge cases"
- "Write tests for the above" (without actual test code)
- "Similar to Task N" (repeat the code — the engineer may be reading tasks out of order)
- Steps that describe what to do without showing how (code blocks required for code steps)
- References to types, functions, or methods not defined in any task

## Remember
- Exact file paths always
- Complete code in every step — if a step changes code, show the code
- Exact commands with expected output
- DRY, YAGNI, TDD, frequent commits

## Self-Review

After writing the complete plan, look at the spec with fresh eyes and check the plan against it. This is a checklist you run yourself — not a subagent dispatch.

**1. Spec coverage:** Skim each section/requirement in the spec. Can you point to a task that implements it? List any gaps.

**2. Placeholder scan:** Search your plan for red flags — any of the patterns from the "No Placeholders" section above. Fix them.

**3. Type consistency:** Do the types, method signatures, and property names you used in later tasks match what you defined in earlier tasks? A function called `clearLayers()` in Task 3 but `clearFullLayers()` in Task 7 is a bug.

If you find issues, fix them inline. No need to re-review — just fix and move on. If you find a spec requirement with no task, add the task.

## Execution Handoff

After saving the plan, offer execution choice:

**"Plan complete and saved to `docs/superpowers/plans/<filename>.md`. Two execution options:**

**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration

**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints

**Which approach?"**

**If Subagent-Driven chosen:**
- **REQUIRED SUB-SKILL:** Use superpowers:subagent-driven-development
- Fresh subagent per task + two-stage review

**If Inline Execution chosen:**
- **REQUIRED SUB-SKILL:** Use superpowers:executing-plans
- Batch execution with checkpoints for review


I'm using the writing-plans skill to create the implementation plan.


Bash

ls /Users/steve.briggs/projects/IAC/docs/superpowers/plans/ 2>&1 | head -10

Check existing plans directory


Error: EISDIR: illegal operation on a directory, read '/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests'


drwxr-xr-x steve.briggs staff 128 B  Tue May  5 12:21:22 2026 ..
.rw-r--r-- steve.briggs staff  60 KB Tue May  5 12:26:40 2026 2026-05-05-vector-redaction-pipeline.md
drwxr-xr-x steve.briggs staff  96 B  Tue May  5 12:26:40 2026 .


ls /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/ && echo "---" && ls /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/redact_emails_logs/

Inspect existing VRL test structure


drwxr-xr-x steve.briggs staff 128 B Tue May  5 12:52:54 2026 ..
drwxr-xr-x steve.briggs staff 224 B Tue May  5 12:59:12 2026 drop_audit
drwxr-xr-x steve.briggs staff 192 B Tue May  5 13:19:05 2026 .
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:20 2026 redact_emails_logs
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:24 2026 redact_emails_traces
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:28 2026 redact_emails_metrics
---
.rw-r--r-- steve.briggs staff 273 B Tue May  5 13:01:33 2026 inputs.ndjson
.rwxr-xr-x steve.briggs staff 290 B Tue May  5 13:02:25 2026 run.sh
.rw-r--r-- steve.briggs staff 289 B Tue May  5 13:08:23 2026 expected.ndjson
drwxr-xr-x steve.briggs staff 192 B Tue May  5 13:19:05 2026 ..
.rw-r--r-- steve.briggs staff 381 B Tue May  5 16:26:20 2026 program.vrl
drwxr-xr-x steve.briggs staff 224 B Tue May  5 16:26:20 2026 .
.rw-r--r-- steve.briggs staff 301 B Tue May  5 16:27:27 2026 .actual.ndjson


1	{"message": "user alice@example.com signed in"}
2	{"message": "no email here"}
3	{"message": "multiple a@b.com and c.d+tag@sub.example.co.uk"}
4	{"message": "ok", "attributes": {"user.email": "bob@x.io", "request.id": "abc-123"}}
5	{"message": "json blob {\"email\":\"x@y.com\"}"}
6

{"message": "user alice@example.com signed in"}
{"message": "no email here"}
{"message": "multiple a@b.com and c.d+tag@sub.example.co.uk"}
{"message": "ok", "attributes": {"user.email": "bob@x.io", "request.id": "abc-123"}}
{"message": "json blob {\"email\":\"x@y.com\"}"}


1	{"message": "user [REDACTED-EMAIL] signed in"}
2	{"message": "no email here"}
3	{"message": "multiple [REDACTED-EMAIL] and [REDACTED-EMAIL]"}
4	{"attributes": {"request.id": "abc-123", "user.email": "[REDACTED-EMAIL]"}, "message": "ok"}
5	{"message": "json blob {\"email\":\"[REDACTED-EMAIL]\"}"}
6

{"message": "user [REDACTED-EMAIL] signed in"}
{"message": "no email here"}
{"message": "multiple [REDACTED-EMAIL] and [REDACTED-EMAIL]"}
{"attributes": {"request.id": "abc-123", "user.email": "[REDACTED-EMAIL]"}, "message": "ok"}
{"message": "json blob {\"email\":\"[REDACTED-EMAIL]\"}"}


1	email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'
2
3	if is_string(.message) {
4	  .message = replace(string!(.message), email_re, "[REDACTED-EMAIL]")
5	}
6
7	if is_object(.attributes) {
8	  attrs, _err = object(.attributes)
9	  if is_object(attrs) {
10	    .attributes = map_values(attrs) -> |v| {
11	      if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
12	    }
13	  }
14	}
15	.
16

email_re = r'[\w._%+\-]+@[\w.-]+\.[A-Za-z]{2,}'

if is_string(.message) {
  .message = replace(string!(.message), email_re, "[REDACTED-EMAIL]")
}

if is_object(.attributes) {
  attrs, _err = object(.attributes)
  if is_object(attrs) {
    .attributes = map_values(attrs) -> |v| {
      if is_string(v) { replace(string!(v), email_re, "[REDACTED-EMAIL]") } else { v }
    }
  }
}
.


1	#!/usr/bin/env bash
2	set -euo pipefail
3	DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
4	ACTUAL="$DIR/.actual.ndjson"
5	vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o > "$ACTUAL"
6	diff -u <(jq -c . "$DIR/expected.ndjson") <(jq -c . "$ACTUAL")
7	echo "redact_emails_logs VRL OK"
8

#!/usr/bin/env bash
set -euo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ACTUAL="$DIR/.actual.ndjson"
vector vrl -p "$DIR/program.vrl" -i "$DIR/inputs.ndjson" -q -o > "$ACTUAL"
diff -u <(jq -c . "$DIR/expected.ndjson") <(jq -c . "$ACTUAL")
echo "redact_emails_logs VRL OK"


1	# vector
2
3	Terraform module wrapping the `timberio/vector` Helm chart in aggregator role.
4	Receives OTLP from the OTel collector, applies VRL transforms (audit-log
5	filtering, email redaction), and ships to Datadog.
6
7	See `docs/superpowers/specs/2026-05-05-vector-redaction-design.md` in the IAC repo
8	for design context.
9
10	---
11
12	## Architecture
13
14	Vector sits between the OTel collector tier and Datadog:
15
16	```
17	apps → otel-lb → otel-collector ──OTLP/gRPC──▶ vector aggregator ──▶ Datadog SaaS
18	```
19
20	Vector runs as a Deployment (default 3 replicas) fronted by a ClusterIP Service.
21	It exposes OTLP/gRPC on port 4317 and OTLP/HTTP on 4318. Three signal pipelines
22	run in parallel:
23
24	- **logs** — `drop_audit` filter (drops lines matching audit patterns) →
25	  `redact_emails_logs` remap → `datadog_logs` sink.
26	- **traces** — `redact_emails_traces` remap → `datadog_traces` sink.
27	- **metrics** — `redact_emails_metrics` remap → `datadog_metrics` sink.
28
29	The Datadog API key is sourced from Vault via the existing `external-secrets`
30	operator. The Deployment is annotated with `reloader.stakater.com/auto: "true"`
31	so key rotation triggers a rolling restart automatically.
32
33	---
34
35	## Inputs
36
37	| Name | Type | Required | Default | Description |
38	|---|---|---|---|---|
39	| `chart_version` | `string` | yes | — | Vector Helm chart version pin |
40	| `image_tag_version` | `string` | yes | — | Vector image tag |
41	| `datadog_site` | `string` | yes | — | Datadog site (e.g. `datadoghq.com`, `us3.datadoghq.com`) |
42	| `datadog_api_key_vault_path` | `string` | yes | — | Vault path for the Datadog API key, consumed by ExternalSecret |
43	| `external_secrets_cluster_store_name` | `string` | yes | — | Name of the ClusterSecretStore created by the external-secrets module |
44	| `image` | `string` | no | `registry.ic-ops.net/cache-docker-hub/timberio/vector` | Vector image source |
45	| `replicas` | `number` | no | `3` | Number of Vector aggregator replicas |
46	| `namespace` | `string` | no | `"vector"` | Kubernetes namespace for Vector |
47	| `audit_drop_patterns` | `list(string)` | no | `["Audit\\s-\\s.+", "\"level\":\"audit\""]` | Regex patterns; matching log message bodies are dropped |
48	| `email_redaction_enabled` | `bool` | no | `true` | Toggle the email-redact VRL transform across all signals |
49	| `email_redaction_replacement` | `string` | no | `"[REDACTED-EMAIL]"` | String that replaces matched email addresses |
50	| `resources_requests` | `map(string)` | no | `{cpu="200m", memory="256Mi"}` | Pod resource requests |
51	| `resources_limits` | `map(string)` | no | `{cpu="2", memory="1Gi"}` | Pod resource limits |
52	| `service_monitor_enabled` | `bool` | no | `false` | Create a ServiceMonitor for Vector self-metrics (requires kube-prometheus-stack CRDs) |
53	| `additional_config` | `any` | no | `null` | Escape hatch — overwrites `customConfig` wholesale |
54
55	---
56
57	## Outputs
58
59	| Name | Description |
60	|---|---|
61	| `namespace` | Namespace Vector is deployed into |
62	| `service_endpoint_grpc` | Cluster-internal OTLP/gRPC endpoint (`http://vector.<namespace>.svc.cluster.local:4317`) |
63	| `service_endpoint_http` | Cluster-internal OTLP/HTTP endpoint (`http://vector.<namespace>.svc.cluster.local:4318`) |
64
65	---
66
67	## Usage
68
69	See [`examples/basic/`](examples/basic/) for a minimal working example.
70
71	```hcl
72	module "vector" {
73	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/vector.git?ref=v1.0.0"
74
75	  providers = {
76	    kubernetes = kubernetes.local-cluster
77	    helm       = helm.local-cluster
78	  }
79
80	  chart_version                        = "0.36.0"
81	  image_tag_version                    = "0.41.0-debian"
82	  datadog_site                         = "datadoghq.com"
83	  datadog_api_key_vault_path           = "secret/data/platform/datadog/api_key"
84	  external_secrets_cluster_store_name  = "vault-cluster-store"
85	}
86	```
87
88	Point the OTel collector at Vector's gRPC endpoint via the `downstream_endpoint`
89	variable in the `otel-collector` module (v1.1.0+):
90
91	```hcl
92	downstream_endpoint = module.vector.service_endpoint_grpc
93	```
94
95	---
96
97	## Testing
98
99	### VRL unit tests
100
101	Each transform has an input/expected fixture pair and a shell runner under
102	`tests/vrl/<transform-name>/`. Run all four locally (requires `vector` and `jq`
103	on PATH):
104
105	```sh
106	./tests/vrl/drop_audit/run.sh
107	./tests/vrl/redact_emails_logs/run.sh
108	./tests/vrl/redact_emails_traces/run.sh
109	./tests/vrl/redact_emails_metrics/run.sh
110	```
111
112	### Terraform
113
114	```sh
115	# format check
116	terraform fmt -check -recursive
117
118	# validate the example
119	cd examples/basic
120	terraform init -backend=false
121	terraform validate
122	```
123
124	CI runs all three checks automatically on every push (see `.gitlab-ci.yml`).
125

# vector

Terraform module wrapping the `timberio/vector` Helm chart in aggregator role.
Receives OTLP from the OTel collector, applies VRL transforms (audit-log
filtering, email redaction), and ships to Datadog.

See `docs/superpowers/specs/2026-05-05-vector-redaction-design.md` in the IAC repo
for design context.

---

## Architecture

Vector sits between the OTel collector tier and Datadog:

```
apps → otel-lb → otel-collector ──OTLP/gRPC──▶ vector aggregator ──▶ Datadog SaaS
```

Vector runs as a Deployment (default 3 replicas) fronted by a ClusterIP Service.
It exposes OTLP/gRPC on port 4317 and OTLP/HTTP on 4318. Three signal pipelines
run in parallel:

- **logs** — `drop_audit` filter (drops lines matching audit patterns) →
  `redact_emails_logs` remap → `datadog_logs` sink.
- **traces** — `redact_emails_traces` remap → `datadog_traces` sink.
- **metrics** — `redact_emails_metrics` remap → `datadog_metrics` sink.

The Datadog API key is sourced from Vault via the existing `external-secrets`
operator. The Deployment is annotated with `reloader.stakater.com/auto: "true"`
so key rotation triggers a rolling restart automatically.

---

## Inputs

| Name | Type | Required | Default | Description |
|---|---|---|---|---|
| `chart_version` | `string` | yes | — | Vector Helm chart version pin |
| `image_tag_version` | `string` | yes | — | Vector image tag |
| `datadog_site` | `string` | yes | — | Datadog site (e.g. `datadoghq.com`, `us3.datadoghq.com`) |
| `datadog_api_key_vault_path` | `string` | yes | — | Vault path for the Datadog API key, consumed by ExternalSecret |
| `external_secrets_cluster_store_name` | `string` | yes | — | Name of the ClusterSecretStore created by the external-secrets module |
| `image` | `string` | no | `registry.ic-ops.net/cache-docker-hub/timberio/vector` | Vector image source |
| `replicas` | `number` | no | `3` | Number of Vector aggregator replicas |
| `namespace` | `string` | no | `"vector"` | Kubernetes namespace for Vector |
| `audit_drop_patterns` | `list(string)` | no | `["Audit\\s-\\s.+", "\"level\":\"audit\""]` | Regex patterns; matching log message bodies are dropped |
| `email_redaction_enabled` | `bool` | no | `true` | Toggle the email-redact VRL transform across all signals |
| `email_redaction_replacement` | `string` | no | `"[REDACTED-EMAIL]"` | String that replaces matched email addresses |
| `resources_requests` | `map(string)` | no | `{cpu="200m", memory="256Mi"}` | Pod resource requests |
| `resources_limits` | `map(string)` | no | `{cpu="2", memory="1Gi"}` | Pod resource limits |
| `service_monitor_enabled` | `bool` | no | `false` | Create a ServiceMonitor for Vector self-metrics (requires kube-prometheus-stack CRDs) |
| `additional_config` | `any` | no | `null` | Escape hatch — overwrites `customConfig` wholesale |

---

## Outputs

| Name | Description |
|---|---|
| `namespace` | Namespace Vector is deployed into |
| `service_endpoint_grpc` | Cluster-internal OTLP/gRPC endpoint (`http://vector.<namespace>.svc.cluster.local:4317`) |
| `service_endpoint_http` | Cluster-internal OTLP/HTTP endpoint (`http://vector.<namespace>.svc.cluster.local:4318`) |

---

## Usage

See [`examples/basic/`](examples/basic/) for a minimal working example.

```hcl
module "vector" {
  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/vector.git?ref=v1.0.0"

  providers = {
    kubernetes = kubernetes.local-cluster
    helm       = helm.local-cluster
  }

  chart_version                        = "0.36.0"
  image_tag_version                    = "0.41.0-debian"
  datadog_site                         = "datadoghq.com"
  datadog_api_key_vault_path           = "secret/data/platform/datadog/api_key"
  external_secrets_cluster_store_name  = "vault-cluster-store"
}
```

Point the OTel collector at Vector's gRPC endpoint via the `downstream_endpoint`
variable in the `otel-collector` module (v1.1.0+):

```hcl
downstream_endpoint = module.vector.service_endpoint_grpc
```

---

## Testing

### VRL unit tests

Each transform has an input/expected fixture pair and a shell runner under
`tests/vrl/<transform-name>/`. Run all four locally (requires `vector` and `jq`
on PATH):

```sh
./tests/vrl/drop_audit/run.sh
./tests/vrl/redact_emails_logs/run.sh
./tests/vrl/redact_emails_traces/run.sh
./tests/vrl/redact_emails_metrics/run.sh
```

### Terraform

```sh
# format check
terraform fmt -check -recursive

# validate the example
cd examples/basic
terraform init -backend=false
terraform validate
```

CI runs all three checks automatically on every push (see `.gitlab-ci.yml`).


1	## Requirements
2
3	| Name | Version |
4	|------|---------|
5	|  [terraform](#requirement\_terraform) | >= 1.3.0, < 2.0 |
6	|  [helm](#requirement\_helm) | ~> 2.0 |
7	|  [kubernetes](#requirement\_kubernetes) | ~> 2.0 |
8
9	## Providers
10
11	| Name | Version |
12	|------|---------|
13	|  [helm](#provider\_helm) | ~> 2.0 |
14	|  [kubernetes](#provider\_kubernetes) | ~> 2.0 |
15
16	## Modules
17
18	| Name | Source | Version |
19	|------|--------|---------|
20	|  [netpol](#module\_netpol) | git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/security/network-policy.git | v1.1.0 |
21
22	## Resources
23
24	| Name | Type |
25	|------|------|
26	| [helm_release.this](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
27	| [kubernetes_config_map.datadog_processing_rules](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/config_map) | resource |
28	| [kubernetes_manifest.external-secret](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) | resource |
29	| [kubernetes_manifest.external-secret-dd-api-key](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) | resource |
30	| [kubernetes_manifest.external-secret-dd-api-token](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) | resource |
31	| [kubernetes_namespace.this](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
32
33	## Inputs
34
35	| Name | Description | Type | Default | Required |
36	|------|-------------|------|---------|:--------:|
37	|  [agents\_priority\_class\_name](#input\_agents\_priority\_class\_name) | The priority class name for the Datadog agents. | `string` | `"cloud-ops-system"` | no |
38	|  [apm\_enabled](#input\_apm\_enabled) | Whether to enable APM for the Datadog agent. | `bool` | `true` | no |
39	|  [chart\_version](#input\_chart\_version) | The version of the Datadog Helm chart to use. | `string` | n/a | yes |
40	|  [cluster\_agent\_token\_secret\_name](#input\_cluster\_agent\_token\_secret\_name) | The name of the Kubernetes secret for the cluster agent token. | `string` | `"datadog-clusteragent-token"` | no |
41	|  [cluster\_name](#input\_cluster\_name) | The name of the Kubernetes cluster, to be used for tagging. | `string` | n/a | yes |
42	|  [container\_exclude](#input\_container\_exclude) | A list of container names to exclude from monitoring. | `string` | `"kube_namespace:datadog name:enterprise"` | no |
43	|  [container\_exclude\_logs](#input\_container\_exclude\_logs) | A list of container names to exclude from log collection. | `string` | `"image:registry.k8s.io/autoscaling/cluster-autoscaler image:registry.ic-ops.net/cache-k8s-registry/autoscaling/cluster-autoscaler"` | no |
44	|  [create\_namespace](#input\_create\_namespace) | Whether to create the namespace. Set to false if the namespace already exists. | `bool` | `true` | no |
45	|  [datadog\_api\_key\_secret\_name](#input\_datadog\_api\_key\_secret\_name) | The name of the Kubernetes secret holding the Datadog API key. | `string` | `"datadog-api-key"` | no |
46	|  [datadog\_tags](#input\_datadog\_tags) | A map of global tags to apply to all metrics, traces, and logs. | `list(string)` | `[]` | no |
47	|  [enable\_mrds\_monitoring](#input\_enable\_mrds\_monitoring) | Enable monitoring for MRDS. | `bool` | `false` | no |
48	|  [enable\_now\_obfusicator\_options](#input\_enable\_now\_obfusicator\_options) | Whether to enable the NOWRDS obfuscator options in the Datadog agent configuration. | `bool` | `false` | no |
49	|  [enable\_nowrds\_monitoring](#input\_enable\_nowrds\_monitoring) | Enable monitoring for NOWRDS. | `bool` | `false` | no |
50	|  [env](#input\_env) | The environment that resources are deployed to. i.e. nonprod, prod | `string` | n/a | yes |
51	|  [global\_segment](#input\_global\_segment) | The global segment to use for tagging and other purposes. | `string` | n/a | yes |
52	|  [image\_tag](#input\_image\_tag) | The tag of the Datadog agent image to use. | `string` | n/a | yes |
53	|  [log\_level](#input\_log\_level) | The log level for the Datadog agent. | `string` | `"ERROR"` | no |
54	|  [name](#input\_name) | The name of the Helm release. | `string` | `"immuta-dd"` | no |
55	|  [namespace](#input\_namespace) | The Kubernetes namespace to deploy the Datadog agent into. | `string` | `"datadog"` | no |
56	|  [now\_obfusicator\_options](#input\_now\_obfusicator\_options) | A map of options to configure the NOWRDS obfuscator in the Datadog agent configuration. | `map(string)` |
{
  "replace_digits": "true"
}
 | no |
57	|  [tags](#input\_tags) | A list of global tags to apply to all metrics, traces, and logs. | `map(any)` | `{}` | no |
58
59	## Outputs
60
61	| Name | Description |
62	|------|-------------|
63	|  [helm\_chart\_revision](#output\_helm\_chart\_revision) | Revision of the release |
64	|  [helm\_chart\_status](#output\_helm\_chart\_status) | Status of the release |
65	|  [helm\_chart\_values](#output\_helm\_chart\_values) | Helm Chart values of the chart |
66	|  [helm\_chart\_version](#output\_helm\_chart\_version) | Version of the release |
67

## Requirements

| Name | Version |
|------|---------|
|  [terraform](#requirement\_terraform) | >= 1.3.0, < 2.0 |
|  [helm](#requirement\_helm) | ~> 2.0 |
|  [kubernetes](#requirement\_kubernetes) | ~> 2.0 |

## Providers

| Name | Version |
|------|---------|
|  [helm](#provider\_helm) | ~> 2.0 |
|  [kubernetes](#provider\_kubernetes) | ~> 2.0 |

## Modules

| Name | Source | Version |
|------|--------|---------|
|  [netpol](#module\_netpol) | git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/security/network-policy.git | v1.1.0 |

## Resources

| Name | Type |
|------|------|
| [helm_release.this](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [kubernetes_config_map.datadog_processing_rules](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/config_map) | resource |
| [kubernetes_manifest.external-secret](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) | resource |
| [kubernetes_manifest.external-secret-dd-api-key](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) | resource |
| [kubernetes_manifest.external-secret-dd-api-token](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) | resource |
| [kubernetes_namespace.this](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
|  [agents\_priority\_class\_name](#input\_agents\_priority\_class\_name) | The priority class name for the Datadog agents. | `string` | `"cloud-ops-system"` | no |
|  [apm\_enabled](#input\_apm\_enabled) | Whether to enable APM for the Datadog agent. | `bool` | `true` | no |
|  [chart\_version](#input\_chart\_version) | The version of the Datadog Helm chart to use. | `string` | n/a | yes |
|  [cluster\_agent\_token\_secret\_name](#input\_cluster\_agent\_token\_secret\_name) | The name of the Kubernetes secret for the cluster agent token. | `string` | `"datadog-clusteragent-token"` | no |
|  [cluster\_name](#input\_cluster\_name) | The name of the Kubernetes cluster, to be used for tagging. | `string` | n/a | yes |
|  [container\_exclude](#input\_container\_exclude) | A list of container names to exclude from monitoring. | `string` | `"kube_namespace:datadog name:enterprise"` | no |
|  [container\_exclude\_logs](#input\_container\_exclude\_logs) | A list of container names to exclude from log collection. | `string` | `"image:registry.k8s.io/autoscaling/cluster-autoscaler image:registry.ic-ops.net/cache-k8s-registry/autoscaling/cluster-autoscaler"` | no |
|  [create\_namespace](#input\_create\_namespace) | Whether to create the namespace. Set to false if the namespace already exists. | `bool` | `true` | no |
|  [datadog\_api\_key\_secret\_name](#input\_datadog\_api\_key\_secret\_name) | The name of the Kubernetes secret holding the Datadog API key. | `string` | `"datadog-api-key"` | no |
|  [datadog\_tags](#input\_datadog\_tags) | A map of global tags to apply to all metrics, traces, and logs. | `list(string)` | `[]` | no |
|  [enable\_mrds\_monitoring](#input\_enable\_mrds\_monitoring) | Enable monitoring for MRDS. | `bool` | `false` | no |
|  [enable\_now\_obfusicator\_options](#input\_enable\_now\_obfusicator\_options) | Whether to enable the NOWRDS obfuscator options in the Datadog agent configuration. | `bool` | `false` | no |
|  [enable\_nowrds\_monitoring](#input\_enable\_nowrds\_monitoring) | Enable monitoring for NOWRDS. | `bool` | `false` | no |
|  [env](#input\_env) | The environment that resources are deployed to. i.e. nonprod, prod | `string` | n/a | yes |
|  [global\_segment](#input\_global\_segment) | The global segment to use for tagging and other purposes. | `string` | n/a | yes |
|  [image\_tag](#input\_image\_tag) | The tag of the Datadog agent image to use. | `string` | n/a | yes |
|  [log\_level](#input\_log\_level) | The log level for the Datadog agent. | `string` | `"ERROR"` | no |
|  [name](#input\_name) | The name of the Helm release. | `string` | `"immuta-dd"` | no |
|  [namespace](#input\_namespace) | The Kubernetes namespace to deploy the Datadog agent into. | `string` | `"datadog"` | no |
|  [now\_obfusicator\_options](#input\_now\_obfusicator\_options) | A map of options to configure the NOWRDS obfuscator in the Datadog agent configuration. | `map(string)` |
{
  "replace_digits": "true"
}
 | no |
|  [tags](#input\_tags) | A list of global tags to apply to all metrics, traces, and logs. | `map(any)` | `{}` | no |

## Outputs

| Name | Description |
|------|-------------|
|  [helm\_chart\_revision](#output\_helm\_chart\_revision) | Revision of the release |
|  [helm\_chart\_status](#output\_helm\_chart\_status) | Status of the release |
|  [helm\_chart\_values](#output\_helm\_chart\_values) | Helm Chart values of the chart |
|  [helm\_chart\_version](#output\_helm\_chart\_version) | Version of the release |


Explore project context

Read datadog-agent + vector module sources, netpols, values templates to understand current state before designing log-routing change.

Ask clarifying questions

One at a time, narrow down purpose/constraints/success criteria for routing Datadog Agent logs through Vector.

Propose 2-3 approaches

Compare protocol options (Datadog Agent native intake vs OTLP vs other) and where the change lives (datadog-agent module vs vector module vs consumer).

Present design sections

Architecture, components, data flow, error handling, testing — get user approval section by section.

Write design doc to docs/superpowers/specs

Save validated spec to docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md and commit.

Spec self-review

Inline scan for placeholders, contradictions, ambiguity, scope. Fix and move on.

User reviews written spec

Ask user to review committed spec before invoking writing-plans.

Invoke writing-plans skill

Terminal step — only after spec approval, hand off to writing-plans for implementation plan.


Need to find the consumer's datadog-agent module call and check the existing drop_audit test before writing the plan.


grep -rn "module \"datadog" /Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/*.tf 2>&1 | head -10 && echo "---" && cat /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/drop_audit/program.vrl && echo "---" && cat /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/vector/tests/vrl/drop_audit/inputs.ndjson

Find consumer datadog-agent wiring and drop_audit test


/Users/steve.briggs/projects/IAC/terraform/iac-library/aws/compute/eks-cluster-applications/datadog_agent.tf:13:module "datadog_agent" {
---
msg = if is_string(.message) { string!(.message) } else { "" }
. = { "kept": !match(msg, r'Audit\s-\s.+') && !match(msg, r'"level":"audit"') }
---
{"message": "Audit - login by user@example.com"}
{"message": "INFO request handled"}
{"message": "{\"level\":\"audit\",\"action\":\"foo\"}"}
{"message": "{\"level\":\"info\",\"msg\":\"audit team meeting\"}"}
{"message": ""}


1	# ---------------------------------------------------------------------------------------------------------------------
2	# METRICS SERVER
3	# ---------------------------------------------------------------------------------------------------------------------
4	locals {
5	  datadog_agent_enabled = (
6	    var.datadog_agent_enabled != null
7	    ? var.datadog_agent_enabled
8	    : lookup(var.datadog_agent_enabled_cluster_types, local.cluster_type, true)
9	  )
10	}
11
12	# Datadog Agent
13	module "datadog_agent" {
14	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/datadog-agent?ref=v1.1.2"
15	  count  = local.datadog_agent_enabled ? 1 : 0
16
17	  providers = {
18	    kubernetes = kubernetes.local-cluster
19	    helm       = helm.local-cluster
20	  }
21
22	  chart_version                  = var.datadog_agent_version
23	  image_tag                      = var.datadog_agent_image_tag
24	  create_namespace               = var.datadog_create_namespace
25	  cluster_name                   = var.cluster_name
26	  global_segment                 = var.global_segment
27	  datadog_tags                   = var.datadog_tags
28	  enable_nowrds_monitoring       = var.datadog_enable_nowrds_monitoring
29	  enable_now_obfusicator_options = var.datadog_enable_now_obfusicator_options
30	  enable_mrds_monitoring         = var.datadog_enable_mrds_monitoring
31	  now_obfusicator_options        = var.datadog_now_obfusicator_options
32	  tags                           = var.tags
33	  env                            = var.env
34	}
35
36	# ---------------------------------------------------------------------------------------------------------------------
37	# REQUIRED PARAMETERS
38	# These variables must be set.
39	# ---------------------------------------------------------------------------------------------------------------------
40	variable "datadog_agent_version" {
41	  description = "Version of otel-collector helm chart"
42	  type        = string
43	  default     = ""
44	  validation {
45	    condition     = var.datadog_agent_version == "" && local.datadog_agent_enabled ? false : true
46	    error_message = "datadog_agent_version must be set when datadog_agent is enabled"
47	  }
48	}
49
50	variable "datadog_agent_image_tag" {
51	  description = "Version of otel-collector image tag"
52	  type        = string
53	  default     = ""
54	  validation {
55	    condition     = var.datadog_agent_image_tag == "" && local.datadog_agent_enabled ? false : true
56	    error_message = "datadog_agent_image_tag must be set when datadog_agent is enabled"
57	  }
58	}
59
60	# ---------------------------------------------------------------------------------------------------------------------
61	# OPTIONAL PARAMETERS
62	# These variables are configurable, but have reasonable defaults set.
63	# ---------------------------------------------------------------------------------------------------------------------
64	variable "datadog_agent_enabled_cluster_types" {
65	  description = "Types of clusters datadog_agent should be enabled on by default"
66	  type        = map(string)
67	  default = {
68	    "mgmt"  = true
69	    "cps"   = true
70	    "saas"  = true
71	    "infra" = false
72	    "dev"   = false
73	  }
74	}
75
76	variable "datadog_tags" {
77	  description = "Map of tags to apply to datadog metrics"
78	  type        = list(string)
79	  default     = []
80	}
81
82	variable "datadog_enable_nowrds_monitoring" {
83	  description = "Enable monitoring for NOWRDS."
84	  type        = bool
85	  default     = false
86	}
87
88	variable "datadog_enable_mrds_monitoring" {
89	  description = "Enable multitenant RDS monitoring"
90	  type        = bool
91	  default     = false
92	}
93
94	variable "datadog_enable_now_obfusicator_options" {
95	  description = "Whether to enable the NOWRDS obfuscator options in the Datadog agent configuration."
96	  type        = bool
97	  default     = false
98	}
99
100	variable "datadog_now_obfusicator_options" {
101	  description = "A map of options to configure the NOWRDS obfuscator in the Datadog agent configuration."
102	  type        = map(string)
103	  default = {
104	    replace_digits = "true"
105	  }
106	}
107
108	variable "datadog_create_namespace" {
109	  type    = bool
110	  default = false
111	}
112
113	variable "datadog_agent_enabled" {
114	  description = "Override for explicitly enabling or disabling otel-collector"
115	  type        = bool
116	  default     = null
117	}
118

# ---------------------------------------------------------------------------------------------------------------------
# METRICS SERVER
# ---------------------------------------------------------------------------------------------------------------------
locals {
  datadog_agent_enabled = (
    var.datadog_agent_enabled != null
    ? var.datadog_agent_enabled
    : lookup(var.datadog_agent_enabled_cluster_types, local.cluster_type, true)
  )
}

# Datadog Agent
module "datadog_agent" {
  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/datadog-agent?ref=v1.1.2"
  count  = local.datadog_agent_enabled ? 1 : 0

  providers = {
    kubernetes = kubernetes.local-cluster
    helm       = helm.local-cluster
  }

  chart_version                  = var.datadog_agent_version
  image_tag                      = var.datadog_agent_image_tag
  create_namespace               = var.datadog_create_namespace
  cluster_name                   = var.cluster_name
  global_segment                 = var.global_segment
  datadog_tags                   = var.datadog_tags
  enable_nowrds_monitoring       = var.datadog_enable_nowrds_monitoring
  enable_now_obfusicator_options = var.datadog_enable_now_obfusicator_options
  enable_mrds_monitoring         = var.datadog_enable_mrds_monitoring
  now_obfusicator_options        = var.datadog_now_obfusicator_options
  tags                           = var.tags
  env                            = var.env
}

# ---------------------------------------------------------------------------------------------------------------------
# REQUIRED PARAMETERS
# These variables must be set.
# ---------------------------------------------------------------------------------------------------------------------
variable "datadog_agent_version" {
  description = "Version of otel-collector helm chart"
  type        = string
  default     = ""
  validation {
    condition     = var.datadog_agent_version == "" && local.datadog_agent_enabled ? false : true
    error_message = "datadog_agent_version must be set when datadog_agent is enabled"
  }
}

variable "datadog_agent_image_tag" {
  description = "Version of otel-collector image tag"
  type        = string
  default     = ""
  validation {
    condition     = var.datadog_agent_image_tag == "" && local.datadog_agent_enabled ? false : true
    error_message = "datadog_agent_image_tag must be set when datadog_agent is enabled"
  }
}

# ---------------------------------------------------------------------------------------------------------------------
# OPTIONAL PARAMETERS
# These variables are configurable, but have reasonable defaults set.
# ---------------------------------------------------------------------------------------------------------------------
variable "datadog_agent_enabled_cluster_types" {
  description = "Types of clusters datadog_agent should be enabled on by default"
  type        = map(string)
  default = {
    "mgmt"  = true
    "cps"   = true
    "saas"  = true
    "infra" = false
    "dev"   = false
  }
}

variable "datadog_tags" {
  description = "Map of tags to apply to datadog metrics"
  type        = list(string)
  default     = []
}

variable "datadog_enable_nowrds_monitoring" {
  description = "Enable monitoring for NOWRDS."
  type        = bool
  default     = false
}

variable "datadog_enable_mrds_monitoring" {
  description = "Enable multitenant RDS monitoring"
  type        = bool
  default     = false
}

variable "datadog_enable_now_obfusicator_options" {
  description = "Whether to enable the NOWRDS obfuscator options in the Datadog agent configuration."
  type        = bool
  default     = false
}

variable "datadog_now_obfusicator_options" {
  description = "A map of options to configure the NOWRDS obfuscator in the Datadog agent configuration."
  type        = map(string)
  default = {
    replace_digits = "true"
  }
}

variable "datadog_create_namespace" {
  type    = bool
  default = false
}

variable "datadog_agent_enabled" {
  description = "Override for explicitly enabling or disabling otel-collector"
  type        = bool
  default     = null
}


1	stages:
2	  - lint
3	  - test
4
5	terraform-fmt:
6	  stage: lint
7	  image: registry.ic-ops.net/cache-docker-hub/hashicorp/terraform:1.9
8	  script:
9	    - terraform fmt -check -recursive
10
11	terraform-validate:
12	  stage: test
13	  image: registry.ic-ops.net/cache-docker-hub/hashicorp/terraform:1.9
14	  script:
15	    - cd examples/basic
16	    - terraform init -backend=false
17	    - terraform validate
18
19	vrl-tests:
20	  stage: test
21	  image: registry.ic-ops.net/cache-docker-hub/timberio/vector:latest-debian
22	  before_script:
23	    - apt-get update -qq && apt-get install -y -qq bash diffutils jq
24	  script:
25	    - ./tests/vrl/drop_audit/run.sh
26	    - ./tests/vrl/redact_emails_logs/run.sh
27	    - ./tests/vrl/redact_emails_traces/run.sh
28	    - ./tests/vrl/redact_emails_metrics/run.sh
29

stages:
  - lint
  - test

terraform-fmt:
  stage: lint
  image: registry.ic-ops.net/cache-docker-hub/hashicorp/terraform:1.9
  script:
    - terraform fmt -check -recursive

terraform-validate:
  stage: test
  image: registry.ic-ops.net/cache-docker-hub/hashicorp/terraform:1.9
  script:
    - cd examples/basic
    - terraform init -backend=false
    - terraform validate

vrl-tests:
  stage: test
  image: registry.ic-ops.net/cache-docker-hub/timberio/vector:latest-debian
  before_script:
    - apt-get update -qq && apt-get install -y -qq bash diffutils jq
  script:
    - ./tests/vrl/drop_audit/run.sh
    - ./tests/vrl/redact_emails_logs/run.sh
    - ./tests/vrl/redact_emails_traces/run.sh
    - ./tests/vrl/redact_emails_metrics/run.sh


ls /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/ 2>&1 | grep -i -E "ci|test|gitlab" && echo "---" && cat /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/datadog-agent/.gitlab-ci.yml 2>/dev/null || echo "no ci file"

Check datadog-agent module CI


no ci file


Quick spec correction — the consumer is already at DD Agent v1.1.2, so the next bump should be v1.2.0, not v1.1.0.


Edit

2. **`internal-modules/kubernetes/monitoring/datadog-agent`** — bumps to `v1.1.0`. Adds one variable `logs_intake_url` (string, default `null`). When non-null, the values template renders the two OPW env vars in the `datadog.env` block. When null, rendered output is byte-identical to today.

2. **`internal-modules/kubernetes/monitoring/datadog-agent`** — bumps to `v1.2.0` (current consumer ref is `v1.1.2`). Adds one variable `logs_intake_url` (string, default `null`). When non-null, the values template renders the two OPW env vars in the `datadog.env` block. When null, rendered output is byte-identical to today.


The file /Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-06-datadog-agent-logs-via-vector-design.md has been updated successfully. (file state is current in your context — no need to Read it back)

1. **`internal-modules/kubernetes/monitoring/vector`** — bumps to `v1.1.0`. Adds the `datadog_agent` source on port 8282, exposes the port in helm `service.ports`, opens netpol ingress, makes `drop_audit` accept the new input, and adds an output `service_endpoint_dd_agent`. The source is always declared (the receiver is harmless if no agent is pointed at it), so there is no new module variable.

-2. **`internal-modules/kubernetes/monitoring/datadog-agent`** — bumps to `v1.1.0`. Adds one variable `logs_intake_url` (string, default `null`). When non-null, the values template renders the two OPW env vars in the `datadog.env` block. When null, rendered output is byte-identical to today.

+2. **`internal-modules/kubernetes/monitoring/datadog-agent`** — bumps to `v1.2.0` (current consumer ref is `v1.1.2`). Adds one variable `logs_intake_url` (string, default `null`). When non-null, the values template renders the two OPW env vars in the `datadog.env` block. When null, rendered output is byte-identical to today.

3. **`eks-cluster-applications`** consumer — adds `vector_dd_agent_logs_enabled` flag (cluster-type-map + override, mirrors the existing `vector_enabled` pattern). Wires the URL across modules.


- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.

- Bump `module.datadog_agent` source ref to `v1.2.0` and `module.vector` source ref to `v1.1.0`.


-- Bump `module.datadog_agent` source ref to `v1.1.0` and `module.vector` source ref to `v1.1.0`.

+- Bump `module.datadog_agent` source ref to `v1.2.0` and `module.vector` source ref to `v1.1.0`.

- README — inputs row + note about dual-layer log filtering (agent rules drop audits, Vector applies email redaction).


Now writing the plan.