You are implementing Task B5 of an implementation plan: create an `examples/downstream/` directory inside the existing `otel-collector` Terraform internal-module. The example demonstrates downstream-mode (i.e., when `downstream_endpoint` is set, routing through Vector). It exists for `terraform validate` smoke testing.

## What's already in place

- Working directory: `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector`
- Module is on branch `otel-batch-size`. Edits made for the v1.1.0 work:
  - `variables.tf` has new `downstream_endpoint` and `downstream_protocol` variables (B3)
  - `otelconfig.tf` rewritten to conditionally swap exporters (B4)
  - `main.tf` gated the ExternalSecret manifest and `extraEnvsFrom` on `downstream_endpoint == ""` (B4)
- `terraform validate` passes at the module root.
- The module currently has NO `examples/` dir at all.

## Task Description

### Task B5: examples/downstream validation

**Files:**
- Create: `examples/downstream/main.tf`
- Create: `examples/downstream/versions.tf`
- Create: `examples/downstream/README.md`

(The plan also mentioned `examples/basic/` but the otel-collector module doesn't have one yet either — focus on `examples/downstream/` since it's specific to the new feature. If you have time you can also write `examples/basic/` for symmetry, but it's not required for this task.)

- [ ] **Step 1: Inspect the module's required variables.** Read `variables.tf` to identify which variables have NO default and are therefore required. Common ones to expect (based on Phase B1 inventory):
  - `chart_version` (helm chart version pin)
  - `image_tag_version`
  - `otellb_image`, `otel_image`
  - `otel_lb_replicas`, `otel_collector_replicas` (or similar — may have defaults)
  - Provider-related variables
  - Any others.

   The exact set is whatever has no `default = ...` line in variables.tf.

- [ ] **Step 2: Write `examples/downstream/versions.tf`:**

```hcl
terraform {
  required_version = ">= 1.9.0"
  required_providers {
    kubernetes = { source = "hashicorp/kubernetes", version = ">= 2.20" }
    helm       = { source = "hashicorp/helm", version = ">= 2.10" }
  }
}

provider "kubernetes" {
  config_path = "~/.kube/config"
}

provider "helm" {
  kubernetes = {
    config_path = "~/.kube/config"
  }
}
```

(Note the helm v3 equals-sign block syntax — `kubernetes = { ... }` not `kubernetes { ... }`.)

- [ ] **Step 3: Write `examples/downstream/main.tf`:**

```hcl
module "otel_collector" {
  source = "../.."

  # Required variables — fill in based on what variables.tf actually requires.
  # Plausible values:
  chart_version     = "0.85.0"
  image_tag_version = "0.85.0"
  otellb_image      = "registry.ic-ops.net/cache-docker-hub/otel/opentelemetry-collector-contrib"
  otel_image        = "registry.ic-ops.net/cache-docker-hub/otel/opentelemetry-collector-contrib"

  # The new downstream-mode flag — this example demonstrates routing through Vector.
  downstream_endpoint = "http://vector.vector.svc.cluster.local:4317"
}
```

Adjust the required-variable set to match what `variables.tf` actually demands. If a required variable doesn't have a sensible example value, use a placeholder like `"REPLACE_ME"` and note it in the README.

- [ ] **Step 4: Write `examples/downstream/README.md`:**

```markdown
# downstream example

Demonstrates routing the otel-collector's terminal exporter through Vector
(via OTLP/gRPC) instead of directly to Datadog. Used by CI for `terraform
validate` smoke testing — does not deploy.

When `downstream_endpoint` is non-empty:
- The module's ExternalSecret for the Datadog API key is NOT created.
- The processor helm release renders `exporters.otlp` (not `exporters.datadog/exporter`).
- All three pipelines (logs, traces, metrics) export to OTLP.
- The trace pipeline still includes the `datadog/connector` for span-derived metrics.

\`\`\`bash
terraform init -backend=false
terraform validate
\`\`\`
```

(Triple-backticks should be literal in the markdown — escape them appropriately in the file content.)

- [ ] **Step 5: Run validation:**

```bash
cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream
mise exec terraform@1.9.0 -- terraform init -backend=false
mise exec terraform@1.9.0 -- terraform validate
```

Expected: `Success! The configuration is valid.`

If validate fails, debug:
- "Missing required argument" → add the missing required variable to `examples/downstream/main.tf` with a sensible example value.
- Provider config error → adjust the helm provider block syntax.
- Module not found → verify `source = "../.."` resolves.

Iterate until clean.

- [ ] **Step 6: Run `terraform fmt -check -recursive`** at module root — exit 0 expected.

## Constraints

- **DO NOT run any git commands.**
- **DO NOT** push, tag, open MRs.
- **DO NOT** run `terraform apply`. Only `init -backend=false` and `validate`.
- **DO NOT** create snapshot tests.

## Spec / plan references

- Spec: `/Users/steve.briggs/projects/IAC/docs/superpowers/specs/2026-05-05-vector-redaction-design.md` §3.1
- Plan: `/Users/steve.briggs/projects/IAC/docs/superpowers/plans/2026-05-05-vector-redaction-pipeline.md` Task B5

## Report Format

- **Status:** DONE | DONE_WITH_CONCERNS | BLOCKED | NEEDS_CONTEXT
- Files created
- Required-variable surface you discovered (from variables.tf inspection)
- `terraform validate` result
- Any iteration: what failed first, what fix you applied
- Self-review findings


I'll start by reading the module's `variables.tf` to understand what's required, then build the example files.


1	# ---------------------------------------------------------------------------------------------------------------------
2	# REQUIRED PARAMETERS
3	# These variables are configurable, and need to be set expicitly.
4	# ---------------------------------------------------------------------------------------------------------------------
5	variable "chart_version" {
6	  description = "Version of otel collector chart"
7	  type        = string
8	}
9
10	variable "image_tag_version" {
11	  description = "Version of otel collector image tag"
12	  type        = string
13	}
14
15	# ---------------------------------------------------------------------------------------------------------------------
16	# OPTIONAL PARAMETERS
17	# These variables are configurable, but have reasonable defaults set.
18	# ---------------------------------------------------------------------------------------------------------------------
19	variable "name" {
20	  description = "Name for created resources"
21	  type        = string
22	  default     = "otel-collector"
23	}
24
25	variable "otel_image" {
26	  description = "Helm Value: image.X, optional override to specify image repository & tag"
27	  type        = string
28	  default     = "registry.ic-ops.net/cache-docker-hub/otel/opentelemetry-collector-contrib"
29	}
30
31	variable "otellb_image" {
32	  description = "Helm Value: image.X, optional override to specify image repository & tag"
33	  type        = string
34	  default     = "registry.ic-ops.net/cache-docker-hub/otel/opentelemetry-collector"
35	}
36
37	variable "otel_lb_replicas" {
38	  description = "number of otel loadbalancer replicas to create (statefulset)"
39	  type        = number
40	  default     = 3
41	}
42
43	variable "otel_collector_replicas" {
44	  description = "number of replicas to create for the otel collector processor"
45	  type        = number
46	  default     = 3
47	}
48
49	variable "otel_sa_name" {
50	  description = "Service Account name to be used for otel collector statefulset resources"
51	  type        = string
52	  default     = "otel-collector-sa"
53	}
54
55	variable "otellb_sa_name" {
56	  description = "Service Account name to be used for otel collector deployment resources"
57	  type        = string
58	  default     = "otel-collector-lb-sa"
59	}
60
61	variable "otel_pv_size" {
62	  description = "Size of the Persistent Volume to be used for otel collector statefulset"
63	  type        = string
64	  default     = "5Gi"
65	}
66
67	variable "otel_storage_class_name" {
68	  description = "Storage Class name to be used for otel collector statefulset Persistent Volume"
69	  type        = string
70	  default     = "ebs-encrypted"
71	}
72	variable "sa_annotations" {
73	  description = "Helm Value: A map providing annotations to apply to the otel collector ServiceAccount"
74	  type        = map(string)
75	  default     = {}
76	}
77
78	variable "namespace" {
79	  description = "Namespace to be used for creation of otel collector resources"
80	  type        = string
81	  default     = "opentelemetry"
82	}
83
84	variable "annotations" {
85	  description = "A map providing annotations to apply to the otel collector pods"
86	  type        = map(string)
87	  default     = {}
88	}
89
90	variable "extra_args" {
91	  description = "A list of additional arguments (CLI args) to apply to the otel collector container"
92	  type        = list(string)
93	  default     = []
94	}
95
96	variable "otellb_extra_args" {
97	  description = "A list of additional arguments (CLI args) to apply to the otel collector loadbalancer container"
98	  type        = list(string)
99	  default     = []
100	}
101
102	variable "tags" {
103	  description = "Map of key-vaue pairs to tag created resources with"
104	  type        = map(any)
105	  default     = {}
106	}
107
108	variable "downstream_endpoint" {
109	  description = "When non-empty, swap the terminal exporter from datadog to OTLP, pointed at this endpoint. Used to route through Vector for redaction."
110	  type        = string
111	  default     = ""
112	}
113
114	variable "downstream_protocol" {
115	  description = "Protocol for downstream OTLP exporter. Currently only otlp/grpc supported."
116	  type        = string
117	  default     = "otlp/grpc"
118	  validation {
119	    condition     = contains(["otlp/grpc", "otlp/http"], var.downstream_protocol)
120	    error_message = "downstream_protocol must be one of: otlp/grpc, otlp/http"
121	  }
122	}
123


Good. Only two required variables: `chart_version` and `image_tag_version`. Everything else has defaults. Now let me check the existing module structure and any provider requirements.


Bash

List otel-collector module files


1	/**
2	*
3	* # OTEL Collector (K8s Addon)
4	* Deploys otel collector via terraform.
5	* ## Created Resources
6	* 1. otel collector.
7	*
8	*/
9	terraform {
10	  required_version = ">= 1.3.0"
11
12	  required_providers {
13	    helm = {
14	      source  = "hashicorp/helm"
15	      version = "~> 2.0"
16	    }
17	    kubernetes = {
18	      source  = "hashicorp/kubernetes"
19	      version = "~> 2.0"
20	    }
21	  }
22	}
23
24	resource "kubernetes_namespace" "this" {
25	  metadata {
26	    name = var.namespace
27
28	    #labels = merge(var.tags, { "sourceModule" = "/internal-modules/kubernetes/monitoring/otel-collector" })
29
30	    annotations = var.annotations
31	  }
32	}
33
34	resource "kubernetes_manifest" "otel-collector" {
35	  count = var.downstream_endpoint == "" ? 1 : 0
36
37	  manifest = {
38	    "apiVersion" = "external-secrets.io/v1"
39	    "kind"       = "ExternalSecret"
40	    "metadata" = {
41	      "name"      = "otel-collector"
42	      "namespace" = var.namespace
43	    }
44	    "spec" = {
45	      "refreshInterval" = "1h0m0s"
46	      "secretStoreRef" = {
47	        "name" = "vault"
48	        "kind" = "ClusterSecretStore"
49	      }
50	      "target" = {
51	        "name"           = "otel-collector"
52	        "creationPolicy" = "Owner"
53	      }
54	      "data" = [
55	        {
56	          "secretKey" = "DD_API_KEY"
57	          "remoteRef" = {
58	            "key"      = "secret/data/saas/infrastructure/datadog-api-key"
59	            "property" = "apiKey"
60	          }
61	        },
62	      ]
63	    }
64	  }
65	  depends_on = [kubernetes_namespace.this]
66	}
67
68
69	resource "helm_release" "otel_collector_lb" {
70	  #name      = var.name
71	  name             = "otel-collector-lb"
72	  namespace        = var.namespace
73	  create_namespace = false
74	  repository       = "https://open-telemetry.github.io/opentelemetry-helm-charts/"
75	  chart            = "opentelemetry-collector"
76	  version          = var.chart_version
77
78	  values = [yamlencode({
79	    mode = "deployment"
80	    image = {
81	      repository = var.otellb_image
82	      tag        = var.image_tag_version
83	    }
84	    replicaCount = var.otel_lb_replicas
85
86	    nameOverride = "collector"
87
88	    annotations = var.annotations
89
90	    podLabels = {
91	      "ic-ops.net/otel"           = "collector"
92	      "app.kubernetes.io/part-of" = "opentelemetry-collector"
93	    }
94
95	    #serviceAccount = {
96	    #  create = true,
97	    #  name   = var.otellb_sa_name,
98	    #  annotations = var.sa_annotations
99	    #}
100
101	    presets = {
102	      # enables the k8sattributesprocessor and adds it to the traces, metrics, and logs pipelines
103	      kubernetesAttributes = {
104	        enabled                  = true
105	        extractAllPodLabels      = false
106	        extractAllPodAnnotations = true
107	      }
108	    }
109
110	    serviceAccount = {
111	      create      = true,
112	      name        = var.otellb_sa_name
113	      annotations = var.sa_annotations
114	    }
115
116	    command = {
117	      extraArgs = var.otellb_extra_args
118	    }
119
120	    ports = {
121	      otlp = {
122	        enabled       = true
123	        containerPort = 4317
124	        servicePort   = 4317
125	        protocol      = "TCP"
126	        appProtocol   = "grpc"
127	      }
128	      otlp-http = {
129	        enabled       = true
130	        containerPort = 4318
131	        servicePort   = 4318
132	        protocol      = "TCP"
133	      }
134	      jaegar-compact = {
135	        enabled = false
136	      }
137	      jaeger-grpc = {
138	        enabled = false
139	      }
140	      jaeger-thrift = {
141	        enabled = false
142	      }
143	      zipkin = {
144	        enabled = false
145	      }
146	    }
147
148	    config = local.lb_config
149	  })]
150
151	  depends_on = [
152	    kubernetes_namespace.this,
153	    kubernetes_manifest.otel-collector,
154	    helm_release.otel_collector_processor
155	  ]
156	}
157
158	resource "helm_release" "otel_collector_processor" {
159	  #name      = var.name
160	  name             = "otel-collector-processor"
161	  namespace        = var.namespace
162	  create_namespace = false
163	  repository       = "https://open-telemetry.github.io/opentelemetry-helm-charts/"
164	  chart            = "opentelemetry-collector"
165	  version          = var.chart_version
166
167	  values = [yamlencode({
168	    mode = "statefulset"
169	    image = {
170	      repository = var.otel_image
171	      tag        = var.image_tag_version
172	    }
173	    statefulset = {
174	      persistentVolumeClaimRetentionPolicy = {
175	        enabled     = true
176	        whenDeleted = "Delete"
177	        whenScaled  = "Delete"
178	      }
179
180	      volumeClaimTemplates = [
181	        {
182	          metadata = {
183	            name = "otel-collector-data"
184	          }
185	          spec = {
186	            accessModes = ["ReadWriteOnce"]
187	            resources = {
188	              requests = {
189	                storage = var.otel_pv_size
190	              }
191	            }
192	            //storageClassName = var.otel_storage_class_name
193	          }
194	        }
195	      ]
196	    }
197
198	    extraVolumeMounts = [{
199	      name      = "otel-collector-data"
200	      mountPath = "/var/lib/storage/queue"
201	    }]
202
203	    initContainers = [{
204	      name    = "init-fs"
205	      image   = "busybox:latest"
206	      command = ["sh", "-c", "chown -R 10001: /var/lib/storage/queue"]
207	      volumeMounts = [{
208	        name      = "otel-collector-data"
209	        mountPath = "/var/lib/storage/queue"
210	      }]
211	    }]
212
213	    nameOverride = "processor"
214
215	    replicaCount = var.otel_collector_replicas
216
217	    service = {
218	      type      = "ClusterIP"
219	      clusterIP = "None" # Headless service for statefulset
220	    }
221
222	    annotations = var.annotations
223	    podLabels = {
224	      "ic-ops.net/otel"           = "processor"
225	      "app.kubernetes.io/part-of" = "opentelemetry-collector"
226	    }
227
228	    serviceAccount = {
229	      create      = true,
230	      name        = var.otel_sa_name,
231	      annotations = var.sa_annotations
232	    }
233
234	    command = {
235	      extraArgs = var.extra_args
236	    }
237
238	    extraEnvsFrom = var.downstream_endpoint == "" ? [{
239	      secretRef = {
240	        name = "otel-collector"
241	      }
242	    }] : []
243
244	    #presets = {
245	    # enables the k8sattributesprocessor and adds it to the traces, metrics, and logs pipelines
246	    #kubernetesAttributes = {
247	    #  enabled = true
248	    #  extractAllPodLabels = true
249	    #  extractAllPodAnnotations = true
250	    #}
251	    #}
252
253	    ports = {
254	      otlp = {
255	        enabled       = true
256	        containerPort = 4317
257	        servicePort   = 4317
258	        protocol      = "TCP"
259	        appProtocol   = "grpc"
260	      }
261	      otlp-http = {
262	        enabled       = true
263	        containerPort = 4318
264	        servicePort   = 4318
265	        protocol      = "TCP"
266	      }
267	      jaegar-compact = {
268	        enabled = false
269	      }
270	      jaeger-grpc = {
271	        enabled = false
272	      }
273	      jaeger-thrift = {
274	        enabled = false
275	      }
276	      zipkin = {
277	        enabled = false
278	      }
279	    }
280
281	    config = local.otel_config
282	  })]
283
284	  depends_on = [
285	    kubernetes_namespace.this,
286	    kubernetes_manifest.otel-collector
287	  ]
288
289	  lifecycle {
290	    create_before_destroy = false
291	  }
292	}
293
294	module "netpol" {
295	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/security/network-policy.git?ref=v1.1.0"
296
297	  name             = join("-", [var.name, "netpol"])
298	  namespace        = var.namespace
299	  network_policies = local.netpols
300
301	  tags = merge(var.tags, { "sourceModule" = "/internal-modules/kubernetes/monitoring/otel-collector" })
302
303	  depends_on = [
304	    kubernetes_namespace.this
305	  ]
306	}
307


.rw-r--r-- steve.briggs staff  70 B  Mon Feb  2 14:16:18 2026 .gitignore
.rw-r--r-- steve.briggs staff 1.2 KB Mon Feb  2 14:16:18 2026 netpol.tf
.rw-r--r-- steve.briggs staff 1.6 KB Mon Feb  2 14:16:18 2026 outputs.tf
drwxr-xr-x steve.briggs staff  96 B  Tue Feb  3 11:35:45 2026 .idea
drwxr-xr-x steve.briggs staff 448 B  Thu Mar  5 16:48:00 2026 .git
drwxr-xr-x steve.briggs staff 224 B  Tue May  5 12:32:56 2026 ..
.rw-r--r-- steve.briggs staff 4.0 KB Tue May  5 14:10:09 2026 variables.tf
drwxr-xr-x steve.briggs staff 128 B  Tue May  5 14:10:36 2026 .terraform
.rw-r--r-- steve.briggs staff 463 B  Tue May  5 14:10:36 2026 .terraform.lock.hcl
.rw-r--r-- steve.briggs staff 2.9 KB Tue May  5 14:13:36 2026 lbconfig.tf
.rw-r--r-- steve.briggs staff 6.9 KB Tue May  5 14:13:36 2026 main.tf
.rw-r--r-- steve.briggs staff 3.2 KB Tue May  5 14:17:09 2026 otelconfig.tf
drwxr-xr-x steve.briggs staff 416 B  Tue May  5 14:17:09 2026 .


The module uses a git-sourced sub-module for netpol. That will be a problem during `terraform init` in the example. Let me also check the `.terraform.lock.hcl` and the existing `.terraform` directory to understand what providers are already cached, then check if there's a way to work around the git module dependency.


List .terraform directory contents


1	# This file is maintained automatically by "terraform init".
2	# Manual edits may be lost in future updates.
3
4	provider "registry.terraform.io/hashicorp/helm" {
5	  version     = "2.17.0"
6	  constraints = "~> 2.0"
7	  hashes = [
8	    "h1:kQMkcPVvHOguOqnxoEU2sm1ND9vCHiT8TvZ2x6v/Rsw=",
9	  ]
10	}
11
12	provider "registry.terraform.io/hashicorp/kubernetes" {
13	  version     = "2.38.0"
14	  constraints = "~> 2.0"
15	  hashes = [
16	    "h1:soK8Lt0SZ6dB+HsypFRDzuX/npqlMU6M0fvyaR1yW0k=",
17	  ]
18	}
19


drwxr-xr-x steve.briggs staff 128 B Tue May  5 14:10:36 2026 modules
drwxr-xr-x steve.briggs staff 128 B Tue May  5 14:10:36 2026 .
drwxr-xr-x steve.briggs staff  96 B Tue May  5 14:10:36 2026 providers
drwxr-xr-x steve.briggs staff 416 B Tue May  5 14:17:09 2026 ..


The module root already has providers initialized. Now the key concern: the example's `terraform init -backend=false` will try to clone the git-sourced `netpol` module from GitLab. That will fail without network access to that internal GitLab instance. Let me check if the module root's `.terraform/modules` has it cached and understand the situation better.


List cached modules


drwxr-xr-x steve.briggs staff 224 B Tue May  5 14:10:35 2026 netpol
drwxr-xr-x steve.briggs staff 128 B Tue May  5 14:10:36 2026 .
.rw-r--r-- steve.briggs staff 247 B Tue May  5 14:10:36 2026 modules.json
drwxr-xr-x steve.briggs staff 128 B Tue May  5 14:10:36 2026 ..


which mise && mise exec terraform@1.9.0 -- terraform version 2>&1 | head -3

Verify terraform availability via mise


mise () {
	local command
	command="${1:-}"
	if [ "$#" = 0 ]
	then
		command /Users/steve.briggs/.local/bin/mise
		return
	fi
	shift
	case "$command" in
		(deactivate|shell|sh) if [[ ! " $@ " =~ " --help " ]] && [[ ! " $@ " =~ " -h " ]]
			then
				eval "$(command /Users/steve.briggs/.local/bin/mise "$command" "$@")"
				return $?
			fi ;;
	esac
	command /Users/steve.briggs/.local/bin/mise "$command" "$@"
}
Terraform v1.9.0
on darwin_arm64
+ provider registry.terraform.io/hashicorp/helm v3.1.1


Good - terraform 1.9.0 is available. The netpol module was already cloned at the root level. Now let me understand: the example's `terraform init` will need to clone that git module again, which requires SSH access to the internal GitLab. That will likely fail. Let me think about the right approach.

The example calls `source = "../.."` which means it uses the parent module as-is, including its internal `module "netpol"` git source. During `terraform init -backend=false` for the example, Terraform needs to resolve that nested git module.

Let me check if we have SSH keys set up for that GitLab instance and test if the git clone would work:


ssh -T git@gitlab.infrastructure.immuta.io 2>&1 | head -3 || true

Test SSH connectivity to internal GitLab


Check what's in the cached netpol module


Welcome to GitLab, @steve.briggs!


.rw-r--r-- steve.briggs staff 1.9 KB Tue May  5 14:10:35 2026 README.md
.rw-r--r-- steve.briggs staff 1.8 KB Tue May  5 14:10:35 2026 main.tf
.rw-r--r-- steve.briggs staff 562 B  Tue May  5 14:10:35 2026 output.tf
drwxr-xr-x steve.briggs staff 224 B  Tue May  5 14:10:35 2026 .
.rw-r--r-- steve.briggs staff 1.5 KB Tue May  5 14:10:35 2026 variables.tf
drwxr-xr-x steve.briggs staff 384 B  Tue May  5 14:10:35 2026 .git
drwxr-xr-x steve.briggs staff 128 B  Tue May  5 14:10:36 2026 ..


SSH to GitLab works. The `terraform init -backend=false` for the example will be able to clone the netpol module. Now I have everything I need. Let me create the example directory and files.


Create examples/downstream directory


(Bash completed with no output)


Write

terraform {
  required_version = ">= 1.9.0"
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 2.20"
    }
    helm = {
      source  = "hashicorp/helm"
      version = ">= 2.10"
    }
  }
}

provider "kubernetes" {
  config_path = "~/.kube/config"
}

provider "helm" {
  kubernetes = {
    config_path = "~/.kube/config"
  }
}


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/versions.tf (file state is current in your context — no need to Read it back)


module "otel_collector" {
  source = "../.."

  # Required variables (no default in variables.tf).
  chart_version     = "0.85.0"
  image_tag_version = "0.85.0"

  # The new downstream-mode flag — routes all three pipelines through Vector
  # via OTLP/gRPC instead of exporting directly to Datadog.
  downstream_endpoint = "http://vector.vector.svc.cluster.local:4317"
}


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/main.tf (file state is current in your context — no need to Read it back)


# downstream example

Demonstrates routing the otel-collector's terminal exporter through Vector
(via OTLP/gRPC) instead of directly to Datadog. Used by CI for `terraform
validate` smoke testing — does not deploy.

When `downstream_endpoint` is non-empty:
- The module's ExternalSecret for the Datadog API key is NOT created.
- The processor helm release renders `exporters.otlp` (not `exporters.datadog/exporter`).
- All three pipelines (logs, traces, metrics) export to OTLP.
- The trace pipeline still includes the `datadog/connector` for span-derived metrics.

```bash
terraform init -backend=false
terraform validate
```


File created successfully at: /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/README.md (file state is current in your context — no need to Read it back)


Now run `terraform init -backend=false` in the example directory:


Bash

cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream && mise exec terraform@1.9.0 -- terraform init -backend=false 2>&1

Run terraform init for the downstream example


Initializing modules...
- otel_collector in ../..
Downloading git::ssh://git@gitlab.infrastructure.immuta.io/infrastructure/terraform/internal-modules/kubernetes/security/network-policy.git?ref=v1.1.0 for otel_collector.netpol...
- otel_collector.netpol in .terraform/modules/otel_collector.netpol
Initializing provider plugins...
- Finding hashicorp/kubernetes versions matching "~> 2.0, >= 2.20.0"...
- Finding hashicorp/helm versions matching "~> 2.0, >= 2.10.0"...
- Installing hashicorp/kubernetes v2.38.0...
- Installed hashicorp/kubernetes v2.38.0 (signed by HashiCorp)
- Installing hashicorp/helm v2.17.0...
- Installed hashicorp/helm v2.17.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.


Init succeeded. Now validate:


cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream && mise exec terraform@1.9.0 -- terraform validate 2>&1

Run terraform validate for the downstream example


Error: Exit code 1
╷
│ Error: Unsupported argument
│
│   on versions.tf line 20, in provider "helm":
│   20:   kubernetes = {
│
│ An argument named "kubernetes" is not expected here. Did you mean to define
│ a block of type "kubernetes"?
╵


The helm provider v2 uses block syntax (`kubernetes { ... }`), not the equals-sign syntax. The plan spec had the helm v3 form, but the pinned version here resolved to v2.17.0. Let me fix the provider block:


Edit

provider "helm" {
  kubernetes = {
    config_path = "~/.kube/config"
  }
}

provider "helm" {
  kubernetes {
    config_path = "~/.kube/config"
  }
}


The file /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/versions.tf has been updated successfully. (file state is current in your context — no need to Read it back)


Bash

cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream && mise exec terraform@1.9.0 -- terraform validate 2>&1

Re-run terraform validate after fixing helm provider syntax


Success! The configuration is valid.


Clean. Now run `terraform fmt -check -recursive` from the module root:


cd /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector && mise exec terraform@1.9.0 -- terraform fmt -check -recursive 2>&1

Run terraform fmt check recursively from module root


Exit 0, no formatting issues. All done.

---

**Status:** DONE

**Files created:**
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/versions.tf`
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/main.tf`
- `/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/examples/downstream/README.md`

**Required-variable surface (from variables.tf inspection):**
Only two variables have no default and are therefore required:
- `chart_version` (string)
- `image_tag_version` (string)

All other variables — `otel_image`, `otellb_image`, `otel_lb_replicas`, `otel_collector_replicas`, `downstream_endpoint`, `downstream_protocol`, etc. — have defaults and were omitted from the example except for `downstream_endpoint`, which is the point of this example.

**`terraform validate` result:** `Success! The configuration is valid.`

**`terraform fmt -check -recursive` result:** Exit 0, no issues.

**Iteration — one fix required:**
The plan spec suggested `kubernetes = { ... }` (equals-sign block syntax) for the helm provider, which is the Helm provider v3 form. The `versions.tf` constraint `>= 2.10` resolved to helm v2.17.0 (matching the parent module's lock), which uses the traditional block syntax without equals sign. Fixed on first validate failure by changing `kubernetes = {` to `kubernetes {`.

**Self-review findings:**
- The `versions.tf` provider version constraints (`>= 2.20` for kubernetes, `>= 2.10` for helm) are loose lower bounds, which is intentional for an example used only for `validate`. They resolved to the same provider versions already pinned in the parent module's lock file (kubernetes 2.38.0, helm 2.17.0).
- SSH access to `gitlab.infrastructure.immuta.io` was available so the netpol git module cloned cleanly. CI would need the same SSH key available for `init` to succeed.
- The `README.md` note says the example "does not deploy" — accurate since `terraform validate` with `-backend=false` only type-checks, no state or cluster interaction occurs.