You are doing the pre-flight inventory pass for Phase B of an implementation plan: a backwards-compatible bump of the existing `otel-collector` Terraform internal-module to add a `downstream_endpoint` variable that swaps its terminal exporter from Datadog to OTLP-pointing-at-Vector.

This task is **research only** — do NOT modify any files. Build a complete picture of how the module currently renders helm values so subsequent tasks (B2–B7) can target the right files.

## Working directory

`/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector`

This is the existing module (currently at v1.0.7). Read it thoroughly.

## Specific questions to answer

Read the module's files and answer the following — your report becomes the input to subsequent tasks. Be specific (file paths, line numbers, exact code snippets where relevant):

1. **File layout.** List all top-level files and notable subdirectories (e.g., `templates/`, `tests/`, `examples/`).

2. **Where is the helm values rendered?**
   - Is there a `templates/values.yaml.tpl` (or similar) consumed by `templatefile()`?
   - Or is the helm values built inline via `yamlencode()` in `main.tf`?
   - Quote the exact code that constructs the helm values, with file path and line numbers.

3. **Where is the `exporters.datadog` block defined?**
   - Find the literal block (or its dynamic equivalent) that emits `exporters.datadog`.
   - Find where `service.pipelines.{logs,traces,metrics}.exporters` lists are set to `[datadog]`.
   - Quote them with file paths and line numbers.

4. **How is the Datadog API key consumed by the pods?**
   - Is there an `env` block setting `DATADOG_API_KEY` from a `secretKeyRef`?
   - What is the source secret name and key?
   - Where in the helm values is this declared? File and line numbers.
   - Is the Datadog secret created by this module (e.g., a kubernetes_manifest ExternalSecret) or assumed pre-existing?

5. **Existing tests, if any.**
   - Are there snapshot tests?
   - Is there a `tests/` or `examples/` dir?
   - List anything testable that already exists.

6. **Existing variables + outputs.**
   - Skim `variables.tf` and report whether any of these names already exist: `downstream_endpoint`, `downstream_protocol`, `vector_endpoint`. (They shouldn't, but verify.)

7. **`README.md` state — does it exist?** If yes, what sections does it have?

8. **Module version pin.** Is the module currently a tagged release? Look for any v1.0.7 reference, a CHANGELOG, etc. (Don't run `git log` — read static files for version evidence.)

9. **Any quirks worth noting** — anything that would surprise an implementer planning to add a conditional `exporters.otlp` block alongside the existing `exporters.datadog`. Examples: tail-sampling processors, k8sattributes, custom helm chart wrapper, multi-tier deployment (otel-lb in front of otel-collector), hardcoded pipeline configurations, etc.

## Constraints

- **DO NOT modify any files.** This is a read-only research task.
- **DO NOT run `terraform apply`, `init`, or `validate`.** Just read.
- **DO NOT run any git commands.**

## Report Format

Structured response with sections matching the 9 questions above. Be terse but specific — file paths, line numbers, exact snippets where they matter. The implementer doing tasks B2-B7 will work directly from your report.

Length: ~400-700 words is the target. Less if the module is simpler than expected; more is fine if there's a real quirk.


I'll read the module files thoroughly. Let me start with a broad inventory.


Bash

find /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector -type f | sort

List all files in the otel-collector module


Now let me read all the relevant Terraform files in parallel.


1	/**
2	*
3	* # OTEL Collector (K8s Addon)
4	* Deploys otel collector via terraform.
5	* ## Created Resources
6	* 1. otel collector.
7	*
8	*/
9	terraform {
10	  required_version = ">= 1.3.0"
11
12	  required_providers {
13	    helm = {
14	      source  = "hashicorp/helm"
15	      version = "~> 2.0"
16	    }
17	    kubernetes = {
18	      source  = "hashicorp/kubernetes"
19	      version = "~> 2.0"
20	    }
21	  }
22	}
23
24	resource "kubernetes_namespace" "this" {
25	  metadata {
26	    name = var.namespace
27
28	    #labels = merge(var.tags, { "sourceModule" = "/internal-modules/kubernetes/monitoring/otel-collector" })
29
30	    annotations = var.annotations
31	  }
32	}
33
34	resource "kubernetes_manifest" "otel-collector" {
35	  manifest = {
36	    "apiVersion" = "external-secrets.io/v1"
37	    "kind"       = "ExternalSecret"
38	    "metadata" = {
39	      "name" = "otel-collector"
40	      "namespace" = var.namespace
41	    }
42	    "spec" = {
43	      "refreshInterval" = "1h0m0s"
44	      "secretStoreRef" = {
45	        "name" = "vault"
46	        "kind" = "ClusterSecretStore"
47	      }
48	      "target" = {
49	        "name"           = "otel-collector"
50	        "creationPolicy" = "Owner"
51	      }
52	      "data" = [
53	        {
54	          "secretKey" = "DD_API_KEY"
55	          "remoteRef" = {
56	            "key"      = "secret/data/saas/infrastructure/datadog-api-key"
57	            "property" = "apiKey"
58	          }
59	        },
60	      ]
61	    }
62	  }
63	  depends_on = [kubernetes_namespace.this]
64	}
65
66
67	resource "helm_release" "otel_collector_lb" {
68	  #name      = var.name
69	  name = "otel-collector-lb"
70	  namespace = var.namespace
71	  create_namespace = false
72	  repository = "https://open-telemetry.github.io/opentelemetry-helm-charts/"
73	  chart      = "opentelemetry-collector"
74	  version    = var.chart_version
75
76	  values = [yamlencode({
77	    mode = "deployment"
78	    image = {
79	      repository = var.otellb_image
80	      tag = var.image_tag_version
81	    }
82	    replicaCount = var.otel_lb_replicas
83
84	    nameOverride = "collector"
85
86	    annotations = var.annotations
87
88	    podLabels = {
89	      "ic-ops.net/otel" = "collector"
90	      "app.kubernetes.io/part-of" = "opentelemetry-collector"
91	    }
92
93	    #serviceAccount = {
94	    #  create = true,
95	    #  name   = var.otellb_sa_name,
96	    #  annotations = var.sa_annotations
97	    #}
98
99	    presets = {
100	      # enables the k8sattributesprocessor and adds it to the traces, metrics, and logs pipelines
101	      kubernetesAttributes = {
102	        enabled = true
103	        extractAllPodLabels = false
104	        extractAllPodAnnotations = true
105	      }
106	    }
107
108	    serviceAccount = {
109	      create = true,
110	      name   = var.otellb_sa_name
111	      annotations = var.sa_annotations
112	    }
113
114	    command = {
115	      extraArgs = var.otellb_extra_args
116	    }
117
118	    ports = {
119	      otlp = {
120	        enabled = true
121	        containerPort = 4317
122	        servicePort = 4317
123	        protocol = "TCP"
124	        appProtocol = "grpc"
125	      }
126	      otlp-http = {
127	        enabled = true
128	        containerPort = 4318
129	        servicePort = 4318
130	        protocol = "TCP"
131	      }
132	      jaegar-compact = {
133	        enabled = false
134	      }
135	      jaeger-grpc = {
136	        enabled = false
137	      }
138	      jaeger-thrift = {
139	        enabled = false
140	      }
141	      zipkin = {
142	        enabled = false
143	      }
144	    }
145
146	    config = local.lb_config
147	  })]
148
149	  depends_on = [
150	    kubernetes_namespace.this,
151	    kubernetes_manifest.otel-collector,
152	    helm_release.otel_collector_processor
153	  ]
154	}
155
156	resource "helm_release" "otel_collector_processor" {
157	  #name      = var.name
158	  name = "otel-collector-processor"
159	  namespace = var.namespace
160	  create_namespace = false
161	  repository = "https://open-telemetry.github.io/opentelemetry-helm-charts/"
162	  chart      = "opentelemetry-collector"
163	  version    = var.chart_version
164
165	  values = [yamlencode({
166	    mode = "statefulset"
167	    image = {
168	      repository = var.otel_image
169	      tag        = var.image_tag_version
170	    }
171	    statefulset = {
172	      persistentVolumeClaimRetentionPolicy = {
173	        enabled     = true
174	        whenDeleted = "Delete"
175	        whenScaled  = "Delete"
176	      }
177
178	      volumeClaimTemplates = [
179	        {
180	          metadata = {
181	            name = "otel-collector-data"
182	          }
183	          spec = {
184	            accessModes = ["ReadWriteOnce"]
185	            resources = {
186	              requests = {
187	                storage = var.otel_pv_size
188	              }
189	            }
190	            //storageClassName = var.otel_storage_class_name
191	          }
192	        }
193	      ]
194	    }
195
196	    extraVolumeMounts = [{
197	      name       = "otel-collector-data"
198	      mountPath  = "/var/lib/storage/queue"
199	    }]
200
201	    initContainers = [{
202	      name  = "init-fs"
203	      image = "busybox:latest"
204	      command = ["sh", "-c", "chown -R 10001: /var/lib/storage/queue"]
205	      volumeMounts = [{
206	        name      = "otel-collector-data"
207	        mountPath = "/var/lib/storage/queue"
208	      }]
209	    }]
210
211	    nameOverride = "processor"
212
213	    replicaCount = var.otel_collector_replicas
214
215	    service = {
216	      type = "ClusterIP"
217	      clusterIP = "None" # Headless service for statefulset
218	    }
219
220	    annotations = var.annotations
221	    podLabels = {
222	      "ic-ops.net/otel" = "processor"
223	      "app.kubernetes.io/part-of" = "opentelemetry-collector"
224	    }
225
226	    serviceAccount = {
227	      create = true,
228	      name   = var.otel_sa_name,
229	      annotations = var.sa_annotations
230	    }
231
232	    command = {
233	      extraArgs = var.extra_args
234	    }
235
236	    extraEnvsFrom = [{
237	      secretRef = {
238	        name = "otel-collector"
239	      }
240	    }]
241
242	    #presets = {
243	      # enables the k8sattributesprocessor and adds it to the traces, metrics, and logs pipelines
244	      #kubernetesAttributes = {
245	      #  enabled = true
246	      #  extractAllPodLabels = true
247	      #  extractAllPodAnnotations = true
248	      #}
249	    #}
250
251	    ports = {
252	      otlp = {
253	        enabled = true
254	        containerPort = 4317
255	        servicePort = 4317
256	        protocol = "TCP"
257	        appProtocol = "grpc"
258	      }
259	      otlp-http = {
260	        enabled = true
261	        containerPort = 4318
262	        servicePort = 4318
263	        protocol = "TCP"
264	      }
265	      jaegar-compact = {
266	        enabled = false
267	      }
268	      jaeger-grpc = {
269	        enabled = false
270	      }
271	      jaeger-thrift = {
272	        enabled = false
273	      }
274	      zipkin = {
275	        enabled = false
276	      }
277	    }
278
279	    config = local.otel_config
280	  })]
281
282	  depends_on = [
283	    kubernetes_namespace.this,
284	    kubernetes_manifest.otel-collector
285	  ]
286
287	  lifecycle {
288	    create_before_destroy = false
289	  }
290	}
291
292	module "netpol" {
293	  source = "git@gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/security/network-policy.git?ref=v1.1.0"
294
295	  name             = join("-", [var.name, "netpol"])
296	  namespace        = var.namespace
297	  network_policies = local.netpols
298
299	  tags = merge(var.tags, { "sourceModule" = "/internal-modules/kubernetes/monitoring/otel-collector" })
300
301	  depends_on = [
302	    kubernetes_namespace.this
303	  ]
304	}
305


1	# ---------------------------------------------------------------------------------------------------------------------
2	# REQUIRED PARAMETERS
3	# These variables are configurable, and need to be set expicitly.
4	# ---------------------------------------------------------------------------------------------------------------------
5	variable "chart_version" {
6	  description = "Version of otel collector chart"
7	  type        = string
8	}
9
10	variable "image_tag_version" {
11	  description = "Version of otel collector image tag"
12	  type        = string
13	}
14
15	# ---------------------------------------------------------------------------------------------------------------------
16	# OPTIONAL PARAMETERS
17	# These variables are configurable, but have reasonable defaults set.
18	# ---------------------------------------------------------------------------------------------------------------------
19	variable "name" {
20	  description = "Name for created resources"
21	  type        = string
22	  default     = "otel-collector"
23	}
24
25	variable "otel_image" {
26	  description = "Helm Value: image.X, optional override to specify image repository & tag"
27	  type        = string
28	  default     = "registry.ic-ops.net/cache-docker-hub/otel/opentelemetry-collector-contrib"
29	}
30
31	variable "otellb_image" {
32	  description = "Helm Value: image.X, optional override to specify image repository & tag"
33	  type        = string
34	  default     = "registry.ic-ops.net/cache-docker-hub/otel/opentelemetry-collector"
35	}
36
37	variable "otel_lb_replicas" {
38	  description = "number of otel loadbalancer replicas to create (statefulset)"
39	  type = number
40	  default = 3
41	}
42
43	variable "otel_collector_replicas" {
44	  description = "number of replicas to create for the otel collector processor"
45	  type = number
46	  default = 3
47	}
48
49	variable "otel_sa_name" {
50	  description = "Service Account name to be used for otel collector statefulset resources"
51	  type        = string
52	  default     = "otel-collector-sa"
53	}
54
55	variable "otellb_sa_name" {
56	  description = "Service Account name to be used for otel collector deployment resources"
57	  type        = string
58	  default     = "otel-collector-lb-sa"
59	}
60
61	variable "otel_pv_size" {
62	    description = "Size of the Persistent Volume to be used for otel collector statefulset"
63	    type        = string
64	    default     = "5Gi"
65	}
66
67	variable otel_storage_class_name {
68	    description = "Storage Class name to be used for otel collector statefulset Persistent Volume"
69	    type        = string
70	    default     = "ebs-encrypted"
71	}
72	variable "sa_annotations" {
73	  description = "Helm Value: A map providing annotations to apply to the otel collector ServiceAccount"
74	  type        = map(string)
75	  default     = {}
76	}
77
78	variable "namespace" {
79	  description = "Namespace to be used for creation of otel collector resources"
80	  type        = string
81	  default     = "opentelemetry"
82	}
83
84	variable "annotations" {
85	  description = "A map providing annotations to apply to the otel collector pods"
86	  type        = map(string)
87	  default     = {}
88	}
89
90	variable "extra_args" {
91	  description = "A list of additional arguments (CLI args) to apply to the otel collector container"
92	  type        = list(string)
93	  default     = []
94	}
95
96	variable "otellb_extra_args" {
97	  description = "A list of additional arguments (CLI args) to apply to the otel collector loadbalancer container"
98	  type        = list(string)
99	  default     = []
100	}
101
102	variable "tags" {
103	  description = "Map of key-vaue pairs to tag created resources with"
104	  type        = map(any)
105	  default     = {}
106	}
107


1	locals {
2	  otel_config = {
3	    receivers = {
4	      otlp = {
5	        protocols = {
6	          grpc = {
7	            endpoint = "0.0.0.0:4317"
8	          }
9	          http = {
10	            endpoint = "0.0.0.0:4318"
11	          }
12	        }
13	      }
14	    }
15
16	    processors = {
17	      batch = {
18	        timeout = "10s"
19	        send_batch_size = 10
20	        send_batch_max_size = 75
21	      }
22	    }
23
24	    connectors = {
25	      "datadog/connector" = {}
26	    }
27
28	    exporters = {
29	      "datadog/exporter" = {
30	        api = {
31	          site = "datadoghq.com"
32	          key = "$${env:DD_API_KEY}"
33	        }
34	        sending_queue = {
35	          enabled = true
36	          storage = "file_storage/persistent_queue"
37	        }
38	      }
39	    }
40
41	    extensions = {
42	      health_check = {
43	        endpoint = "$${env:MY_POD_IP}:13133"
44	      }
45	      "file_storage/persistent_queue" = {
46	        directory = "/var/lib/storage/queue"
47	      }
48	    }
49
50	    service = {
51	      extensions = ["health_check","file_storage/persistent_queue"]
52	      pipelines = {
53	        traces = {
54	          receivers = ["otlp"]
55	          processors = []
56	          exporters = ["datadog/connector", "datadog/exporter"]
57	        }
58
59	        logs = {
60	          receivers = ["otlp"]
61	          processors = []
62	          exporters = ["datadog/exporter"]
63	        }
64
65	        metrics = {
66	          receivers = ["otlp", "datadog/connector"]
67	          processors = []
68	          exporters = ["datadog/exporter"]
69	        }
70	      }
71	      telemetry = {
72	        logs = {
73	          encoding = "json"
74	        }
75	      }
76	    }
77	  }
78	}
79


1	locals {
2	  lb_config = {
3	    receivers = {
4	      otlp = {
5	        protocols = {
6	          grpc = {
7	            endpoint = "0.0.0.0:4317"
8	          }
9	          http = {
10	            endpoint = "0.0.0.0:4318"
11	          }
12	        }
13	      }
14	    }
15
16	    processors = {
17	      k8sattributes = {
18	        auth_type = "serviceAccount"
19	        extract = {
20	          metadata = [
21	            "k8s.namespace.name",
22	            "k8s.deployment.name",
23	            "k8s.statefulset.name",
24	            "k8s.daemonset.name",
25	            "k8s.cronjob.name",
26	            "k8s.job.name",
27	            "k8s.node.name",
28	            "k8s.pod.name",
29	            "k8s.pod.uid",
30	            "k8s.pod.start_time"
31	          ]
32	          labels = [{
33	            key_regex = "tags\\.datadoghq\\.com/(.*)$"
34	            tag_name = "$$1"
35	            from = "pod"
36	          }]
37	        }
38	        passthrough = false
39	        pod_association = [
40	          {
41	            sources = [{
42	              from = "resource_attribute"
43	              name = "k8s.pod.ip"
44	            }]
45	          },
46	          {
47	            sources = [{
48	              from = "resource_attribute"
49	              name = "k8s.pod.uid"
50	            }]
51	          },
52	          {
53	            sources = [{
54	              from = "connection"
55	            }]
56	          }
57	        ]
58	      }
59	    }
60
61	    exporters = {
62	      loadbalancing = {
63	        routing_key = "traceID"
64	        protocol = {
65	          otlp = {
66	            # all options from the OTLP exporter are supported
67	            # except the endpoint
68	            timeout = "2s"
69	            tls = {
70	              insecure = true
71	            }
72	          }
73	        }
74	        resolver = {
75	          # Notice to config a headless service DNS in Kubernetes
76	          dns = {
77	            hostname = "otel-collector-processor.${var.namespace}.svc.cluster.local"
78	          }
79	        }
80	      }
81	      #metrics loadbalancing
82	      "loadbalancing/metrics" = {
83	        routing_key = "service"
84	        protocol = {
85	          otlp = {
86	            # all options from the OTLP exporter are supported
87	            # except the endpoint
88	            timeout = "2s"
89	            tls = {
90	              insecure = true
91	            }
92	          }
93	        }
94	        resolver = {
95	          # Notice to config a headless service DNS in Kubernetes
96	          dns = {
97	            hostname = "otel-collector-processor.${var.namespace}.svc.cluster.local"
98	          }
99	        }
100	      }
101	    }
102
103	    service = {
104	      pipelines = {
105	        traces = {
106	          receivers = ["otlp"]
107	          processors = ["k8sattributes"]
108	          exporters = ["loadbalancing"]
109	        }
110
111	        logs = {
112	          receivers = ["otlp"]
113	          processors = ["k8sattributes"]
114	          exporters = ["loadbalancing"]
115	        }
116
117	        metrics = {
118	          receivers = ["otlp"]
119	          processors = ["k8sattributes"]
120	          exporters = ["loadbalancing/metrics"]
121	        }
122	      }
123	      telemetry = {
124	        logs = {
125	          encoding = "json"
126	        }
127	      }
128	    }
129	  }
130	}
131


1	locals {
2	  netpols = {
3	    "${var.name}" = local.metrics_server_netpol
4	  }
5	  metrics_server_netpol = {
6	    endpointSelector = {
7	      matchLabels = {
8	        "app.kubernetes.io/part-of" = "opentelemetry-collector"
9	      }
10	    }
11	    ingress = [
12	      {
13	        fromEntities = ["host"]
14	        toPorts = [{
15	          ports = [
16	            {
17	              port = "4317"
18	            },
19	            {
20	              port = "4318"
21	            },
22	            {
23	              port = "6831"
24	            }
25	          ]
26	        }]
27	      },
28	      {
29	        fromEntities = ["kube-apiserver"]
30	      },
31	      {
32	        fromEntities = ["cluster"]
33	        toPorts = [{
34	          ports = [
35	            {
36	              port = "4317"
37	            },
38	            {
39	              port = "4318"
40	            },
41	            {
42	              port = "6831"
43	            }
44	          ]
45	        }]
46	      },
47	      {
48	        fromEndpoints = [{}]
49	      }
50	    ]
51	    egress = [
52	      {
53	        toEntities = ["world"]
54	        toPorts = [{
55	          ports = [{
56	            port = "443"
57	          }]
58	        }]
59	      },
60	      {
61	        toEntities = ["kube-apiserver"]
62	      },
63	      {
64	        toEntities = ["cluster"]
65	      }
66	    ]
67	  }
68	}
69


1	output "helm_chart_lb_status" {
2	  description = "Status of the cluster-autoscaler release"
3	  value       = helm_release.otel_collector_lb.status
4	}
5
6	output "helm_chart_lb_version" {
7	  description = "Version of the cluster-autoscaler release"
8	  value       = helm_release.otel_collector_lb.version
9	}
10
11	output "helm_chart_lb_revision" {
12	  description = "Revision of the cluster-autoscaler release"
13	  value       = helm_release.otel_collector_lb.metadata[0].revision
14	}
15
16	output "helm_chart_lb_app_version" {
17	  description = "Appication version of cluster-autoscaler"
18	  value       = helm_release.otel_collector_lb.metadata[0].app_version
19	}
20
21	output "helm_chart_lb_values" {
22	  description = "Helm Chart values of the cluster-autoscaler"
23	  value       = helm_release.otel_collector_lb.metadata[0].values
24	}
25
26	output "helm_chart_collector_status" {
27	  description = "Status of the cluster-autoscaler release"
28	  value       = helm_release.otel_collector_processor.status
29	}
30
31	output "helm_chart_collector_version" {
32	  description = "Version of the cluster-autoscaler release"
33	  value       = helm_release.otel_collector_processor.version
34	}
35
36	output "helm_chart_collector_revision" {
37	  description = "Revision of the cluster-autoscaler release"
38	  value       = helm_release.otel_collector_processor.metadata[0].revision
39	}
40
41	output "helm_chart_collector_app_version" {
42	  description = "Appication version of cluster-autoscaler"
43	  value       = helm_release.otel_collector_processor.metadata[0].app_version
44	}
45
46	output "helm_chart_collector_values" {
47	  description = "Helm Chart values of the cluster-autoscaler"
48	  value       = helm_release.otel_collector_processor.metadata[0].values
49	}
50
51


Now let me check for README, CHANGELOG, and any tests/examples directories.


ls -la /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/ 2>/dev/null | grep -v "^total\|\.git"

List top-level files excluding .git


.rw-r--r-- steve.briggs staff 1.2 KB Mon Feb  2 14:16:18 2026 netpol.tf
.rw-r--r-- steve.briggs staff 1.6 KB Mon Feb  2 14:16:18 2026 outputs.tf
drwxr-xr-x steve.briggs staff  96 B  Tue Feb  3 11:35:45 2026 .idea
.rw-r--r-- steve.briggs staff 6.6 KB Thu Feb 19 09:45:02 2026 main.tf
.rw-r--r-- steve.briggs staff 3.4 KB Thu Feb 19 09:45:02 2026 variables.tf
.rw-r--r-- steve.briggs staff 2.9 KB Thu Mar  5 16:42:45 2026 lbconfig.tf
.rw-r--r-- steve.briggs staff 1.5 KB Thu Mar  5 16:47:53 2026 otelconfig.tf
drwxr-xr-x steve.briggs staff 352 B  Thu Mar  5 16:47:53 2026 .
drwxr-xr-x steve.briggs staff 224 B  Tue May  5 12:32:56 2026 ..


find /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector -maxdepth 3 -name "*.md" -o -name "CHANGELOG*" -o -name "VERSION" 2>/dev/null | grep -v ".git"

Find README, CHANGELOG, VERSION files


(Bash completed with no output)


grep -r "v1.0.7\|version\s*=" /Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/*.tf 2>/dev/null | grep -v ".git"

Search for version references in tf files


1	# pack-refs with: peeled fully-peeled sorted
2	11e07a9fa7b253004f57a9740bd71195b56048f7 refs/remotes/origin/master
3	fd4f582efcd52ff7bb604bd5122f98bca680fec2 refs/remotes/origin/steve
4	4bd0f79242540a51175ef3f9bf892a5063ee40f6 refs/tags/v1.0.0
5	^50f7fec21792e51c71a3eb8849dd62ea82283e01
6	7a8d8e7ccb9f42bb9fab73a5ca399c40ef68620f refs/tags/v1.0.1
7	78c7ee79510ee8583397ea52646318482acf620c refs/tags/v1.0.2
8	^1b9001d74afc7aa473e8aba9f787b6826434f8e4
9	f85be6c135b7d709fe5e3347840e20b5a21ba1bf refs/tags/v1.0.3
10	^a09a1f79b74a701fbdd9702b067f821bc6780dbd
11	bd65bbf291314da710af44163167fc5f70fb51fa refs/tags/v1.0.4
12	^e743a81054e7dfd2fab170e8cf7e5da32896c19d
13	ad25395aa4d04728a8d30af923235de92e65ae48 refs/tags/v1.0.5
14	^11e07a9fa7b253004f57a9740bd71195b56048f7
15


/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/main.tf:  required_version = ">= 1.3.0"
/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/main.tf:      version = "~> 2.0"
/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/main.tf:      version = "~> 2.0"
/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/main.tf:  version    = var.chart_version
/Users/steve.briggs/projects/IAC/terraform/internal-modules/kubernetes/monitoring/otel-collector/main.tf:  version    = var.chart_version


1	lower to 75
2


1	0000000000000000000000000000000000000000 11e07a9fa7b253004f57a9740bd71195b56048f7 mr0xb <47467008+mr0xb@users.noreply.github.com> 1770059778 -0500	clone: from gitlab.infrastructure.immuta.io:infrastructure/terraform/internal-modules/kubernetes/monitoring/otel-collector.git
2	11e07a9fa7b253004f57a9740bd71195b56048f7 11e07a9fa7b253004f57a9740bd71195b56048f7 mr0xb <47467008+mr0xb@users.noreply.github.com> 1770066566 -0500	checkout: moving from master to SRE-2701
3	11e07a9fa7b253004f57a9740bd71195b56048f7 49f968b1977d7f74946bb7ff2bb29dbfe2e8d70d mr0xb <47467008+mr0xb@users.noreply.github.com> 1770066579 -0500	commit: updates
4	49f968b1977d7f74946bb7ff2bb29dbfe2e8d70d 270f043289240d6c7354331ecdbe208510189f13 mr0xb <47467008+mr0xb@users.noreply.github.com> 1770067452 -0500	commit: updates
5	270f043289240d6c7354331ecdbe208510189f13 270f043289240d6c7354331ecdbe208510189f13 mr0xb <47467008+mr0xb@users.noreply.github.com> 1771512273 -0500	reset: moving to HEAD
6	270f043289240d6c7354331ecdbe208510189f13 11e07a9fa7b253004f57a9740bd71195b56048f7 mr0xb <47467008+mr0xb@users.noreply.github.com> 1771512275 -0500	checkout: moving from SRE-2701 to master
7	11e07a9fa7b253004f57a9740bd71195b56048f7 270f043289240d6c7354331ecdbe208510189f13 mr0xb <47467008+mr0xb@users.noreply.github.com> 1771512302 -0500	checkout: moving from master to SRE-2701
8	270f043289240d6c7354331ecdbe208510189f13 270f043289240d6c7354331ecdbe208510189f13 mr0xb <47467008+mr0xb@users.noreply.github.com> 1772746952 -0500	checkout: moving from SRE-2701 to otel-batch-size
9	270f043289240d6c7354331ecdbe208510189f13 226791ca809bd133f6624f6364c62b5f0ecda4ff mr0xb <47467008+mr0xb@users.noreply.github.com> 1772746972 -0500	commit: lower batch size
10	226791ca809bd133f6624f6364c62b5f0ecda4ff 37de6af5cd9524fb9e5a514b20e132433bcfda37 mr0xb <47467008+mr0xb@users.noreply.github.com> 1772747028 -0500	commit: remove debug logging
11	37de6af5cd9524fb9e5a514b20e132433bcfda37 a96caed846cb5b66abc7c301931cd5cbfd0fb0e5 mr0xb <47467008+mr0xb@users.noreply.github.com> 1772747280 -0500	commit: lower to 75
12


I now have everything I need. Here is the complete inventory report.

---

## Phase B Pre-flight Inventory — `otel-collector` module

---

### 1. File Layout

Top-level files (no `templates/`, `tests/`, or `examples/` subdirectories exist):

```
main.tf          — helm_release resources, ExternalSecret, namespace, netpol module call
variables.tf     — all input variables
otelconfig.tf    — local.otel_config (processor/collector helm config)
lbconfig.tf      — local.lb_config (load-balancer helm config)
netpol.tf        — local.netpols (Cilium NetworkPolicy)
outputs.tf       — helm release metadata outputs
.gitignore
.idea/workspace.xml
```

No `README.md`, no `CHANGELOG`, no `tests/`, no `examples/`.

---

### 2. Where Helm Values Are Rendered

Both helm releases use **inline `yamlencode()` in `main.tf`**, not a template file. There is no `templatefile()` call anywhere.

- **`otel-collector-lb`** (`helm_release.otel_collector_lb`): `main.tf` lines 76–147. The `config` key is set to `local.lb_config` (defined in `lbconfig.tf`).
- **`otel-collector-processor`** (`helm_release.otel_collector_processor`): `main.tf` lines 165–280. The `config` key is set to `local.otel_config` (defined in `otelconfig.tf`).

Relevant snippet from `main.tf`:
```hcl
# line 146 (lb)
config = local.lb_config

# line 279 (processor)
config = local.otel_config
```

---

### 3. Where `exporters.datadog` Is Defined

**All Datadog exporter config lives in `otelconfig.tf`** (the processor tier only — the LB tier exports to `loadbalancing`, not Datadog).

- **`exporters."datadog/exporter"` block**: `otelconfig.tf` lines 28–39
```hcl
"datadog/exporter" = {
  api = {
    site = "datadoghq.com"
    key = "$${env:DD_API_KEY}"
  }
  sending_queue = {
    enabled = true
    storage = "file_storage/persistent_queue"
  }
}
```

- **`connectors."datadog/connector"`**: `otelconfig.tf` line 25–26 (empty block, used for trace-to-metrics conversion)

- **Pipeline exporter lists**: `otelconfig.tf` lines 52–69
```hcl
traces:  exporters = ["datadog/connector", "datadog/exporter"]
logs:    exporters = ["datadog/exporter"]
metrics: exporters = ["datadog/exporter"]   # receiver also includes "datadog/connector"
```

---

### 4. Datadog API Key Consumption

The key is injected as an **environment variable** via a Kubernetes secret mounted with `extraEnvsFrom`, **not** an individual `env[].valueFrom.secretKeyRef` block.

- **`main.tf` lines 236–240** (inside `helm_release.otel_collector_processor`):
```hcl
extraEnvsFrom = [{
  secretRef = {
    name = "otel-collector"
  }
}]
```
This mounts all keys from secret `otel-collector` as env vars. The secret key `DD_API_KEY` is then referenced in the config as `$${env:DD_API_KEY}` (`otelconfig.tf` line 32).

The secret `otel-collector` is **created by this module** via a `kubernetes_manifest` ExternalSecret resource (`main.tf` lines 34–64). It pulls from Vault path `secret/data/saas/infrastructure/datadog-api-key`, property `apiKey`, mapped to secret key `DD_API_KEY`.

---

### 5. Existing Tests

None. No `tests/` directory, no `examples/` directory, no snapshot tests, no `.tftest.hcl` files. Zero testable artifacts exist.

---

### 6. Existing Variables — Name Collision Check

None of these names exist in `variables.tf`:
- `downstream_endpoint` — **absent**
- `downstream_protocol` — **absent**
- `vector_endpoint` — **absent**

All clear for B2.

---

### 7. README.md State

**Does not exist.** The only "documentation" is the comment block at the top of `main.tf` (lines 1–8), which is a terraform-docs-style header with a single sentence and a resource list.

---

### 8. Module Version Pin

The `packed-refs` file shows the highest tag is **`v1.0.5`** (commit `ad25395`), not v1.0.7. The current HEAD on branch `otel-batch-size` is at commit `a96caed` (message: "lower to 75"), which is **untagged** and ahead of v1.0.5. There is no CHANGELOG file. The v1.0.7 reference in the task description is not reflected in any static file in this checkout — the latest tag present is v1.0.5.

---

### 9. Quirks Worth Noting for B2–B7

**Two-tier deployment** — this is the most significant structural detail. There are two separate `helm_release` resources:

1. **`otel-collector-lb`** (Deployment, 3 replicas) — receives OTLP from workloads, runs `k8sattributes` processor, then load-balances via `loadbalancing` / `loadbalancing/metrics` exporters to the processor tier using DNS on `otel-collector-processor.<namespace>.svc.cluster.local`. The LB tier has **no Datadog knowledge whatsoever**.

2. **`otel-collector-processor`** (StatefulSet, 3 replicas, headless service) — receives from the LB tier, runs `batch` processor, and exports to Datadog. This is the **only tier that needs the OTLP exporter swap**.

**`datadog/connector` for trace-to-metrics conversion** — the processor pipeline uses `datadog/connector` as both an exporter for traces and a receiver for metrics. This is Datadog's APM connector that derives metrics from trace spans. If swapping to OTLP, this connector either needs to be removed (losing DD-derived metrics) or replaced with `spanmetrics/connector`. This is a non-trivial semantic difference that B2–B7 tasks must decide on.

**Persistent queue on PVC** — `sending_queue.storage = "file_storage/persistent_queue"` with a 5Gi PVC per pod. If the OTLP exporter is added conditionally, it should also get a `sending_queue` config pointing at the same `file_storage/persistent_queue` extension, or a separate one.

**`extraEnvsFrom` mounts the entire `otel-collector` secret** — adding an OTLP endpoint doesn't require touching the secret mount, but if a new secret (e.g., for mTLS) is ever needed, it would require either a new `extraEnvsFrom` entry or extending the existing ExternalSecret. For now, OTLP without auth only requires a new variable wired into the config.

**No `batch` processor in pipelines** — despite `batch` being declared in `processors` (`otelconfig.tf` lines 17–21), it is **not referenced in any pipeline's `processors` list** (all three pipelines have `processors = []`). This is a latent bug; the batch processor is configured but silently unused.

**`send_batch_max_size = 75`** — recent commit history ("lower batch size", "lower to 75") indicates this was recently tuned, suggesting active performance work on the processor tier.