otelcol: drop unparseable container logs quietly + filter nil container.name

[samcm]

Filelog container operator was logging add_metadata_from_filepath failures at error level; otelcol then tailed its own stderr and re-shipped those stack traces (~10x amplification). on_error: drop_quiet silences them and the inverted filter expr drops entries with nil container.name as a safety net.

Mirrors ethpandaops/blob-devnets#15. Only devnet-3 has the otelcol_contrib_config block.

[qu0b-reviewer]

🤖 qu0b-reviewer

Summary

Two changes: added on_error: drop_quiet to the container operator (good — silently drops unparseable logs instead of errors), and flip-flopped the filter expr from != nil and matches to == nil or matches. The filter inversion changes pass/drop behavior for nil container.name logs from drop to pass — this is either a logical error or an intentional split, but the code comments and PR title strongly suggest the author believes they're equivalent (they're not).

Issues

• 🟡 ansible/inventories/devnet-3/group_vars/all/all.yaml:341 — Filter change is not logically equivalent to the stated goal.
  attributes["container.name"] == nil or attributes["container.name"] matches "^(otelcol|...)$" vs. old attributes["container.name"] != nil and attributes["container.name"] matches "^(otelcol|...)$"

  For nil container names, OLD drops, NEW passes. If the goal is to drop logs from containers whose name failed to parse (drop_quiet on the container operator), adding == nil to the pass list is counterproductive — you're passing those same nil logs downstream through the pipeline instead of filtering them out.
  If the intent is "keep logs from known containers or logs that failed container parsing," this achieves that. But it's a semantic change that goes beyond the PR title's claim, and it's potentially unintended.

• 🟢 ansible/inventories/devnet-3/group_vars/all/all.yaml:341 — The old != nil and matches pattern (John's law: keep only known-infra containers) is now completely replaced. There's no comment explaining why nil container names should be passed. A one-line # keep known infra, or logs without container name (e.g. OTLP-ingested) would prevent future confusion.

Suggestions

• Consider two filters instead of one combined expr:
  - type: filter with expr: 'attributes["container.name"] != nil' (drop unparseable != nil that also didn't match known pattern), followed by the original != nil and matches filter. This separates concerns and makes each filter's purpose auditable.
  OR: if you genuinely want to pass-through nil container.name logs (because something else handles them), document why.

---

_{Reviewed @ c48c7523
"Sleep is for people without children."}