Help with Opentelemetry setup when using Lambda RIE locally with docker compose services

[ericchaves]

Hi Folks,

I'm building and AI AGent on a AWS Lambda using Strands Agents and a couple of other frameworks.
My Agent is writen in Python and I need to send traces and spans to two different OTEL collectors: one is a backend used to monitor the application "logic" (ie graphana) and the other is a dedicated servie to monitor Agent/LLM metrics (Phoenix OSS from Arize) .

During my development lifecycle, to ensure everything is ok I run both otel collectores as docker on my laptop. Than I perform some unit/functional tests invoking pytest whose tests imports and executes my lambda handler directly.

Once everything is ok I ran some e2e tests by building a docker image for the lambda using the public.ecr.aws/lambda/python:3.12 as base image and then I invoke the lambda using RIE by posting some HTTP requests.

The issue I'm facing is that traces in collector service invoking RIE are not handled similarly from traces generated by invoking pytests. When I access the Phoneix dashboards and console, traces from pytests are handled properly and are displayed as expected.

But when I lookup traces feeded from invocations made through RIE, the span traces are there but they are shown like if some openinference attributes were never set (or never reached the server).

Given that the code, the OTEL setup, environment variables in both cases are the same, and the only difference I can think of is the invocation method, I'm thinking if there is any OTEL specific configuration in RIE that I should be doing?

Does anyone has any idea on what I could investigate?

Thanks in advance for any help!
If needed I can provide some screenshots to ilustrate the issue further.