Possible service interruption due to Java agent and impact to host JVM??

[jamey-clari]

I was wondering what the failure modes might be for the OTEL java agent and, namely, if it can fail in a way that could impact the service JVM which it is instrumenting? In short, because it intercepts inbound HTTP requests for header processing and to propagate distributed tracing context can it be hung in a way that could result in a service interruption to the host JVM/application. We are trying to debug recent and, at this point, mysterious service interruption seemingly between our load balancer and Java application. I'm trying to imagine ways that inbound requests could be throttled prior to reaching the main application code. We are currently on v 0.8.0 but hoping to shortly soon move to 0.13.1 or whatever is most recent and compatible with the most recent release of the New Relic exporter. Thanks for any input!

[trask]

hey @jamey-clari! sorry for the late reply. since the javaagent runs inside the jvm, it's possible for it to affect your application. i'd recommend looking at gc logs to make sure the jvm isn't running OOM and ending up in stop-the-world GC events. if it is, then i'd recommend capturing and analyzing a heap dump to see if the javaagent is leaking memory. if there's no GC issues, then i'd look at some thread dumps from the JVM to see if you can get an idea where the JVM is blocked.

[jamey-clari]

Thanks @trask - I believe we have this pinned to another of our recent changes and thus are moving forward on the assumption the agent is behaving correctly, thanks!