java.lang.NoClassDefFoundError: Could not initialize class sun.net.www.protocol.https.DelegateHttpsURLConnection #3691
Comments
github-actions
bot
added
agent-java
community
|
I assume it's started with Can you try starting the agent with a delay, eg adding |
|
Sorry no, this was programmatically attached in the WebApplicationInitializer of a Spring MVC war. |
Then can you delay this attachment (e.g. schedule it on a |
|
With that delay we're seeing a different error We are logging tomcat with log4j2 not juli. |
|
That is very weird. The agent internally uses log4j2 aswell, but it loads it in an isolated classloader.
Just as a test/workaround, could you try adding |
|
@JonasKunz with that change the apm agent is working. My full changes were: import co.elastic.apm.attach.ElasticApmAttacher;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
import org.springframework.web.WebApplicationInitializer;
import org.springframework.web.context.ContextLoaderListener;
import org.springframework.web.context.support.AnnotationConfigWebApplicationContext;
import javax.servlet.ServletContext;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
public class WebAppInitializer implements WebApplicationInitializer {
+ private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
+ private static final Logger LOG = LogManager.getLogger(WebAppInitializer.class);
@Override
public void onStartup(ServletContext container) {
if ("true".equalsIgnoreCase(System.getProperty("elastic.apm.enabled"))) {
- ElasticApmAttacher.attach();
+ try {
+ Class.forName("sun.net.www.protocol.http.HttpURLConnection", true, getClass().getClassLoader());
+ } catch (ClassNotFoundException e) {
+ LOG.warn("TEST: Failed to load HttpURLConnection class", e);
+ }
+ scheduler.schedule(() -> ElasticApmAttacher.attach(), 10, java.util.concurrent.TimeUnit.SECONDS);
} |
|
I think you can also get rid of the delay with this workaround. There must be something unusual around classloading in your setup, because we have many users (of which I'd assume many to have the log4j2-jul-bridge setup) with tomcat and haven't encountered this problem before. If you are able to provide a minimal sample project to reproduce this problem that would be great, this way we could attempt to fix the root cause. |
|
That may be a tricky ask, as I mentioned I haven't been able to reproduce outside of k8s yet; the same applications running on my local system with the same openjdk and tomcat versions do not reproduce. I'll see if I can get anywhere with docker... |
|
I made a docker image using the same tomcat base image that we deploy to k8s and the same application but wasn't able to reproduce there either unfortunately. We have paid support as well and I provided them with debug logs but it wasn't helpful. This seems to be a very tricky case. |
|
Do you have a workaround that's sufficiently valid for you? If so, we may just leave this as not priority |
|
I am now able to reproduce in docker with a minimal application. The base tomcat and jdk images we use are not public so I don't yet have one I can share here; but we will be pursing it further with paid support. Otherwise the workaround @JonasKunz provided is sufficient for us. |
|
Zip file contains a docker directory with Dockerfile and a very small JAVA application that can reproduce the issue. |
|
Reproduces the issue, thanks! |
|
The apache/logging-log4j2#2112 change in log4j2 (added in 2.23.0) changed how the lmax disruptor is handled. Because of how log4j2 initializes and how the agent initializes, there is a race condition in which classloader loads the various classes, and in this specific scenario it causes a failure where the class is loaded in both class loaders but an attempt is made to use the class from one classloader in the other. The fix to revert to the log4j behaviour is simple but needs to be done in log4j2. A workaround in the agent would be the same as above (load |
Describe the bug
This is an odd one, I'm seeing it in our k8s environments but not on my local machine and I don't have criteria for it, but it happens every time / with every war we're running.
The jdk is:
The server is
Apache Tomcat/9.0.87Elastic is configured through jvm arguments as follows:
-Delastic.apm.enabled=true -Delastic.apm.service_name=rssweb -Delastic.apm.application_packages=com.example -Delastic.apm.server_url=https://redacted.example.com/apm-server -Delastic.apm.api_key=<redacted> -Delastic.apm.environment=example -Delastic.apm.log_level=OFF -Delastic.apm.disable_instrumentations=spring-service-name -Delastic.apm.ignore_urls=*/about.htmlElastic apm agent is 1.50.0
The text was updated successfully, but these errors were encountered: