Spark job Running forever after completed

[sontt710]

Hello,

I try to use appender with the following version to submit spark job (spark-pi example) on k8s, but the job running forever and not shutdown after complete, can you help me to check

Version

Spark 3.3.4
Log4j2
Appender 5.3.2
hadoop-aws-3.3.2.jar
aws-java-sdk-bundle-1.11.1026.jar
aws-java-sdk-s3-1.12.772.jar

Driver log

[bluedenim]

If you are quitting a process, see this to shut down: https://github.com/bluedenim/log4j-s3-search-samples/blob/master/appender-log4j2-sample/src/main/java/com/van/example/Main.java#L43

[sontt710]

Thanks @bluedenim, I understand, I will have to shut it down myself in the application, there's no way to configure it from the log4j2 XML file?

[bluedenim]

There is no easy way for the event log thread to know when your program should end. Only your program knows when it's time to end.