java.io.FileNotFoundException: /sys/fs/cgroup/cpuacct/cpuacct.usage (No such file or directory)

[cometta]

my executor threw above error message . i check the folder in the docker container , does not have such file. Available files are

 cpu.idle               cpu.max.burst          cpuset.cpus            cpuset.cpus.partition  cpuset.mems.effective  cpu.uclamp.max         cpu.weight
cpu.max                cpu.pressure           cpuset.cpus.effective  cpuset.mems            cpu.stat               cpu.uclamp.min         cpu.weight.nice

can advice?

i'm using docker image apache/spark:3.5.1

[LucaCanali]

I cannot reproduce your error, maybe you can give more details about the configuration that throws this error.
I have checked apache/spark:3.5.1 and it does have /sys/fs/cgroup/cpuacct/cpuacct.usage

[cometta]

@LucaCanali does the issue caused by apache/spark:3.5.1 does not have that file? may i know when run your docker image in kubernetes, do you need to run on priviledge mode?

my config

    "spark.jars": "https://repo1.maven.org/maven2/ch/cern/sparkmeasure/spark-measure_2.12/0.24/spark-measure_2.12-0.24.jar"

     "spark.jars.packages": "ch.cern.sparkmeasure:spark-measure_2.12:0.24,ch.cern.sparkmeasure:spark-plugins_2.12:0.3",
    "spark.plugins": "ch.cern.HDFSMetrics,ch.cern.CgroupMetrics,ch.cern.CloudFSMetrics",
    "spark.cernSparkPlugin.cloudFsName": "s3a",
    "spark.executor.metrics.fileSystemSchemes": "file"

[cometta]

may i know which version of k8s are you using. might be related. see this kubernetes/kubernetes#119669 (comment)

[LucaCanali]

I have been testing this with Kubernetes up to v 1.25.3 so far + using cgroups v1.

[cometta]

i think only happen in cgroups v2