[CDAP-21070] Wrap input/output format methods to throw wrapped stage exception

[itsankit-google]

Tested in sandbox:

2024-10-03 14:14:14,445 - ERROR [SparkRunner-phase-1:i.c.c.i.a.r.ProgramControllerServiceAdapter@98] - Spark Program 'phase-1' failed.
java.util.concurrent.ExecutionException: io.cdap.cdap.api.exception.WrappedStageException: Stage 'GCS Multi File' encountered : io.cdap.cdap.api.exception.ProgramFailureException: xxxxxx.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object. Permission 'storage.objects.get' denied on resource (or it may not exist).
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
	at io.cdap.cdap.app.runtime.spark.submit.AbstractSparkJobFuture.get(AbstractSparkJobFuture.java:119)
	at io.cdap.cdap.app.runtime.spark.SparkRuntimeService.run(SparkRuntimeService.java:444)
	at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
	at io.cdap.cdap.app.runtime.spark.SparkRuntimeService.lambda$null$2(SparkRuntimeService.java:525)
	at java.lang.Thread.run(Thread.java:750)
Caused by: io.cdap.cdap.api.exception.WrappedStageException: Stage 'GCS Multi File' encountered : io.cdap.cdap.api.exception.ProgramFailureException: xxxxxx.iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object. Permission 'storage.objects.get' denied on resource (or it may not exist).
	at io.cdap.cdap.etl.spark.io.StageTrackingOutputCommitter.setupJob(StageTrackingOutputCommitter.java:52)
	at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.setupJob(HadoopMapReduceCommitProtocol.scala:188)
	at org.apache.spark.internal.io.SparkHadoopWriter$.write(SparkHadoopWriter.scala:79)
	at org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsNewAPIHadoopDataset$1(PairRDDFunctions.scala:1078)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)

2024-10-03 14:31:24,648 - ERROR [SparkRunner-phase-1:i.c.c.i.a.r.ProgramControllerServiceAdapter@98] - Spark Program 'phase-1' failed.
java.util.concurrent.ExecutionException: io.cdap.cdap.api.exception.WrappedStageException: Stage 'NYT Best Sellers Raw Data' encountered : java.io.IOException: Failed to get result: java.io.IOException: Error listing gs://xxxxx
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
	at io.cdap.cdap.app.runtime.spark.submit.AbstractSparkJobFuture.get(AbstractSparkJobFuture.java:119)
	at io.cdap.cdap.app.runtime.spark.SparkRuntimeService.run(SparkRuntimeService.java:444)
	at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52)
	at io.cdap.cdap.app.runtime.spark.SparkRuntimeService.lambda$null$2(SparkRuntimeService.java:525)
	at java.lang.Thread.run(Thread.java:750)
Caused by: io.cdap.cdap.api.exception.WrappedStageException: Stage 'NYT Best Sellers Raw Data' encountered : java.io.IOException: Failed to get result: java.io.IOException: Error listing gs://xxxxx
	at io.cdap.cdap.etl.spark.io.StageTrackingInputFormat.getSplits(StageTrackingInputFormat.java:54)
	at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:136)
	at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:292)
	at scala.Option.getOrElse(Option.scala:189)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:288)

[sonarcloud]

Quality Gate failed

Failed conditions
65.4% Coverage on New Code (required ≥ 80%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint