[Feature Request]: Update Java ExpansionService to use arbitrary PipelineOptions set through an ExpansionRequest

[chamikaramj]

What would you like to happen?

Currently ExpansionService only forwards specific selected options to the transforms being expanded.

beam/sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionService.java

Line 727 in 738a76d

PipelineOptions effectiveOpts = PipelineOptionsFactory.create();

We should update this so that arbitrary PipelineOptions, sent through an ExpansionRequest are made available the the transforms.

beam/model/job-management/src/main/proto/org/apache/beam/model/job_management/v1/beam_expansion_api.proto

Line 64 in 738a76d

google.protobuf.Struct pipeline_options = 6;

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner