[QUERY] How to use Linked Services with Cosmos DB Spark connector Throughput control in Synapse Analytics

[golfalot]

Query/Question

What is the equivalent to spark.synapse.linkedService for use with spark.cosmos.throughputControl.globalControl.database

I need to use private endpoints because of DEP, and having credentials stored with Linked Services is very handy.

Why is this not a Bug or a feature Request?

Feels like I'm missing something really obvious, but the docs don't seem to cover what I'm seeking,

Setup (please complete the following information if applicable):

• OS: Spark 3.4 Python 3.10 Scala
• IDE: Synapse Analytics Web based
• Library/Libraries: default Spark Pool config

PySpark

(
    df_floodre
    .withColumn('id', F.col('uprn').cast('string'))
    .write
    .format("cosmos.oltp")
    .mode("Append")
    .option("spark.synapse.linkedService", "CosmosDB_GraphQL_databasets")
    .option("spark.cosmos.container", 'floodRe')
    .option('spark.cosmos.throughputControl.enabled','true')
    .option('spark.cosmos.throughputControl.name','floodRe_Loader_Jobs_Group') 
    .option('spark.cosmos.throughputControl.globalControl.database','CosmosDB_GraphQL_databasets_ThroughputControl') 
    .option('spark.cosmos.throughputControl.globalControl.container','ThroughputControl-floodre')
    .option('spark.cosmos.throughputControl.targetThroughput',7000)
    .save()
)

I've tried a few likely nonsensical permutations and got 403 or 404 errors. Guidance much appreciated.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @kushagraThapar @pjohari-ms @TheovanKraay.

[kushagraThapar]

@tvaron3 please take a look at this, thanks!