How does builder.Configuration.AddDaprSecretStore Work?

[sameert89]

When the app and Dapr start at different times the app has to wait for getting secrets and Dapr has to wait for the app to start listening if I understand it right. This appears like a circular dependency then how exactly does this function work?

There can be network delay and there is a Wait timeout option which I am guessing is for the time taken to get the secrets.

So does it pause the host startup until then? Sorry for creating this as an issue but I can't seem to find much documentation around it.

[WhitWaldo]

Host startup is intentionally delayed until the load completes. It shouldn't take that long to retrieve and populate the secrets themselves - the delay is really just waiting for healthiness to be reported, and there shouldn't be much of a delay in a healthy environment.

The timeout value you mentioned puts an upper bound on waiting for failed retries to complete.

[sameert89]

Host startup is intentionally delayed until the load completes. It shouldn't take that long to retrieve and populate the secrets themselves - the delay is really just waiting for healthiness to be reported, and there shouldn't be much of a delay in a healthy environment.

The timeout value you mentioned puts an upper bound on waiting for failed retries to complete.

Thanks for the explanation, but what if the application is started before Dapr? How would the secrets be fetched if Dapr is not ready. I am confused about the circular dependency/race condition I mentioned.

I read this in an old issue: #452 (comment)

But can't understand how it's handled, because app won't be up until this function succeeds.

[WhitWaldo]

Host startup is intentionally delayed until the load completes. It shouldn't take that long to retrieve and populate the secrets themselves - the delay is really just waiting for healthiness to be reported, and there shouldn't be much of a delay in a healthy environment.
The timeout value you mentioned puts an upper bound on waiting for failed retries to complete.

Thanks for the explanation, but what if the application is started before Dapr? How would the secrets be fetched if Dapr is not ready. I am confused about the circular dependency/race condition I mentioned.

I read this in an old issue: #452 (comment)

But can't understand how it's handled, because app won't be up until this function succeeds.

Whoops, yes, I neglected to speak to that.

Put simply, you cannot use any secrets from the client which might result in such a circular dependency and that makes sense, right? Take the following example:

• You require an authentication token to communicate with the Dapr sidecar
• You intend to retrieve this token from your remote secret store
• You register your DaprClient instance with the DI system and specify that the authentication token should come from your injected IConfiguration instance.
• When you configure the secret store using AddDaprSecretStore, you create a scope and pass in a DI-retrieved instance of your DaprClient

But of course when your code runs through AddDaprSecretStore, it'll fault because the key you expect from your IConfiguration isn't present because this module was intended to populate it (and it hasn't run yet).

So yes, you have to be careful about specifically which secrets you need and where you're sourcing them especially during startup. Personally, I put my secrets in Azure Key Vault and typically access them via the secret store functionality, except that I specifically register the Dapr API token as a referenced Key Vault Secret in the environment variables for my service (I often use Azure Container Apps here).

By doing this, you can instead opt to read in your environment variables at the top (populating this as authentication happens at the infrastructure level instead):

var builder = WebApplication.CreateBuilder(args);

builder.Configuration.AddEnvironmentVariables();

builder.Services.AddDaprClient((serviceProvider, daprClientBuilder) => {
  var configuration = serviceProvider.GetRequiredService<IConfiguration>();
  var daprApiToken = configuration["DaprApiToken"];

  daprClientBuilder.UseDaprApiToken(daprApiToken);
});

//...

var app = builder.Build();

//Provision an instance of the DaprClient to use
await using var scope = app.Services.CreateAsyncScope();
var daprClient = scope.Services.GetRequiredService<DaprClient>();

builder.Configuration. AddDaprSecretStore("my-store", [], daprClient);

Now if you strictly follow this sort of structure and you're definitely not injecting (and thus not resolving) any other services that need secrets from IConfiguration, they'll be pulled at resolution-time and should populate just fine.

[sameert89]

Thanks for the excellent explanation. So to summarize:

1. We need to keep secrets that can stop app.build() accessible in local environment.
2. For other secrets we can call AddDaprSecretStore() anytime before app.Run() and it will populate it.

So point 2 implies Dapr will serve the secrets even before app.Run() is called i.e. application has started listening.

[WhitWaldo]

Well, they're not going to stop app.Build() because Microsoft.Extensions.DependencyInjection doesn't immediately resolve the references at build - it simply finalizes the container. Only when you actually use any of the registered entities the first time does it resolve it and then cache or not depending on the registration (e.g. singleton, scope, transient).

As demonstrated in my example above then, that has the less-than-obvious, but helpful side effect that you can continue to populate your configuration after the container is built so long as you also don't force the resolution of anything that's going to take that IConfiguration dependency.

Dapr will pull the secrets specified once the builder.Configuration.AddDaprSecretStore has successfully finished loading (e.g. health/readiness check completes against the sidecar) - while this pull happens asynchronously, this is done as a blocking operation so your very next operation doesn't attempt to resolve while the pull is underway (as that would cause some real problems).

So I'd instead sum up with:

1. Keep Dapr API key available in the local environment (as this impacts communication with the sidecar)
2. If you aren't using a managed identity and need a secret to authenticate to the secret store, you'll need this in the local environment as well
3. For other secrets, you'll need AddDaprSecretStore after app.Build() so the DaprClient is resolvable (unless you're statically building it just for this purpose). For anything depending on these secrets to populate, don't resolve them until after AddDaprSecretStore has loaded.

[sameert89]

Thanks, that clears it all!

[WhitWaldo]

Please don't hesitate to reach out if you have any other questions!