Replay of result from injected service in Orchestrator function

[FFranz93]

Since I don't know if this would be a valid feature request, I just wanted to ask if this would make sense and if anyone else has an opinion about that.

So here is my problem:

My company is using a custom solution for end-to-end logging, and my team was asked to use this in our APIs that are implemented as Durable Functions (at least most of them).

To test this, we implemented an interface and a service class in one of our Function Apps, to see if things work as expected because we thought about creating a NuGet package from our solution, since we do not want to copy and paste the code to all our projects.

The steps this logging service does to log to the solution my company uses are the following:

• Start a process on the logging server (each log entry there is represented as a process, and a process contains all steps that are required for this specific process)
• For all activity functions called by our orchestrator function create a new step within the process (with the name of the step as well as some information about the step)
• Stop the process depending on the result of the orchestration (if the orchestrator function catches an exception, the process is marked as failure, if not it's marked as successful)

Since the logging application needs to keep track of what activity belongs to which process, it returns a ProcessId after the initialization.

So in our orchestrator function, we came up with the following solution (just a small example):

internal class MyOrchestrator
{
    private const string WorkflowName = "MyWorkflow";
    private readonly IExampleLogger _logger;

    public MyOrchestrator(IExampleLogger  logger)
    {
        _logger = logger;
    }

    [FunctionName(nameof(MyOrchestrator))]
    public async Task Run([OrchestrationTrigger] IDurableOrchestrationContext context)
    {
        var processId = await _logger.StartProcess(WorkflowName);
        try
        {
           //Execute all activities
        }
        catch (Exception ex)
        {
            _logger.StopProcess(isSuccess: false);
        }

        _logger.StopProcess(isSuccess: true);
    }
}

The problem we encountered is that (unlike the execution of activity functions) the result of the service invocation is never replayed.
So every time the control is yield back to the orchestrator, it executes the start of the process again, so we end up with n-processes (one for each activity executed by the orchestrator), instead of a single process that shows the complete workflow with all its activities.

So my question is, would it be possible to replay the result returned by service classes?
We know that we could also implement logging as activities, but this would kind of keep us from creating a NuGet package for the logging stuff (or at least we would have to create and maintain 2 different packages, one that implements logging as activities (if possible) and one that has a logging service and provides and interface that can be injected).

[ltouro]

You should be doing something like that instead:

var processId = context.CallActivityAsync(nameof(StartLoggerProcess), WorkflowName);
_logger.StartProcess(processId);

That is: for replay behaviour, you have to use CallActivityAsync

You can also hide that call to CallActivityAsync behing the logger if you really want to (it will need the context):

var processId = await _logger.StartProcess(WorkflowName, context);

[FFranz93]

Thanks for your response.

I'm aware of the possibility to do that with an activity function, because then we would have the expected replay behaviour, and it would work as expected.

The thing I was wondering about is, if it would even be possible to adjust the Durable Functions Framework in a way that it also replays the response the orchestrator gets from calling service classes.

Since we have multiple Function Apps (and other projects), we don't want to copy and paste the necessary activity functions to all projects.
I suppose we could even create a NuGet package from these activities, and use them in the Function Apps, but for the other projects we would have to create a separate package that offers and interface that can be injected.
This would lead to a duplication in effort whenever the logging would have to be extended or somehow adjusted.

[ltouro]

I see. Can't you use the instance ID for your logger in that case?

Anyway, you can do your best to hide the CallActivityAsync method from the orchestration code, but if you want to rely on the replay behavior, it will have to be somewhere in the implementation.

Maybe you could use objects like proxies to hide this implementation detail and make the code look more like any other C# code. For that, you would need to hook the orchestration context instance in the DI container so you could pull it out in each proxy implementation as a scoped dependency (not sure it is possible currently).

However, since the coding constraints for Durable orchestration are so important, do your best to make activity concept (and its trade-offs) explicit, especially when considering changes to code that is already deployed and running. If generating logger ID is an important step in the process, that should be clear. I think that passing the workflow context explicitly to a function call can help with that.

[FFranz93]

Sorry for the late response!

Since I had not much time left to investigate on this, I just followed the general design pattern for Durable Functions and implemented the logging as an Activity Function.

Thanks for your help and your advice!