Correctly propagate orchestration results of a failed orchestration

[sebastianburckhardt]

As diagnosed in #2743, the current code in OutOfProcMiddleware.CallOrchestratorAsync does not correctly propagate the results of a failed orchestration to the backend.

The problem is that when an orchestrator function fails, the OrchestratorExecutionResult that was constructed by the executor is ignored, and instead an alternate OrchestratorExecutionResult.ForFailure is constructed from the exception returned by the function result. This is a problem because

1. the original failure details are lost instead of being persisted in the history.
2. any orchestrator actions that took place prior to the failure (e.g. sending lock release messages) are ignored.

This PR fixes the problem by propagating the original OrchestratorExecutionResult that describes the failure (if there is one).

• Otherwise: I've added my notes to release_notes.md
• I have added all required tests (Unit tests, E2E tests)
• My changes do not change the version of the WebJobs.Extensions.DurableTask package
• My changes should be added to v3.x branch.

[lilyjma]

@cgillum / @jviau - requested for your review because there are some customers asking about this. There's a DF release starting next week, so it'd be great if this fix could go out with that.

[cgillum]

LGTM.

I'm working on a process for creating end-to-end tests. Is there an orchestration that you used for testing this fix which we can borrow and include as part of an end-to-end test? (Creating such a test can be done separately from this PR - I'm just asking now in case I forget later)

[sebastianburckhardt]

Is there an orchestration that you used for testing this fix which we can borrow and include as part of an end-to-end test?

Yes. All the tests are in here: #2612
The specific test that would have detected this problem is called FaultyCriticalSection.