Skip to content

Commit

Permalink
Merge branch 'v1.13' into bulk-retrieval-deserialization
Browse files Browse the repository at this point in the history
  • Loading branch information
hhunter-ms authored Nov 21, 2023
2 parents 625623e + 869e908 commit 1195794
Show file tree
Hide file tree
Showing 109 changed files with 869 additions and 638 deletions.
6 changes: 6 additions & 0 deletions .github/holopin.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
organization: dapr
defaultSticker: clmjkxscc122740fl0mkmb7egi
stickers:
-
id: clmjkxscc122740fl0mkmb7egi
alias: ghc2023
14 changes: 13 additions & 1 deletion daprdocs/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,18 @@ id = "G-60C6Q1ETC1"
source = "../translations/docs-zh/content/sdks_dotnet"
target = "content/developing-applications/sdks/dotnet"
lang = "zh-hans"
[[module.mounts]]
source = "../translations/docs-zh/content/sdks_java"
target = "content/developing-applications/sdks/java"
lang = "zh-hans"
[[module.mounts]]
source = "../translations/docs-zh/content/sdks_go"
target = "content/developing-applications/sdks/go"
lang = "zh-hans"
[[module.mounts]]
source = "../translations/docs-zh/content/sdks_js"
target = "content/developing-applications/sdks/js"
lang = "zh-hans"

# Markdown Engine - Allow inline html
[markup]
Expand Down Expand Up @@ -181,7 +193,7 @@ url_latest_version = "https://docs.dapr.io"
url = "#"
[[params.versions]]
version = "v1.12 (latest)"
url = "https://docs.dapr.io"
url = "#"
[[params.versions]]
version = "v1.11"
url = "https://v1-11.docs.dapr.io"
Expand Down
2 changes: 1 addition & 1 deletion daprdocs/content/en/concepts/security-concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ The diagram below shows how the Sentry system service issues certificates for ap

### Preventing IP addresses on Dapr

To prevent Dapr sidecars from being called on any IP address (especially in production environments such as Kubernetes), Dapr restricts its listening IP addresses only to `localhost`. Use the [dapr-listen-addresses]({{<ref arguments-annotations-overview>}}) setting you need to enable other addresses.
To prevent Dapr sidecars from being called on any IP address (especially in production environments such as Kubernetes), Dapr restricts its listening IP addresses to `localhost`. Use the [dapr-listen-addresses]({{<ref arguments-annotations-overview>}}) setting if you need to enable access from external addresses.

## Secure Dapr to application communication

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ public void ConfigureServices(IServiceCollection services)
services.AddSingleton<BankService>();
}
```
[See the .NET SDK documentation on registring actors]({{< ref "dotnet-actors-usage.md#registring-actors" >}}).
[See the .NET SDK documentation on registering actors]({{< ref "dotnet-actors-usage.md#registring-actors" >}}).

{{% /codetab %}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Refer [api spec]({{< ref "actors_api.md#invoke-reminder" >}}) for more details.

## Error handling

When an actor's method completes successfully, the runtime will contineu to invoke the method at the specified timer or reminder schedule. However, if the method throws an exception, the runtime catches it and logs the error message in the Dapr sidecar logs, without retrying.
When an actor's method completes successfully, the runtime will continue to invoke the method at the specified timer or reminder schedule. However, if the method throws an exception, the runtime catches it and logs the error message in the Dapr sidecar logs, without retrying.

To allow actors to recover from failures and retry after a crash or restart, you can persist an actor's state by configuring a state store, like Redis or Azure Cosmos DB.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ public void ConfigureServices(IServiceCollection services)
}
```

[See the .NET SDK documentation on registring actors]({{< ref "dotnet-actors-usage.md#registring-actors" >}}).
[See the .NET SDK documentation on registering actors]({{< ref "dotnet-actors-usage.md#registring-actors" >}}).

{{% /codetab %}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Alternatively, you can use [Dapr SDKs to use actors]({{< ref "developing-applica

## Save state with actors

You can interact with Dapr via HTTP/gRPC endpoints to save state reliably using the Dapr actor state mangement capabaility.
You can interact with Dapr via HTTP/gRPC endpoints to save state reliably using the Dapr actor state management capabaility.

To use actors, your state store must support multi-item transactions. This means your state store component must implement the `TransactionalStore` interface.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,14 @@ Read the [Use output bindings to interface with external resources guide]({{< re

## Binding directions (optional)

You can provide the `direction` metadata field to indicate the direction(s) supported by the binding component. In doing so, the Dapr sidecar avoids the `"wait for the app to become ready"` state reducing the lifecycle dependency between the Dapr sidecar and the application:
You can provide the `direction` metadata field to indicate the direction(s) supported by the binding component. In doing so, the Dapr sidecar avoids the `"wait for the app to become ready"` state, reducing the lifecycle dependency between the Dapr sidecar and the application:

- `"input"`
- `"output"`
- `"input, output"`

{{% alert title="Note" color="primary" %}}
It is highly recommended that all bindings should include the `direction` property.
It is highly recommended that all input bindings should include the `direction` property.
{{% /alert %}}

[See a full example of the bindings `direction` metadata.]({{< ref "bindings_api.md#binding-direction-optional" >}})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ While both HTTP and gRPC are supported in the alpha release, using the gRPC APIs

### Cryptographic components

The Dapr cryptography building block incldues two kinds of components:
The Dapr cryptography building block includes two kinds of components:

- **Components that allow interacting with management services or vaults ("key vaults").**
Similar to how Dapr offers an "abstraction layer" on top of various secret stores or state stores, these components allow interacting with various key vaults such as Azure Key Vault (with more coming in future Dapr releases). With these components, cryptographic operations on the private keys are performed within the vaults and Dapr never sees your private keys.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,13 @@ async function start() {
{
entryID: "entry-2",
contentType: "application/cloudevents+json",
event: {
event: {
specversion: "1.0",
source: "/some/source",
type: "example",
id: "1234",
data: "foo message 2",
datacontenttype: "text/plain"
id: "1234",
data: "foo message 2",
datacontenttype: "text/plain"
},
},
{
Expand Down Expand Up @@ -115,7 +115,7 @@ using System.Collections.Generic;
using Dapr.Client;

const string PubsubName = "my-pubsub-name";
const string TopicName = "topic-a";
const string TopicName = "topic-a";
IReadOnlyList<object> BulkPublishData = new List<object>() {
new { Id = "17", Amount = 10m },
new { Id = "18", Amount = 20m },
Expand All @@ -130,10 +130,10 @@ if (res == null) {
}
if (res.FailedEntries.Count > 0)
{
Console.WriteLine("Some events failed to be published!");
Console.WriteLine("Some events failed to be published!");
foreach (var failedEntry in res.FailedEntries)
{
Console.WriteLine("EntryId: " + failedEntry.Entry.EntryId + " Error message: " +
Console.WriteLine("EntryId: " + failedEntry.Entry.EntryId + " Error message: " +
failedEntry.ErrorMessage);
}
}
Expand Down Expand Up @@ -205,7 +205,7 @@ func main() {
{
"entryId": "b1f40bd6-4af2-11ed-b878-0242ac120002",
"event": {
"message": "second JSON message"
"message": "second JSON message"
},
"contentType": "application/json"
}
Expand Down Expand Up @@ -236,7 +236,7 @@ curl -X POST http://localhost:3500/v1.0-alpha1/publish/bulk/my-pubsub-name/topic
{
"entryId": "b1f40bd6-4af2-11ed-b878-0242ac120002",
"event": {
"message": "second JSON message"
"message": "second JSON message"
},
"contentType": "application/json"
},
Expand All @@ -258,7 +258,7 @@ Invoke-RestMethod -Method Post -ContentType 'application/json' -Uri 'http://loca
{
"entryId": "b1f40bd6-4af2-11ed-b878-0242ac120002",
"event": {
"message": "second JSON message"
"message": "second JSON message"
},
"contentType": "application/json"
},
Expand All @@ -271,7 +271,7 @@ Invoke-RestMethod -Method Post -ContentType 'application/json' -Uri 'http://loca

## Subscribing messages in bulk

The bulk subscribe API allows you to subscribe multiple messages from a topic in a single request.
The bulk subscribe API allows you to subscribe multiple messages from a topic in a single request.
As we know from [How to: Publish & Subscribe to topics]({{< ref howto-publish-subscribe.md >}}), there are two ways to subscribe to topic(s):

- **Declaratively** - subscriptions are defined in an external file.
Expand All @@ -286,7 +286,7 @@ metadata:
name: order-pub-sub
spec:
topic: orders
routes:
routes:
default: /checkout
pubsubname: order-pub-sub
bulkSubscribe:
Expand All @@ -300,11 +300,11 @@ scopes:
In the example above, `bulkSubscribe` is _optional_. If you use `bulkSubscribe`, then:
- `enabled` is mandatory and enables or disables bulk subscriptions on this topic
- You can optionally configure the max number of messages (`maxMessagesCount`) delivered in a bulk message.
Default value of `maxMessagesCount` for components not supporting bulk subscribe is 100 i.e. for default bulk events between App and Dapr. Please refer [How components handle publishing and subscribing to bulk messages]({{< ref pubsub-bulk >}}).
- You can optionally configure the max number of messages (`maxMessagesCount`) delivered in a bulk message.
Default value of `maxMessagesCount` for components not supporting bulk subscribe is 100 i.e. for default bulk events between App and Dapr. Please refer [How components handle publishing and subscribing to bulk messages]({{< ref pubsub-bulk >}}).
If a component supports bulk subscribe, then default value for this parameter can be found in that component doc.
- You can optionally provide the max duration to wait (`maxAwaitDurationMs`) before a bulk message is sent to the app.
Default value of `maxAwaitDurationMs` for components not supporting bulk subscribe is 1000 i.e. for default bulk events between App and Dapr. Please refer [How components handle publishing and subscribing to bulk messages]({{< ref pubsub-bulk >}}).
Default value of `maxAwaitDurationMs` for components not supporting bulk subscribe is 1000 i.e. for default bulk events between App and Dapr. Please refer [How components handle publishing and subscribing to bulk messages]({{< ref pubsub-bulk >}}).
If a component supports bulk subscribe, then default value for this parameter can be found in that component doc.

The application receives an `EntryId` associated with each entry (individual message) in the bulk message. This `EntryId` must be used by the app to communicate the status of that particular entry. If the app fails to notify on an `EntryId` status, it's considered a `RETRY`.
Expand All @@ -313,16 +313,16 @@ A JSON-encoded payload body with the processing status against each entry needs

```json
{
"statuses":
[
"statuses":
[
{
"entryId": "<entryId1>",
"status": "<status>"
},
},
{
"entryId": "<entryId2>",
"status": "<status>"
}
}
]
}
```
Expand Down Expand Up @@ -477,37 +477,21 @@ For event publish/subscribe, two kinds of network transfers are involved.
1. From/To *App* To/From *Dapr*.
1. From/To *Dapr* To/From *Pubsub Broker*.

These are the opportunities where optimization is possible. When optimized, a Bulk requests are, which reduce number of overall calls and thus increase throughput and provide better latency.
These are the opportunities where optimization is possible. When optimized, Bulk requests are made, which reduce the overall number of calls and thus increases throughput and provides better latency.

On enabling Bulk Publish and/or Bulk Subscribe, the communication between the App and Dapr sidecar (Point 1 above) is optimized for **all components**.

Optimization from Dapr sidecar to the pub/sub broker would depend on a number of factors, for example:
- If the broker inherently supports Bulk pub/sub
- If the Dapr component is updated to support the use of bulk APIs provided by the broker.
Optimization from Dapr sidecar to the pub/sub broker depends on a number of factors, for example:
- Broker must inherently support Bulk pub/sub
- The Dapr component must be updated to support the use of bulk APIs provided by the broker

Currently, the following components are updated to support this level of optimization:
<table width="100%">
<tr>
<th>Component</th>
<th>Bulk Publish</th>
<th>Bulk Subscribe</th>
</tr>
<tr>
<td>Kafka</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>Azure Servicebus</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>Azure Eventhubs</td>
<td>Yes</td>
<td>Yes</td>
</tr>
</table>

| Component | Bulk Publish | Bulk Subscribe |
|:--------------------:|:--------:|--------|
| Kafka | Yes | Yes |
| Azure Servicebus | Yes | Yes |
| Azure Eventhubs | Yes | Yes |

## Demos

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ with DaprClient() as client:
result = client.publish_event(
pubsub_name='order_pub_sub',
topic_name='orders',
publish_metadata={'cloudevent.id: 'd99b228f-6c73-4e78-8c4d-3f80a043d317', cloudevent.source: 'payment'}
publish_metadata={'cloudevent.id': 'd99b228f-6c73-4e78-8c4d-3f80a043d317', 'cloudevent.source': 'payment'}
)
```

Expand Down Expand Up @@ -160,7 +160,7 @@ The JSON payload then reflects the new `source` and `id` values:
```

{{% alert title="Important" color="warning" %}}
While you can replace `traceid`/`traceparent` and `tracestate`, doing this may interfere with tracing events and report inconsistent results in tracing tools. It's recommended to use Open Telementry for distributed traces. [Learn more about distributed tracing.]({{< ref tracing-overview.md >}})
While you can replace `traceid`/`traceparent` and `tracestate`, doing this may interfere with tracing events and report inconsistent results in tracing tools. It's recommended to use Open Telemetry for distributed traces. [Learn more about distributed tracing.]({{< ref tracing-overview.md >}})

{{% /alert %}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ All Dapr pub/sub components support the at-least-once guarantee.

### Consumer groups and competing consumers pattern

Dapr handles the burden of dealing with consumer groups and the competing consumers pattern. In the competing consumers pattern, multiple application instances using a single consumer group compete for the message. Dapr enforces the competing consumer pattern when replicas use the same `app-id` without explict consumer group overrides.
Dapr handles the burden of dealing with consumer groups and the competing consumers pattern. In the competing consumers pattern, multiple application instances using a single consumer group compete for the message. Dapr enforces the competing consumer pattern when replicas use the same `app-id` without explicit consumer group overrides.

When multiple instances of the same application (with same `app-id`) subscribe to a topic, Dapr delivers each message to *only one instance of **that** application*. This concept is illustrated in the diagram below.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,4 +109,4 @@ spec:
Watch [this video for an overview of the outbox pattern](https://youtu.be/rTovKpG0rhY?t=1338):

<div class="embed-responsive embed-responsive-16by9">
<iframe width="360" height="315" src="https://youtu.be/rTovKpG0rhY?t=1338" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<iframe width="360" height="315" src="https://www.youtube-nocookie.com/embed/rTovKpG0rhY?si=1xlS54vcdYnLLtOL&amp;start=1338" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,6 @@ When you create an application with workflow code and run it with Dapr, you can

[Learn more about how manage a workflow using HTTP calls.]({{< ref workflow_api.md >}})

### Manage other workflow runtimes with workflow components

You can call other workflow runtimes (for example, Temporal and Netflix Conductor) by writing your own workflow component.

## Workflow patterns

Dapr Workflow simplifies complex, stateful coordination requirements in microservice architectures. The following sections describe several application patterns that can benefit from Dapr Workflow.
Expand Down Expand Up @@ -109,8 +105,9 @@ Want to skip the quickstarts? Not a problem. You can try out the workflow buildi

With Dapr Workflow in beta stage comes the following limitation(s):

- **State stores:** For the {{% dapr-latest-version cli="true" %}} beta release of Dapr Workflow, you're not able to use NoSQL databases. Only SQL databases are supported in the latest release.
- **Application instances:** For the {{% dapr-latest-version cli="true" %}} beta release of Dapr Workflow, only a maximum of 2 application instances is supported.
- **State stores:** For the {{% dapr-latest-version cli="true" %}} beta release of Dapr Workflow, using the NoSQL databases as a state store results in limitations around storing internal states. For example, CosmosDB has a maximum single operation item limit of only 100 states in a single request.

- **Horizontal scaling:** For the {{% dapr-latest-version cli="true" %}} beta release of Dapr Workflow, if you scale out Dapr sidecars or your application pods to more than 2, then the concurrency of the workflow execution drops. It is recommended to test with 1 or 2 instances, and no more than 2.

## Watch the demo

Expand Down
Loading

0 comments on commit 1195794

Please sign in to comment.