Skip to content

Commit

Permalink
chore: add docs for retry (numaproj#2024)
Browse files Browse the repository at this point in the history
Signed-off-by: Sidhant Kohli <[email protected]>
  • Loading branch information
kohlisid authored Sep 4, 2024
1 parent 8d8b9e2 commit 8fc99bb
Show file tree
Hide file tree
Showing 2 changed files with 69 additions and 0 deletions.
68 changes: 68 additions & 0 deletions docs/user-guide/sinks/retry-strategy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Retry Strategy

### Overview
The `RetryStrategy` is used to configure the behavior for a sink after encountering failures during a write operation.
This structure allows the user to specify how Numaflow should respond to different fail-over scenarios for Sinks, ensuring that the writing can be resilient and handle
unexpected issues efficiently.


### Struct Explanation


`retryStrategy` is optional, and can be added to the Sink spec configurations where retry logic is necessary.



```yaml
sink:
retryStrategy:
# Optional
backoff:
duration: 1s # Optional
steps: 3 # Optional, number of retries (including the 1st try)
# Optional
onFailure: retry|fallback|drop
```
Note: If no custom fields are defined for retryStrategy then the **default** values are used.
- `BackOff` - Defines the timing for retries, including the interval and the maximum attempts.
- `duration`: the time interval to wait before retry attempts
- Default: _1ms_
- `steps`: the limit on the number of times to try the sink write operation including retries
- Default: _Infinite_
- `OnFailure` - Specifies the action to be undertaken if number of retries are exhausted
- retry: continue with the retry logic again
- fallback: write the leftover messages to a [fallback](https://numaflow.numaproj.io/user-guide/sinks/fallback/) sink
- drop: any messages left to be processed are dropped
- Default: _retry_


### Constraints

1) If the `onFailure` is defined as fallback, then there should be a fallback sink specified in the spec.

2) The steps defined should always be `> 0`


## Example

```yaml
sink:
retryStrategy:
backoff:
interval: "500ms"
steps: 10
onFailure: "fallback"
udsink:
container:
image: my-sink-image
fallback:
udsink:
container:
image: my-fallback-sink
```
### Explanation

- Normal Operation: Data is processed by the primary sink container specified by `UDSink`.
The system retries up to 10 times for a batch write operation to succeed with an interval of 500 milliseconds between each retry.
- After Maximum Retries: If all retries fail, data is then routed to a fallback sink instead.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ nav:
- user-guide/sinks/blackhole.md
- User-defined Sinks: "user-guide/sinks/user-defined-sinks.md"
- Fallback Sink: "user-guide/sinks/fallback.md"
- Retry Strategy: "user-guide/sinks/retry-strategy.md"
- User-defined Functions:
- Overview: "user-guide/user-defined-functions/user-defined-functions.md"
- Map:
Expand Down

0 comments on commit 8fc99bb

Please sign in to comment.