-
Notifications
You must be signed in to change notification settings - Fork 3
Adding docs for event retries in snap-ins #82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 17 commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
d9ed601
Init docs for snap-in configuration
codeon 2fe2c41
init docs and fixes
codeon bfeb8c9
rewrite
codeon b29924a
review comments
codeon 4b95597
update configuration
codeon 6a78385
Add code blocks to retry mechanism
codeon 5112a85
Apply suggestions from code review
codeon 7a9500c
code review suggestions
codeon bb0c1f6
Fix as per comments
codeon 0ba0897
add pages to sidebar nav
codeon 8063665
Merge remote-tracking branch 'origin/main' into ISS-98485-ISS-100260-…
codeon f2982dc
Review comments
codeon 293238b
Remove docs for retries for now as we finalize approach
codeon da64d4a
remove snap-components link
codeon 3dfcb00
Merge remote-tracking branch 'origin/main' into ISS-98485-ISS-100260-…
codeon f7d7e51
Revert "Remove docs for retries for now as we finalize approach"
codeon b7ba56d
fix path
codeon 70e896d
Update fern/docs/pages/retry-mechanism.mdx
shashankcube 50f8bfa
Update fern/docs/pages/retry-mechanism.mdx
shashankcube e055cff
Update fern/docs/pages/retry-mechanism.mdx
shashankcube File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
# Event reliability in DevRev snap-ins | ||
|
||
The DevRev snap-ins platform offers event reliability features to ensure smooth and resilient event processing. This document provides an overview of these features and how developers can leverage them to build reliable snap-ins. | ||
|
||
## Getting started | ||
|
||
To start using the event reliability features in your Snap-ins, follow these steps: | ||
|
||
1. Update your DevRev SDK to the latest version. | ||
2. Define retryable errors using the `FunctionExecutionError` interface in your Snap-in code. | ||
3. Configure the retry behavior in your Snap-in manifest. | ||
4. Handle errors appropriately in your Snap-in function. | ||
shashankcube marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Retryable errors | ||
|
||
Snap-ins can define retryable errors using the `FunctionExecutionError` interface provided by the DevRev SDK. This enables the platform to automatically retry events that encounter intermittent or transient errors, improving overall reliability. | ||
|
||
## Retry configuration | ||
|
||
Developers can configure the retry behavior of their Snap-in functions using the Snap-in manifest. The following options are available: | ||
shashankcube marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
```yaml | ||
functions: | ||
- name: function_name | ||
config: | ||
runtime: | ||
max_retries: 5 # number of retries before failing the event | ||
min_interval: 120 # interval in seconds between each retry | ||
``` | ||
|
||
- `max_retries`: The maximum number of retries before marking the event as failed. | ||
- `min_interval`: The minimum interval in seconds between each retry attempt. This interval may be adjusted based on the timeout of the corresponding function. | ||
|
||
## Error handling | ||
|
||
The DevRev snap-in platform handles errors based on the ordering guarantees of the snap-in function. | ||
|
||
For snap-in functions with relaxed ordering, non-retryable errors are marked as failed, and the errored event is propagated to the DevRev platform for tracking. Retryable errors are automatically retried based on the specified retry configuration. If the maximum number of retries is exhausted, the errored events are moved to a dead-letter queue (DLQ) for further handling. | ||
|
||
## Error interface | ||
|
||
The DevRev SDK defines the `FunctionExecutionError` type to represent errors returned from the snap-in's run function. Developers can use this type to provide additional error details and indicate whether an error is retryable. | ||
|
||
```typescript | ||
class FunctionExecutionError extends Error { | ||
/** | ||
* Toggle to determine if the event should be retried or not. If not set or set to false, | ||
* the event is not retried. | ||
*/ | ||
retry: boolean; | ||
|
||
/** | ||
* Whether to retry the event payload with updated metadata | ||
* that platform provides. Useful when the event payload is | ||
* not in a state to be directly processed, and may need new | ||
* keyrings/service account tokens or new inputs. | ||
*/ | ||
refresh?: boolean; | ||
|
||
constructor(message: string, retry: boolean, refresh?: boolean) { | ||
super(message); | ||
this.retry = retry; | ||
this.refresh = refresh; | ||
} | ||
} | ||
``` | ||
|
||
## Example usage | ||
|
||
Here's an example of how to use the `FunctionExecutionError` in your Snap-in code: | ||
|
||
```typescript | ||
import { FunctionExecutionError } from '@devrev/typescript-sdk/dist/snap-ins/types'; | ||
|
||
export const run = async (events: any[]) => { | ||
/* | ||
Put your code here to handle the event. | ||
*/ | ||
console.log('Events: ', JSON.stringify(events)); | ||
|
||
try { | ||
// Your event processing logic here | ||
// ... | ||
|
||
// Retryable error | ||
console.log('Retrying....'); | ||
const runtimeError = new FunctionExecutionError('Runtime Retryable Error', true, false); | ||
throw runtimeError; | ||
|
||
// Non-retryable error | ||
// const runtimeError = new FunctionExecutionError("Runtime Non-Retryable Error", false, false); | ||
// throw runtimeError; | ||
|
||
} catch (error) { | ||
if (error instanceof FunctionExecutionError) { | ||
throw error; | ||
} else { | ||
// Handle other types of errors | ||
// ... | ||
} | ||
} | ||
}; | ||
|
||
export default run; | ||
``` | ||
|
||
In this example, the Snap-in function's `run` method processes the events and can throw a `FunctionExecutionError` to indicate whether the error is retryable or not. The Snap-in platform handles the error based on the `retry` and `refresh` properties set in the error object. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.