Skip to content
This repository has been archived by the owner on May 29, 2024. It is now read-only.

Update core logic to be less verbose & more simple in abstraction #182

Merged
merged 13 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ To use the template, run the following command(s):
docker run -p 8080:8080 -p 7300:7300 --env-file=config.env -it -v ${PWD}/genesis.json:/app/genesis.json ghcr.io/base-org/pessimism:latest
```

**Note**: If you want to bootstrap the application and run specific heuristics/pipelines upon start, update config.env `BOOTSTRAP_PATH` value to the location of your genesis.json file then run
**Note**: If you want to bootstrap the application and run specific heuristics/paths upon start, update config.env `BOOTSTRAP_PATH` value to the location of your genesis.json file then run

### Building and Running New Images

Expand Down Expand Up @@ -143,7 +143,6 @@ A bootstrap config file is used to define the initial state of the pessimism ser
[
{
"network": "layer1",
"pipeline_type": "live",
"type": "contract_event",
"start_height": null,
"alerting_params": {
Expand All @@ -157,7 +156,6 @@ A bootstrap config file is used to define the initial state of the pessimism ser
},
{
"network": "layer1",
"pipeline_type": "live",
"type": "balance_enforcement",
"start_height": null,
"alerting_params": {
Expand Down
2 changes: 1 addition & 1 deletion cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ func RunPessimism(_ *cli.Context) error {
return err
}

logger.Info("Received bootstrapped session UUIDs", zap.Any(logging.SUUIDKey, ids))
logger.Info("Received bootstrapped session UUIDs", zap.Any(logging.Session, ids))

logger.Debug("Application state successfully bootstrapped")
}
Expand Down
4 changes: 2 additions & 2 deletions config.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
L1_RPC_ENDPOINT=
L2_RPC_ENDPOINT=

# Oracle Geth Block Poll Intervals (ms)
# Chain
L1_POLL_INTERVAL=5000
L2_POLL_INTERVAL=5000

Expand Down Expand Up @@ -39,4 +39,4 @@ ENABLE_METRICS=1 # 0 to disable, 1 to enable
METRICS_READ_HEADER_TIMEOUT=60

# Concurrency Management
MAX_PIPELINE_COUNT=10
MAX_PATH_COUNT=10
6 changes: 3 additions & 3 deletions docs/Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ GEM
octokit (~> 4.0)
public_suffix (>= 3.0, < 5.0)
typhoeus (~> 1.3)
html-pipeline (2.14.3)
html-path (2.14.3)
activesupport (>= 2)
nokogiri (>= 1.4)
http_parser.rb (0.8.0)
Expand Down Expand Up @@ -125,7 +125,7 @@ GEM
jekyll-include-cache (0.2.1)
jekyll (>= 3.7, < 5.0)
jekyll-mentions (1.6.0)
html-pipeline (~> 2.3)
html-path (~> 2.3)
jekyll (>= 3.7, < 5.0)
jekyll-optional-front-matter (0.3.2)
jekyll (>= 3.0, < 5.0)
Expand Down Expand Up @@ -194,7 +194,7 @@ GEM
listen (~> 3.0)
jemoji (0.12.0)
gemoji (~> 3.0)
html-pipeline (~> 2.2)
html-path (~> 2.2)
jekyll (>= 3.0, < 5.0)
kramdown (2.3.2)
rexml
Expand Down
7 changes: 3 additions & 4 deletions docs/architecture/alerting.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ end

subgraph AM["Alerting Manager"]
alertingRelay --> |Alert|EL
EL[eventLoop] --> |Alert SUUID|AS["AlertStore"]
EL[eventLoop] --> |Alert UUID|AS["AlertStore"]
AS --> |Alert Policy|EL
EL --> |Submit alert|SR["SeverityRouter"]
SR --> SH["Slack"]
Expand All @@ -48,7 +48,7 @@ An `Alert` type stores all necessary metadata for external consumption by a down
The alert store is a persistent storage layer that is used to store alerting entries. As of now, the alert store only supports configurable alert destinations for each alerting entry. Ie:

```
(SUUID) --> (AlertDestination)
(UUID) --> (AlertDestination)
```

### Alert Destinations
Expand Down Expand Up @@ -77,7 +77,7 @@ Done! You should now see any generated alerts being forwarded to your specified

#### PagerDuty

The PagerDuty alert destination is a configurable destination that allows alerts to be sent to a specific PagerDuty services via the use of integration keys. Pessimism also uses the SUUID associated with an alert as a deduplication key for PagerDuty. This is done to ensure that PagerDuty will not be spammed with duplicate or incidents.
The PagerDuty alert destination is a configurable destination that allows alerts to be sent to a specific PagerDuty services via the use of integration keys. Pessimism also uses the UUID associated with an alert as a deduplication key for PagerDuty. This is done to ensure that PagerDuty will not be spammed with duplicate or incidents.

### Alert CoolDowns

Expand All @@ -88,7 +88,6 @@ An example of this is shown below:
```json
{
"network": "layer1",
"pipeline_type": "live",
"type": "balance_enforcement",
"start_height": null,
"alerting_params": {
Expand Down
6 changes: 3 additions & 3 deletions docs/architecture/api.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,17 @@ Currently, interactive endpoint documentation is hosted via [Swagger UI](https:/

### Configuration

The API can be customly configured using environment variables stored in a `config.env` file. The following environment variables are used to configure the API:
The API can be configured using environment variables stored in a `config.env` file. The following environment variables are used to configure the API:

- `SERVER_HOST`: The host address to serve the API on (eg. `localhost`)
- `SERVER_PORT`: The port to serve the API on (eg. `8080`)
- `SERVER_KEEP_ALIVE`: The keep alive second duration for the server (eg. `10`)
- `SERVER_READ_TIMEOUT`: The read timeout second duration for the server (eg. `10`)
- `SERVER_WRITE_TIMEOUT`: The write timeout second duration for the server (eg. `10`)

### Components
### Processes

The Pessimism API is broken down into the following constituent components:
The Pessimism API is broken down into the following constituent processes:

- `handlers`: The handlers package contains the HTTP handlers for the API. Each handler is responsible for handling a specific endpoint and is responsible for parsing the request, calling the appropriate service method, and renders a response.
- `service`: The service package contains the business logic for the API. The service is responsible for handling calls to the core Pessimism subsystems and is responsible for validating incoming requests.
Expand Down
6 changes: 3 additions & 3 deletions docs/architecture/architecture.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ permalink: /architecture

There are *three subsystems* that drive Pessimism’s architecture:

1. [ETL](./etl.markdown) - Modularized data extraction system for retrieving and processing external chain data in the form of a DAG known as the Pipeline DAG
2. [Risk Engine](./engine.markdown) - Logical execution platform that runs a set of heuristics on the data funneled from the Pipeline DAG
1. [ETL](./etl.markdown) - Modularized data extraction system for retrieving and processing external chain data in the form of a DAG known as the Path DAG
2. [Risk Engine](./engine.markdown) - Logical execution platform that runs a set of heuristics on the data funneled from the Path DAG
3. [Alerting](./alerting.markdown) - Alerting system that is used to notify users of heuristic failures

These systems will be accessible by a client through the use of a JSON-RPC API that has unilateral access to all three primary subsystems.
Expand All @@ -23,7 +23,7 @@ The API will be supported to allow Pessimism users via client to:
## Diagram

The following diagram illustrates the core interaction flow between the three primary subsystems, API, and external data sources:
![high level component diagram](../assets/images/high_level_diagram.png)
![high level process diagram](../assets/images/high_level_diagram.png)

## Shared State

Expand Down
31 changes: 8 additions & 23 deletions docs/architecture/engine.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ graph LR;
subgraph A["Engine Manager"]
C(eventLoop) -.-> C;
C --> |input|D("execute()")
D --> |input.PUUID|E[SessionStore]
D --> |input.PathID|E[SessionStore]
E --> |"[]sessions"|D
D --> |"[]sessions"|F[Risk Engine]
F --> |"[]outcome"|D
Expand All @@ -45,28 +45,13 @@ The ETL publishes `Heuristic Input` to the Risk Engine using a relay channel. Th

## Heuristic Session

A heuristic session refers to the execution and representation of a single heuristic. A heuristic session is uniquely identified by a `SUUID` and is associated with a single `PUUID`. A heuristic session is created by the `EngineManager` when a user requests to run an active session. The `EngineManager` will create a new `HeuristicSession` and pass it to the `RiskEngine` to be executed. The `RiskEngine` will then execute the heuristic session and return an `InvalidationOutcome` to the `EngineManager`. The `EngineManager` will then create an `Alert` using the `InvalidationOutcome` and publish it to the Alerting system.

## Session UUID (SUUID)

The SUUID is a unique identifier that is used to identify a specific heuristic session. The SUUID is generated by the `EngineManager` when a user requests to run a new heuristic session. The SUUID is used to uniquely identify a specific heuristic session. This allows the `EngineManager` to perform operations on a specific heuristic session such as removing it or updating it.

A `SUUID` constitutes of both a unique `UUID` and a `PID`.

A `SessionPID` is encoded using the following 3 byte array sequence:

```
0 1 2 3
|-----------|-----------|-----------|
network pipeline heuristic
type type type
```
A heuristic session refers to the execution and representation of a single heuristic. A heuristic session is uniquely identified by a `UUID` and is associated with a single `PathID`. A heuristic session is created by the `EngineManager` when a user requests to run an active session. The `EngineManager` will create a new `HeuristicSession` and pass it to the `RiskEngine` to be executed. The `RiskEngine` will then execute the heuristic session and return an `InvalidationOutcome` to the `EngineManager`. The `EngineManager` will then create an `Alert` using the `InvalidationOutcome` and publish it to the Alerting system.

## Heuristic Input

The heuristic input is a struct that contains the following fields:

* `PUUID` - The ID of the heuristic that the input data is intended for
* `PathID` - The ID of the heuristic that the input data is intended for
* `Input` - Transit data that was generated by the ETL

## Heuristic
Expand All @@ -80,17 +65,17 @@ Every hardcoded heuristic must implement an `Heuristic` interface to be compatib
```
type Heuristic interface {
Addressing() bool
InputType() core.RegisterType
Invalidate(core.TransitData) (*core.InvalOutcome, bool, error)
SUUID() core.SUUID
SetSUUID(core.SUUID)
InputType() core.TopicType
Invalidate(core.Event) (*core.InvalOutcome, bool, error)
UUID() core.UUID
SetUUID(core.UUID)
}

```

### Heuristic Input Type

The heuristic input type is a `RegisterType` that defines the type of data that the heuristic will receive as input. The heuristic input type is defined by the `InputType()` method of the `Heuristic` interface. The heuristic input type is used by the `RiskEngine` to determine if the input data is compatible with the heuristic. If the input data is not compatible with the heuristic, the `RiskEngine` will not execute the heuristic and will return an error.
The heuristic input type is a `TopicType` that defines the type of data that the heuristic will receive as input. The heuristic input type is defined by the `InputType()` method of the `Heuristic` interface. The heuristic input type is used by the `RiskEngine` to determine if the input data is compatible with the heuristic. If the input data is not compatible with the heuristic, the `RiskEngine` will not execute the heuristic and will return an error.

### Addressing

Expand Down
Loading
Loading