-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flow EVM Gateway FLIP #235
base: main
Are you sure you want to change the base?
Changes from all commits
a646e86
0751850
20045d2
c8a550a
bce3b32
61f7b20
65222ac
8c434ce
595b65b
0b916d9
c56e73e
d1c2d9f
948cf2c
0d74ad6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,311 @@ | ||
--- | ||
status: implemented | ||
flip: [243] | ||
authors: Ardit Marku ([email protected]), Gregor Gololicic ([email protected]) | ||
sponsor: Jerome Pimmel ([email protected]) | ||
updated: 2024-01-03 | ||
--- | ||
|
||
# FLIP 235: Flow EVM Gateway | ||
|
||
An implementation of the Ethereum JSON-RPC specification. | ||
|
||
## Objective | ||
|
||
This proposal outlines the design and implementation of the [Ethereum JSON-RPC specification](https://ethereum.github.io/execution-apis/api-documentation/) into a standalone | ||
service, which aims to provide a gateway to Flow EVM for developers and the | ||
various existing tools. | ||
|
||
## Motivation | ||
|
||
The [JSON-RPC API](https://ethereum.org/en/developers/docs/apis/json-rpc/) specifies a | ||
uniform set of methods that software applications, such as MetaMask or an Ethereum client, | ||
can use to interact with the Ethereum blockchain, either by reading blockchain data or | ||
sending transactions to the network. To achieve EVM compatibility, it is crucial to | ||
provide a JSON-RPC API, which among other things, facilitates the onboarding of developers | ||
that are already familiar with it and enables interoperability with the existing tools | ||
built on top of the JSON-RPC API. | ||
|
||
## User Benefit | ||
|
||
Users will be able to use an EVM-compatible client to connect to the Flow EVM network, | ||
by providing the network name, JSON-RPC URL, chain ID, currency etc. | ||
Developers will be able to easily retrieve all sorts of information from Flow EVM, | ||
such as block/transaction/account data, as well as define filters and subscribe | ||
to changes regarding the previous entities. This will allow developers to focus | ||
on building new tools and products, without having to write custom indexers from | ||
scratch. | ||
|
||
## Design Proposal | ||
|
||
### Context | ||
|
||
JSON-RPC is a stateless, light-weight remote procedure call (RPC) protocol. | ||
It defines several data structures and the rules around their processing. | ||
It is transport agnostic in that the concepts can be used within the same | ||
process, over sockets, over HTTP, or in many various message passing | ||
environments. It uses JSON (RFC 4627) as the data format. | ||
|
||
### Existing RPC Server implementations | ||
|
||
The Geth client offers a generic [RPC Server](https://github.com/ethereum/go-ethereum/blob/master/rpc/server.go) implementation, which will be used in the implementation | ||
of the Flow EVM Gateway as well. | ||
|
||
```go | ||
type BlockChainAPI struct { | ||
config *Config | ||
Store *storage.Store | ||
} | ||
|
||
func NewBlockChainAPI(config *Config, store *storage.Store) *BlockChainAPI { | ||
return &BlockChainAPI{ | ||
config: config, | ||
Store: store, | ||
} | ||
} | ||
|
||
// eth_chainId | ||
// Returns the chain ID used for signing replay-protected transactions. | ||
func (api *BlockChainAPI) ChainId() *hexutil.Big { | ||
return (*hexutil.Big)(api.config.ChainID) | ||
} | ||
|
||
// eth_blockNumber | ||
// BlockNumber returns the block number of the chain head. | ||
func (api *BlockChainAPI) BlockNumber() hexutil.Uint64 { | ||
latestBlockHeight, err := api.Store.LatestBlockHeight(context.Background()) | ||
if err != nil { | ||
panic(fmt.Errorf("failed to fetch the latest block number: %v", err)) | ||
} | ||
return hexutil.Uint64(latestBlockHeight) | ||
} | ||
|
||
// Create an RPC server. | ||
srv := rpc.NewServer() | ||
|
||
apis := []rpc.API{ | ||
{ | ||
Namespace: "eth", | ||
Service: NewBlockChainAPI(config, store), | ||
}, | ||
} | ||
|
||
// Register all the APIs exposed by the services | ||
for _, api := range apis { | ||
if err := srv.RegisterName(api.Namespace, api.Service); err != nil { | ||
return err | ||
} | ||
} | ||
``` | ||
|
||
The above code snippet instantiates a new RPC server and registers a | ||
service object (`*BlockChainAPI`) under the `eth` namespace. | ||
|
||
The server can now accept HTTP requests with the following body: | ||
|
||
```json | ||
{"jsonrpc":"2.0","id":1,"method":"eth_chainId","params": []} | ||
``` | ||
|
||
and respond with an appropriate result, such as: | ||
|
||
```json | ||
{"jsonrpc":"2.0","id":1,"result":"0x29a"} | ||
``` | ||
|
||
Under the hood, the RPC server reads the value of the `"method"` key. | ||
In this case `"eth_chainId"`, the part before the `_` is the namespace | ||
(`eth`) and the part after the `_` is the method which should be called | ||
on the registered service object (`ChainId`). | ||
|
||
Some sample HTTP calls to the Flow EVM Gateway's JSON-RPC server, can | ||
be seen below: | ||
|
||
![Flow EVM Gateway JSON-RPC over HTTP](./2024-01-03-evm-gateway-resources/flow-evm-gateway-curl-http.png) | ||
|
||
### Supported transport protocols | ||
|
||
There are three transport protocols available in Geth: IPC, HTTP and WebSockets. | ||
|
||
For the gateway, we are interested in supporting only JSON-RPC over HTTP and | ||
JSON-RPC over WebSockets. The RPC methods with event subscription are available | ||
only for WebSocket connections. | ||
|
||
Again, the Geth client has an implementation of an [httpServer](https://github.com/ethereum/go-ethereum/blob/master/node/rpcstack.go) which is basically a proxy that routes `HTTP` & | ||
`WebSocket` requests to a dedicated `rpc.Server` instance for each of the two | ||
transport mediums. This allows for more fine-grained control over which namespaces | ||
& RPC methods are available on which transport, and allows for better scaling. | ||
|
||
Unfortunately, none of the relevant types in the `go-ethereum` package are exported, | ||
so we cannot pull them directly from the `go-ethereum` package. But even if they | ||
were exported, it comes with quite a lot of extras, such as `CORS` / `JWT` / `GZIP` | ||
compression which might be overkill for what we currently need. For the time being, | ||
we have extracted a minified version of it [here](https://github.com/onflow/flow-evm-gateway/blob/aaf3ba3f2ab78321fa377ecf5165a2f7dbc44865/api/server.go). | ||
|
||
### Indexing FlowEVM events | ||
|
||
Quoting some content from [EVM support](https://github.com/onflow/flips/pull/225) FLIP: | ||
|
||
> To better understand the approach proposed in this Flip, consider EVM on Flow as a virtual blockchain deployed to the Flow blockchain at a specific address (e.g., a service account). EVM on Flow functions as a smart contract that emulates the EVM with its own dedicated chain-ID. Signed transactions are inputted, and a chain of blocks is produced as output. Similar to other built-in standard contracts (e.g., RLP encoding), this EVM environment can be imported into any Flow transaction or script. | ||
This is made possible using selective integration of the core EVM runtime without the supporting software stack in which it currently exists on Ethereum. For equivalence we also provide an EVM compatible JSON-RPC API implementation to facilitate EVM on Flow interactions from existing EVM clients. | ||
|
||
Since the gateway is not the only entrypoint for interacting with Flow EVM, we need | ||
to track every event emitted from the `EVM` smart contract, consume it and build an | ||
index which reflects the latest state of the chain. | ||
|
||
For the indexing, we will be using the Event Streaming API offered from the | ||
`flow-go-sdk`. A sample code snippet can be seen below: | ||
|
||
```go | ||
flowClient, err := grpc.NewBaseClient( | ||
grpc.EmulatorHost, | ||
goGrpc.WithTransportCredentials(insecure.NewCredentials()), | ||
) | ||
if err != nil { | ||
logger.Error().Msgf("could not create flow client: %v", err) | ||
} | ||
|
||
data, errChan, initErr := flowClient.SubscribeEventsByBlockHeight( | ||
ctx, | ||
latestBlockHeader.Height, | ||
flow.EventFilter{ | ||
EventTypes: []string{ | ||
"flow.evm.BlockExecuted", | ||
"flow.evm.TransactionExecuted", | ||
}, | ||
}, | ||
grpc.WithHeartbeatInterval(1), | ||
) | ||
|
||
// track the most recently seen block height. we will use this when reconnecting. | ||
// the first response should be for latestBlockHeader.Height | ||
lastHeight := latestBlockHeader.Height - 1 | ||
for { | ||
select { | ||
case <-ctx.Done(): | ||
return | ||
|
||
case response, ok := <-data: | ||
if !ok { | ||
if ctx.Err() != nil { | ||
return // graceful shutdown | ||
} | ||
logger.Error().Msg("subscription closed - reconnecting") | ||
connect(lastHeight+1, 10) | ||
continue | ||
} | ||
|
||
if response.Height != lastHeight+1 { | ||
logger.Error().Msgf("missed events response for block %d", lastHeight+1) | ||
connect(lastHeight, 10) | ||
continue | ||
} | ||
|
||
logger.Info().Msgf("block %d %s:", response.Height, response.BlockID) | ||
if len(response.Events) > 0 { | ||
store.StoreBlockHeight(ctx, response.Height) | ||
} | ||
for _, event := range response.Events { | ||
logger.Info().Msgf(" %s", event.Value) | ||
} | ||
|
||
lastHeight = response.Height | ||
|
||
case err, ok := <-errChan: | ||
... | ||
} | ||
} | ||
``` | ||
|
||
The main events from the `EVM` smart contract are two: | ||
- `flow.evm.BlockExecuted`, | ||
- `flow.evm.TransactionExecuted` | ||
|
||
To see the indexer & the events' payloads in action: | ||
|
||
![Flow EVM Events](./2024-01-03-evm-gateway-resources/flow-evm-events.png) | ||
|
||
### Storage Integrity | ||
|
||
While consuming events, it is very critical to make sure that they are processed | ||
in a sequential manner, and no events response is missed for a single block. | ||
If this were to happen, the storage index could end up corrupted, and we would | ||
return incorrect data for things such as account balances, nonces etc. | ||
|
||
In the code snippet below, we specify a heartbeat interval of `1` and check | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would say this would be good to change in the future as a performance improvement, ideally we would get multiple events and still preserve the order and integrity, however this currently might not be possible yet due to API not providing the heights, but it might be worth linking to an issue and keeping a promise to improve it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just to clarify, even with a heartbeat interval of
|
||
that each received events response is for the expected block height, before | ||
committing any changes to the storage. | ||
|
||
```go | ||
data, errChan, initErr := flowClient.SubscribeEventsByBlockHeight( | ||
..., | ||
grpc.WithHeartbeatInterval(1), | ||
// this will yield an events response for each block, | ||
// even if it has no events matching the filter criteria. | ||
) | ||
|
||
// track the most recently seen block height. we will use this when reconnecting. | ||
// the first response should be for latestBlockHeader.Height | ||
lastHeight := latestBlockHeader.Height - 1 | ||
for { | ||
select { | ||
case response, ok := <-data: | ||
... | ||
if response.Height != lastHeight+1 { | ||
logger.Error().Msgf("missed events response for block %d", lastHeight+1) | ||
connect(lastHeight, 10) | ||
continue | ||
} | ||
// make sure that the response is for the expected block height, | ||
// before committing any changes to the storage. | ||
... | ||
|
||
// mark the block height of the events response as processed. | ||
lastHeight = response.Height | ||
} | ||
} | ||
``` | ||
|
||
### Performance Implications | ||
|
||
We should have some end-to-end tests, perhaps with the usage of the Flow Emulator. | ||
As well as some benchmarks, to assess the service degradation with regards to the | ||
increase of the storage, the incoming requests and possibly the emitted events. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would also create stress tests that could be run as part of the Flow benchnet, with good metrics on the gateway side we could measure performance. The metrics should also be used in production for monitoring. It would be also worth exploring tracing with OTEL. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Very good suggestion 👏 |
||
### Dependencies | ||
|
||
The Flow EVM Gateway is a standalone service, hosted in its own repository, so | ||
it does not add any dependencies to existing core repositories. It is | ||
build around some of Flow's core repositories, such as `flow-go-sdk` and | ||
`flow-go`, so it depends on their public APIs. | ||
Later on, it would be useful to expose the Flow EVM Gateway from the Emulator | ||
as well, as we have done with the gRPC & REST Access API. | ||
|
||
### Tutorials and Examples | ||
|
||
A reference implementation addressing parts of this proposal can be found in | ||
this [PR](https://github.com/onflow/flow-evm-gateway/pull/11#issue-2006802251). | ||
|
||
### User Impact | ||
|
||
The Flow EVM Gateway is a standalone service that complements the Flow EVM | ||
functionality. It should be released at around the same time as Flow EVM, | ||
in order to allow for better beta testing. | ||
|
||
## Related Issues | ||
|
||
As with any service, we need to think about logging, alerting, monitoring, | ||
storage capacity and throughput. | ||
Dockerizing the gateway would be nice, in order to allow developers to easily | ||
start using it for development and testing. | ||
|
||
## Prior Art | ||
|
||
As explained above, the [Geth](https://github.com/ethereum/go-ethereum) client | ||
was the main source of inspiration for the work on the JSON-RPC server. | ||
|
||
## Questions and Discussion | ||
|
||
- Do we expect to have one official EVM Gateway run by the Flow team and | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We expect the community to maintain the EVM Gateways the same way as currently are for AN. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sounds good, I guess I should introduce more flexible config options then. |
||
individual developers/team can also run their own? As is the case with | ||
Access Nodes, for example. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The integrity mechanism outlined below is a good start though we may need to build in more logic to recover missed events. While the goal is to not miss events we should plan for a reality where it will eventually happen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, certainly. Back-filling of past events is something we should account for.