Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Describe using pindexer #32

Merged
merged 2 commits into from
Sep 24, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions pages/node/pd/indexing-events.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
The `pd` software emits ABCI events while processing blocks. By default,
these blocks are stored in CometBFT's key-value database locally, but node operators
can opt-in to writing the events to an external PostgreSQL database.
Furthermore, the [`pindexer`](#using-pindexer) software can be used to take these raw ABCI events,
and produce penumbra-specific "app views" with rich formatted data.

## Configuring a Penumbra node to write events to postgres

Expand All @@ -18,6 +20,55 @@ The format for `DATABASE_URL` is specified in the [Postgres docs](https://www.po
After the node is running, check the logs for errors. Query the database with `SELECT height FROM blocks ORDER BY height DESC LIMIT 10;` and confirm
you're seeing the latest blocks being added to the database.

## Using Pindexer

`pindexer` reads from a Postgres ABCI event database, as described in the section above,
and produces a derived database with rich tables for Penumbra-specific views.
This is useful because the raw events database isn't easily or efficiently queryable
for Penumbra-specific use cases, such as seeing the current status of the DEX,
or the total amount of fees paid so far on the life of the chain.

To run `pindexer`, you'll need to have configured the Penumbra node to index events into Postgres. Then:

1. Create a Postgres database for the output of `pindexer`, say `$PINDEXER_DB`.
2. Run `pindexer --chain-id <CHAIN_ID> --genesis-json <ORIGINAL_GENESIS_FILE> --src-database-url postgresql://localhost:5432/$DATABASE_NAME --dst-database-url postgresql://localhost:5432/$PINDEXER_DB`

This assumes that Postgres is running locally on port 5432; for another setup the URL should change.

The `<ORIGINAL_GENESIS_FILE>` must be the `genesis.json` file for the chain, **before any upgrades.**
Note, in particular, that after an upgrade, there will be a new genesis file containing only a checkpoint.
`pindexer` specifically needs the original genesis file, because it needs to read information
about the start of the chain, such as initial allocations, to track changes over time accurately.

### Example: Total Supply Indexing

For example, after running `pindexer`, here's a query to get the total amount of the native
staking token up to the current height:

```sql
SELECT (staked_um + unstaked_um + auction + dex)::NUMERIC / 10^6 as total
FROM (
SELECT SUM(um) as staked_um
FROM (
SELECT *
FROM supply_validators
) validators
LEFT JOIN LATERAL (
SELECT um
FROM supply_total_staked
WHERE validator_id = id
ORDER BY height DESC
LIMIT 1
) ON TRUE
) staked
LEFT JOIN LATERAL (
SELECT um as unstaked_um, auction, dex
FROM supply_total_unstaked
ORDER BY height DESC
LIMIT 1
) on TRUE
```

## Running an archive node

If you are joining the network after a chain upgrade, the events behind the upgrade boundary
Expand Down