Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IA.v3] Update miscellaneous out-of-date stuff #2012

Merged
merged 6 commits into from
Jul 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ In its default configuration, the `rippled` server automatically deletes outdate

## Warnings

Storing full history is expensive. As of 2020-11-10, the full history of the XRP Ledger occupies approximately **14 terabytes** of disk space, which must be entirely stored on fast solid state disk drives for proper server performance. Such a large amount of solid state storage is not cheap, and the total amount of history you must store increases by approximately 12 GB per day.
Storing full history is expensive. As of 2023-07-19, the full history of the XRP Ledger occupies approximately **26 terabytes** of disk space, which must be entirely stored on fast solid state disk drives for proper server performance. Such a large amount of solid state storage is not cheap, and the total amount of history you must store increases by approximately 12 GB per day.

Additionally, storing full history in NuDB requires single files that are larger than the 16 TB limit of ext4 filesystems, which is the default on many Linux distributions. You must use a filesystem with a larger single-file limit, such as XFS (recommended) or ZFS.

Acquiring full history from the peer-to-peer network takes a long time (several months) and requires that your server has enough system and network resources to acquire older history while keeping up with new ledger progress. To get a faster start on acquiring ledger history, you may want to find another server operator who has a large amount of history already downloaded, who can give you a database dump or at least allow your server to explicitly peer with theirs for a long time to acquire history. The server can load ledger history from a file and verify the integrity of the historical ledgers it imports.

Expand Down
46 changes: 21 additions & 25 deletions content/infrastructure/rippled/installation/capacity-planning.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,23 +50,19 @@ If you set the `[node_size]` parameter to an invalid value, the [server fails to

The `type` field in the `[node_db]` stanza of the `rippled.cfg` file sets the type of key-value store that `rippled` uses to hold the ledger store.

This setting does not directly configure RAM settings, but the choice of key-value store has important implications for RAM usage because of the different ways these technologies cache and index data for fast lookup.
For almost all purposes, use `NuDB`. A fast SSD is required. [Learn more](#more-about-using-nudb)

- For most cases, use `NuDB` because its performance is constant even with large amounts of data on disk. A fast SSD is required. [Learn more](#more-about-using-nudb)

- If you are using rotational disks (not recommended) or an unusually slow SSD, use `RocksDB`. You should avoid this setting for production servers. [Learn more](#more-about-using-rocksdb)
The `RocksDB` setting is available for legacy purposes, but is generally not recommended. [Learn more](#more-about-using-rocksdb)

The example `rippled-example.cfg` file has the `type` field in the `[node_db]` stanza set to `NuDB`.

#### More About Using RocksDB

[RocksDB](https://rocksdb.org/docs/getting-started.html) is an persistent key-value store built into `rippled`.

**Caution:** As of late 2021, the total size of the ledger has grown large enough that servers using RocksDB often struggle to maintain sync with the Mainnet. Large amounts of RAM can help, but you should generally use NuDB instead.
[RocksDB](https://rocksdb.org/docs/getting-started.html) is a persistent key-value store built into `rippled`. **Support for RocksDB is considered legacy.** Servers using RocksDB usually struggle to maintain sync with the Mainnet due to the memory requirements of maintaining a large database. Generally, you should use NuDB instead.

RocksDB is intended to work on either solid-state disks or rotational disks. It requires approximately one-third less [disk storage](#disk-space) than NuDB and provides better I/O latency. However, the better I/O latency comes as result of the large amount of RAM RocksDB requires to store data indexes.
Cases where you might use RocksDB include if you need to load historical data saved in RocksDB format, or if you are storing data on slow SSDs or rotational disks. While rotational disks won't be able to keep up with Mainnet, you can probably run offline tests or small private networks on them.
mDuo13 marked this conversation as resolved.
Show resolved Hide resolved

RocksDB has performance-related configuration options that you can tweak for more transaction processing throughput. Here is a recommended `[node_db]` configuration for RocksDB:
RocksDB has performance-related configuration options that you can tweak for more transaction processing throughput. Here is an example `[node_db]` configuration for RocksDB:

```
[node_db]
Expand All @@ -83,13 +79,13 @@ advisory_delete=0

(Adjust the `path` to the directory where you want to keep the ledger store on disk. Adjust the `online_delete` and `advisory_delete` settings as desired for your configuration.)

#### More About Using NuDb
#### More About Using NuDB

[NuDB](https://github.com/vinniefalco/nudb#introduction) is an append-only key-value store that is optimized for SSD drives.

NuDB has nearly constant performance and memory footprints regardless of the [amount of data being stored](#disk-space). NuDB _requires_ a solid-state drive, but uses much less RAM than RocksDB to access a large database.
NuDB has nearly constant performance and memory footprints regardless of the [amount of data being stored](#disk-space). NuDB _requires_ a solid-state drive. Scalability testing has shown that NuDB has equivalent or better performance than RocksDB in production and comparable configurations.

Production servers should be configured to use NuDB and to store the amount of historical data required for the use case.
Production servers should be configured to use NuDB and to store the amount of historical data required for your use case.

NuDB does not have performance-related configuration options available in `rippled.cfg`. Here is the recommended `[node_db]` configuration for a `rippled` server using NuDB:

Expand All @@ -101,7 +97,7 @@ online_delete=2000
advisory_delete=0
```

(Adjust the `path` to the directory where you want to keep the ledger store on disk. Adjust the `online_delete` and `advisory_delete` settings as desired for your configuration.)
Adjust the `path` to the directory where you want to keep the ledger store on disk. Adjust the `online_delete` and `advisory_delete` settings as desired for your configuration. For more details about these settings, see [Configure Online Deletion](configure-online-deletion.html) and [Configure Advisory Deletion](configure-advisory-deletion.html).


### Log Level
Expand Down Expand Up @@ -139,18 +135,18 @@ The `[node_db]` stanza controls the server's _ledger store_, which holds [ledger

You can control how much data you keep with [online deletion](online-deletion.html); the default config file has the server keep the latest 2000 ledger versions. Without online deletion, the server's disk requirements grow without bounds.

The following table approximates the requirements for different amounts of history, at the time of writing (2018-12-13):

| Real Time Amount | Number of Ledger Versions | Disk Space Required (RocksDB) | Disk Space Required (NuDB) |
|:-----------------|:--------------------------|:------------------------------|:--|
| 2 hours | 2,000 | 250 MB | 450 MB |
| 1 day | 25,000 | 8 GB | 12 GB |
| 14 days | 350,000 | 112 GB | 168 GB |
| 30 days | 750,000 | 240 GB | 360 GB |
| 90 days | 2,250,000 | 720 GB | 1 TB |
| 1 year | 10,000,000 | 3 TB | 4.5 TB |
| 2 years | 20,000,000 | 6 TB | 9 TB |
| Full history (as of 2022-12-18) | 76,500,000+ | (Not recommended) | ~22.3 TB |
The following table approximates the requirements for different amounts of history, at the time of writing (2023-07-19):

| Real Time Amount | Number of Ledger Versions | Disk Space Required (NuDB) |
|:-----------------|:--------------------------|:---------------------------|
| 2 hours | 2,000 | 450 MB |
| 1 day | 25,000 | 12 GB |
| 14 days | 350,000 | 168 GB |
| 30 days | 750,000 | 360 GB |
| 90 days | 2,250,000 | 1 TB |
| 1 year | 10,000,000 | 4.5 TB |
| 2 years | 20,000,000 | 9 TB |
| Full history | 81,000,000+ | ~26 TB |

These numbers are estimates. They depend on several factors, most importantly the volume of transactions in the network. As transaction volume increases, each ledger version stores more unique data. You should provision extra storage capacity to prepare for future growth.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ labels:

For reliable performance in production environments, it is recommended to run an XRP Ledger (`rippled`) server on bare metal with the following characteristics or better:

- Operating System: Ubuntu (LTS) or CentOS or Red Hat Enterprise Linux (latest release).
- Operating System: Ubuntu (LTS), Red Hat Enterprise Linux (latest release), or a compatible Linux distribution.
- CPU: Intel Xeon 3+ GHz processor with 8+ cores and hyperthreading enabled.
- Disk: SSD / NVMe (10,000 IOPS sustained - not burst or peak - or better). Minimum 50 GB for the database partition. Do not use Amazon Elastic Block Store (AWS EBS) because its latency is too high to sync reliably.
- RAM: 64 GB.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ The exact look and feel of the user interface depend on your computer's operatin

The application in this tutorial _doesn't_ have the ability to send or trade [tokens](issued-currencies.html) or use other [payment types](payment-types.html) like Escrow or Payment Channels. However, it provides a foundation that you can implement those and other features on top of.

In addition to the above features, you'll also learn a little bit about graphical user interface (GUI) programming, threading, and asynchronous (async) code in Python.
Other topics mentioned in this tutorial include graphical user interface (GUI) programming, threading, and asynchronous (async) code in Python.

## Steps

Expand Down Expand Up @@ -67,8 +67,6 @@ On Windows, you can build apps using either Windows natively or by using the Win

On native Windows, the GUI uses native Windows controls and should run without any dependencies beyond those mentioned above.

**Caution:** As of 2022-02-01, the latest wxPython is not compatible with Python 3.10 on Windows. You should be able to follow this tutorial if you downgrade to the latest release of Python 3.9.

On WSL, you may need to install `libnotify-dev` as follows:

```sh
Expand Down Expand Up @@ -143,7 +141,7 @@ Finally, change the code to start the app (at the end of the file) slightly:

{{ include_code("_code-samples/build-a-wallet/py/2_threaded.py", language="py", start_with="if __name__") }}

Since the app uses a WebSocket client instead of the JSON-RPC client now, the code has to be use WebSocket URL to connect.
Since the app uses a WebSocket client instead of the JSON-RPC client now, the code has to use a WebSocket URL to connect.

**Tip:** If you [run your own `rippled` server](networks-and-servers.html#reasons-to-run-your-own-server) you can connect to it using `ws://localhost:6006` as the URL. You can also use the WebSocket URLs of [public servers](public-servers.html) to connect to the Mainnet or other test networks.

Expand Down
Loading