Merge remote-tracking branch 'upstream/develop' into action

* upstream/develop: docs: update SECURITY.md (4338) refactor: simplify `TxFormats` common fields logic (4637) APIv2(ledger_entry): return invalidParams for bad parameters (4630) docs(rippled-example.cfg): clarify ssl_cert vs ssl_chain (4667) Introduce replacement for getting and setting thread name: (4312) Set version to 1.12.0 Set version to 1.12.0-rc4 amm_info: fetch by amm account id; add AMM object entry (4682) AMMBid: use tecINTERNAL for 'impossible' errors (4674)
XRPLF · Sep 11, 2023 · f01002f · f01002f
2 parents f568ac0 + a955057
commit f01002f
Show file tree

Hide file tree

Showing 42 changed files with 1,234 additions and 662 deletions.
diff --git a/Builds/CMake/RippledCore.cmake b/Builds/CMake/RippledCore.cmake
@@ -24,7 +24,6 @@ add_library(xrpl::libxrpl ALIAS libxrpl)
 #]===============================]
 target_sources (xrpl_core PRIVATE
   src/ripple/beast/clock/basic_seconds_clock.cpp
-  src/ripple/beast/core/CurrentThreadName.cpp
   src/ripple/beast/core/SemanticVersion.cpp
   src/ripple/beast/hash/impl/xxhash.cpp
   src/ripple/beast/insight/impl/Collector.cpp
@@ -56,6 +55,7 @@ target_sources (xrpl_core PRIVATE
   src/ripple/basics/impl/Log.cpp
   src/ripple/basics/impl/Number.cpp
   src/ripple/basics/impl/StringUtilities.cpp
+  src/ripple/basics/impl/ThreadUtilities.cpp
   #[===============================[
     main sources:
       subdir: json
@@ -200,6 +200,7 @@ install (
     src/ripple/basics/TaggedCache.h
     src/ripple/basics/tagged_integer.h
     src/ripple/basics/ThreadSafetyAnalysis.h
+    src/ripple/basics/ThreadUtilities.h
     src/ripple/basics/ToString.h
     src/ripple/basics/UnorderedContainers.h
     src/ripple/basics/UptimeClock.h
@@ -838,6 +839,7 @@ if (tests)
     src/test/basics/Slice_test.cpp
     src/test/basics/StringUtilities_test.cpp
     src/test/basics/TaggedCache_test.cpp
+    src/test/basics/ThreadName_test.cpp
     src/test/basics/XRPAmount_test.cpp
     src/test/basics/base64_test.cpp
     src/test/basics/base_uint_test.cpp
@@ -855,7 +857,6 @@ if (tests)
     src/test/beast/LexicalCast_test.cpp
     src/test/beast/SemanticVersion_test.cpp
     src/test/beast/aged_associative_container_test.cpp
-    src/test/beast/beast_CurrentThreadName_test.cpp
     src/test/beast/beast_Journal_test.cpp
     src/test/beast/beast_PropertyStream_test.cpp
     src/test/beast/beast_Zero_test.cpp

diff --git a/RELEASENOTES.md b/RELEASENOTES.md
@@ -7,6 +7,135 @@ This document contains the release notes for `rippled`, the reference server imp
 
 Have new ideas? Need help with setting up your node? [Please open an issue here](https://github.com/xrplf/rippled/issues/new/choose).
 
+# Introducing XRP Ledger version 1.12.0
+
+Version 1.12.0 of `rippled`, the reference server implementation of the XRP Ledger protocol, is now available. This release adds new features and bug fixes, and introduces these amendments:
+
+- `AMM`
+- `Clawback`
+- `fixReducedOffersV1`
+
+[Sign Up for Future Release Announcements](https://groups.google.com/g/ripple-server)
+
+<!-- BREAK -->
+
+## Action Required
+
+Three new amendments are now open for voting according to the XRP Ledger's [amendment process](https://xrpl.org/amendments.html), which enables protocol changes following two weeks of >80% support from trusted validators.
+
+If you operate an XRP Ledger server, upgrade to version 1.12.0 by September 20, 2023 to ensure service continuity. The exact time that protocol changes take effect depends on the voting decisions of the decentralized network.
+
+
+## Install / Upgrade
+
+On supported platforms, see the [instructions on installing or updating `rippled`](https://xrpl.org/install-rippled.html).
+
+
+## Changelog
+
+### New Features and Amendments
+
+- **`AMM`**: Introduces an automated market maker (AMM) protocol to the XRP Ledger's decentralized exchange, enabling you to trade assets without a counterparty. For more information about AMMs, see: [Automated Market Maker](https://opensource.ripple.com/docs/xls-30d-amm/amm-uc/). [#4294](https://github.com/XRPLF/rippled/pull/4294)
+
+- **`Clawback`**: Allows issuers to add the `lsfAllowTrustLineClawback` flag to an issuing account. This enables the account to recover, or _claw back_, issued tokens after they're distributed to accounts. For additional documentation on this feature, see: [#4553](https://github.com/XRPLF/rippled/pull/4553).
+
+- **`fixReducedOffersV1`**: Reduces the occurrence of order books that are blocked by reduced offers. [#4512](https://github.com/XRPLF/rippled/pull/4512)
+
+- Added binary hardening and linker flags to enhance security during the build process. [#4603](https://github.com/XRPLF/rippled/pull/4603)
+
+- Updated build dependencies to the most recent versions in Conan Center. [#4595](https://github.com/XRPLF/rippled/pull/4595)
+
+- Updated Conan recipe for NuDB. [#4615](https://github.com/XRPLF/rippled/pull/4615)
+
+- Added a pre-commit hook that runs the clang-format linter locally before committing changes. To install this feature, see: [CONTRIBUTING](https://github.com/XRPLF/xrpl-dev-portal/blob/master/CONTRIBUTING.md). [#4599](https://github.com/XRPLF/rippled/pull/4599)
+
+- Added quality-of-life improvements to workflows, using new [concurrency control](https://docs.github.com/en/actions/using-jobs/using-concurrency) features. [#4597](https://github.com/XRPLF/rippled/pull/4597)
+
+- Added an Artifactory to the `nix` workflow to improve build times. [#4556](https://github.com/XRPLF/rippled/pull/4556)
+
+- Added WebSocket and RPC port info to `server_info` responses. [#4427](https://github.com/XRPLF/rippled/pull/4427)
+
+
+
+### Bug Fixes and Performance Improvements
+
+- Fixed an incorrect error response when there are missing fields in the API v2 `ledger_entry` method. [#4552](https://github.com/XRPLF/rippled/pull/4552)
+
+- Updated checkout versions to resolve warnings during Github jobs. [#4598](https://github.com/XRPLF/rippled/pull/4598)
+
+- Added an error response to the API v2 `account_info` method when you include an invalid `signer_lists` value. [#4585](https://github.com/XRPLF/rippled/pull/4585)
+
+- Fixed an issue with the debug package build. [#4591](https://github.com/XRPLF/rippled/pull/4591)
+
+- Added additional error responses to the API v2 `AccountTx` method. [#4571](https://github.com/XRPLF/rippled/pull/4571)
+
+- Fixed build references to deleted `ServerHandlerImp`. [#4592](https://github.com/XRPLF/rippled/pull/4592)
+
+- Fixed package definitions for Conan. [#4485](https://github.com/XRPLF/rippled/pull/4485)
+
+- Changed the output type of the `mulDiv()` function from `std::pair<bool, uint64_t>` to `std::optional`. [#4243](https://github.com/XRPLF/rippled/pull/4243)
+
+- Updated `Handler::Condition` enum values to make the code less brittle. [#4239](https://github.com/XRPLF/rippled/pull/4239)
+
+- Renamed `ServerHandlerImp` to `ServerHandler`. [#4516](https://github.com/XRPLF/rippled/pull/4516)
+
+- Removed the deprecated `accepted`, `seq`, `hash`, and `totalCoins` fields from the `ledger` method. [#4244](https://github.com/XRPLF/rippled/pull/4244)
+
+- Replaced hand-rolled code with `std::from_chars` for better maintainability. [#4473](https://github.com/XRPLF/rippled/pull/4473)
+
+- Removed an unused `TypedField` move constructor. [#4567](https://github.com/XRPLF/rippled/pull/4567)
+
+- Enabled the `BETA_RPC_API` flag in the default unit tests config, making the API v2 available to all unit tests. [#4573](https://github.com/XRPLF/rippled/pull/4573)
+
+
+### Docs
+
+- Updated build instructions with additional steps to take after updating dependencies. [#4623](https://github.com/XRPLF/rippled/pull/4623)
+
+- Updated contributing doc to clarify that beta releases should also be pushed to the `release` branch. [#4589](https://github.com/XRPLF/rippled/pull/4589)
+
+
+[Full Commit Log](https://github.com/XRPLF/rippled/compare/1.11.0...1.12.0)
+
+
+### GitHub
+
+The public source code repository for `rippled` is hosted on GitHub at <https://github.com/XRPLF/rippled>.
+
+We welcome all contributions and invite everyone to join the community of XRP Ledger developers to help build the Internet of Value.
+
+
+## Credits
+
+The following people contributed directly to this release:
+
+- Alphonse N. Mousse <[email protected]>
+- Arihant Kothari <[email protected]>
+- Chenna Keshava B S <[email protected]>
+- Denis Angell <[email protected]>
+- Ed Hennis <[email protected]>
+- Elliot Lee <[email protected]>
+- Gregory Tsipenyuk <[email protected]>
+- Howard Hinnant <[email protected]>
+- Ikko Eltociear Ashimine <[email protected]>
+- John Freeman <[email protected]>
+- Manoj Doshi <[email protected]>
+- Mark Travis <[email protected]>
+- Mayukha Vadari <[email protected]>
+- Michael Legleux <[email protected]>
+- Peter Chen <[email protected]>
+- RichardAH <[email protected]>
+- Rome Reginelli <[email protected]>
+- Scott Schurr <[email protected]>
+- Shawn Xie <[email protected]>
+- drlongle <[email protected]>
+
+Bug Bounties and Responsible Disclosures:
+
+We welcome reviews of the rippled code and urge researchers to responsibly disclose any issues they may find.
+
+To report a bug, please send a detailed report to: <[email protected]>
+
 
 # Introducing XRP Ledger version 1.11.0
 

diff --git a/SECURITY.md b/SECURITY.md
@@ -60,12 +60,12 @@ While we commit to responding with 24 hours of your initial report with our tria
 
 ## Bug Bounty Program
 
-[Ripple](https://ripple.com) is generously sponsoring a bug bounty program for vulnerabilities in [`rippled`](https://github.com/ripple/rippled) (and other related projects, like [`ripple-lib`](https://github.com/ripple/ripple-lib)).
+[Ripple](https://ripple.com) is generously sponsoring a bug bounty program for vulnerabilities in [`rippled`](https://github.com/XRPLF/rippled) (and other related projects, like [`xrpl.js`](https://github.com/XRPLF/xrpl.js), [`xrpl-py`](https://github.com/XRPLF/xrpl-py), [`xrpl4j`](https://github.com/XRPLF/xrpl4j)).
 
-This program allows us to recognize and reward individuals or groups that identify and report bugs. In summary, order to qualify for a bounty, the bug must be:
+This program allows us to recognize and reward individuals or groups that identify and report bugs. In summary, in order to qualify for a bounty, the bug must be:
 
-1. **In scope**. Only bugs in software under the scope of the program qualify. Currently, that means `rippled` and `ripple-lib`.
-2. **Relevant**. A security issue, posing a danger to user funds, privacy or the operation of the XRP Ledger.
+1. **In scope**. Only bugs in software under the scope of the program qualify. Currently, that means `rippled`, `xrpl.js`, `xrpl-py`, `xrpl4j`.
+2. **Relevant**. A security issue, posing a danger to user funds, privacy, or the operation of the XRP Ledger.
 3. **Original and previously unknown**. Bugs that are already known and discussed in public do not qualify. Previously reported bugs, even if publicly unknown, are not eligible.
 4. **Specific**. We welcome general security advice or recommendations, but we cannot pay bounties for that.
 5. **Fixable**. There has to be something we can do to permanently fix the problem. Note that bugs in other people’s software may still qualify in some cases. For example, if you find a bug in a library that we use which can compromise the security of software that is in scope and we can get it fixed, you may qualify for a bounty.

diff --git a/cfg/rippled-example.cfg b/cfg/rippled-example.cfg
@@ -283,12 +283,14 @@
 #       ssl_cert
 #
 #           Specifies the path to the SSL certificate file in PEM format.
-#           This is not needed if the chain includes it.
+#           This is not needed if the chain includes it. Use ssl_chain if
+#           your certificate includes one or more intermediates.
 #
 #       ssl_chain
 #
 #           If you need a certificate chain, specify the path to the
 #           certificate chain here. The chain may include the end certificate.
+#           This must be used if the certificate includes intermediates.
 #
 #   ssl_ciphers = <cipherlist>
 #

diff --git a/src/ripple/app/ledger/impl/LedgerCleaner.cpp b/src/ripple/app/ledger/impl/LedgerCleaner.cpp
@@ -21,7 +21,7 @@
 #include <ripple/app/ledger/LedgerCleaner.h>
 #include <ripple/app/ledger/LedgerMaster.h>
 #include <ripple/app/misc/LoadFeeTrack.h>
-#include <ripple/beast/core/CurrentThreadName.h>
+#include <ripple/basics/ThreadUtilities.h>
 #include <ripple/protocol/jss.h>
 
 namespace ripple {
@@ -218,7 +218,7 @@ class LedgerCleanerImp : public LedgerCleaner
     void
     run()
     {
-        beast::setCurrentThreadName("LedgerCleaner");
+        this_thread::set_name("LedgerCleaner");
         JLOG(j_.debug()) << "Started";
 
         while (true)

diff --git a/src/ripple/app/main/BasicApp.cpp b/src/ripple/app/main/BasicApp.cpp
@@ -18,7 +18,7 @@
 //==============================================================================
 
 #include <ripple/app/main/BasicApp.h>
-#include <ripple/beast/core/CurrentThreadName.h>
+#include <ripple/basics/ThreadUtilities.h>
 
 BasicApp::BasicApp(std::size_t numberOfThreads)
 {
@@ -28,7 +28,7 @@ BasicApp::BasicApp(std::size_t numberOfThreads)
     while (numberOfThreads--)
     {
         threads_.emplace_back([this, numberOfThreads]() {
-            beast::setCurrentThreadName(
+            ripple::this_thread::set_name(
                 "io svc #" + std::to_string(numberOfThreads));
             this->io_service_.run();
         });

diff --git a/src/ripple/app/main/GRPCServer.cpp b/src/ripple/app/main/GRPCServer.cpp
@@ -19,7 +19,7 @@
 
 #include <ripple/app/main/GRPCServer.h>
 #include <ripple/app/reporting/P2pProxy.h>
-#include <ripple/beast/core/CurrentThreadName.h>
+#include <ripple/basics/ThreadUtilities.h>
 #include <ripple/resource/Fees.h>
 
 #include <ripple/beast/net/IPAddressConversion.h>
@@ -692,9 +692,8 @@ GRPCServer::start()
     if (running_ = impl_.start(); running_)
     {
         thread_ = std::thread([this]() {
-            beast::setCurrentThreadName("rippled : GRPCServer");
             // Start the event loop and begin handling requests
-            beast::setCurrentThreadName("rippled: grpc");
+            this_thread::set_name("rippled: grpc");
             this->impl_.handleRpcs();
         });
     }

diff --git a/src/ripple/app/main/LoadManager.cpp b/src/ripple/app/main/LoadManager.cpp
@@ -21,8 +21,8 @@
 #include <ripple/app/main/LoadManager.h>
 #include <ripple/app/misc/LoadFeeTrack.h>
 #include <ripple/app/misc/NetworkOPs.h>
+#include <ripple/basics/ThreadUtilities.h>
 #include <ripple/basics/UptimeClock.h>
-#include <ripple/beast/core/CurrentThreadName.h>
 #include <ripple/json/to_string.h>
 #include <memory>
 #include <mutex>
@@ -99,7 +99,7 @@ LoadManager::stop()
 void
 LoadManager::run()
 {
-    beast::setCurrentThreadName("LoadManager");
+    this_thread::set_name("LoadManager");
 
     using namespace std::chrono_literals;
     using clock_type = std::chrono::steady_clock;

diff --git a/src/ripple/app/main/Main.cpp b/src/ripple/app/main/Main.cpp
@@ -22,9 +22,9 @@
 #include <ripple/app/rdb/Vacuum.h>
 #include <ripple/basics/Log.h>
 #include <ripple/basics/StringUtilities.h>
+#include <ripple/basics/ThreadUtilities.h>
 #include <ripple/basics/contract.h>
 #include <ripple/beast/clock/basic_seconds_clock.h>
-#include <ripple/beast/core/CurrentThreadName.h>
 #include <ripple/core/Config.h>
 #include <ripple/core/ConfigSections.h>
 #include <ripple/core/TimeKeeper.h>
@@ -348,8 +348,7 @@ run(int argc, char** argv)
 {
     using namespace std;
 
-    beast::setCurrentThreadName(
-        "rippled: main " + BuildInfo::getVersionString());
+    this_thread::set_name("main " + BuildInfo::getVersionString());
 
     po::variables_map vm;
 
@@ -777,7 +776,7 @@ run(int argc, char** argv)
     }
 
     // We have an RPC command to process:
-    beast::setCurrentThreadName("rippled: rpc");
+    this_thread::set_name("rippled: rpc");
     return RPCCall::fromCommandLine(
         *config, vm["parameters"].as<std::vector<std::string>>(), *logs);
 }

diff --git a/src/ripple/app/misc/SHAMapStoreImp.cpp b/src/ripple/app/misc/SHAMapStoreImp.cpp
@@ -23,7 +23,7 @@
 #include <ripple/app/misc/NetworkOPs.h>
 #include <ripple/app/rdb/State.h>
 #include <ripple/app/rdb/backend/SQLiteDatabase.h>
-#include <ripple/beast/core/CurrentThreadName.h>
+#include <ripple/basics/ThreadUtilities.h>
 #include <ripple/core/ConfigSections.h>
 #include <ripple/core/Pg.h>
 #include <ripple/nodestore/Scheduler.h>
@@ -286,7 +286,7 @@ SHAMapStoreImp::run()
             "Reporting does not support online_delete. Remove "
             "online_delete info from config");
     }
-    beast::setCurrentThreadName("SHAMapStore");
+    this_thread::set_name("SHAMapStore");
     LedgerIndex lastRotated = state_db_.getState().lastRotated;
     netOPs_ = &app_.getOPs();
     ledgerMaster_ = &app_.getLedgerMaster();

diff --git a/src/ripple/app/misc/impl/AMMUtils.cpp b/src/ripple/app/misc/impl/AMMUtils.cpp
@@ -200,14 +200,17 @@ deleteAMMTrustLines(
         keylet::ownerDir(ammAccountID),
         [&](LedgerEntryType nodeType,
             uint256 const&,
-            std::shared_ptr<SLE>& sleItem) -> TER {
+            std::shared_ptr<SLE>& sleItem) -> std::pair<TER, SkipEntry> {
+            // Skip AMM
+            if (nodeType == LedgerEntryType::ltAMM)
+                return {tesSUCCESS, SkipEntry::Yes};
             // Should only have the trustlines
             if (nodeType != LedgerEntryType::ltRIPPLE_STATE)
             {
                 JLOG(j.error())
                     << "deleteAMMTrustLines: deleting non-trustline "
                     << nodeType;
-                return tecINTERNAL;
+                return {tecINTERNAL, SkipEntry::No};
             }
 
             // Trustlines must have zero balance
@@ -216,10 +219,12 @@ deleteAMMTrustLines(
                 JLOG(j.error())
                     << "deleteAMMTrustLines: deleting trustline with "
                        "non-zero balance.";
-                return tecINTERNAL;
+                return {tecINTERNAL, SkipEntry::No};
             }
 
-            return deleteAMMTrustLine(sb, sleItem, ammAccountID, j);
+            return {
+                deleteAMMTrustLine(sb, sleItem, ammAccountID, j),
+                SkipEntry::No};
         },
         j,
         maxTrustlinesToDelete);
@@ -255,6 +260,12 @@ deleteAMMAccount(
         return ter;
 
     auto const ownerDirKeylet = keylet::ownerDir(ammAccountID);
+    if (!sb.dirRemove(
+            ownerDirKeylet, (*ammSle)[sfOwnerNode], ammSle->key(), false))
+    {
+        JLOG(j.error()) << "deleteAMMAccount: failed to remove dir link";
+        return tecINTERNAL;
+    }
     if (sb.exists(ownerDirKeylet) && !sb.emptyDirDelete(ownerDirKeylet))
     {
         JLOG(j.error()) << "deleteAMMAccount: cannot delete root dir node of "

diff --git a/src/ripple/app/reporting/ETLSource.cpp b/src/ripple/app/reporting/ETLSource.cpp
@@ -19,7 +19,6 @@
 
 #include <ripple/app/reporting/ETLSource.h>
 #include <ripple/app/reporting/ReportingETL.h>
-#include <ripple/beast/core/CurrentThreadName.h>
 #include <ripple/json/json_reader.h>
 #include <ripple/json/json_writer.h>