Skip to content

Commit

Permalink
[libc++][hardening] In production hardening modes, trap rather than a…
Browse files Browse the repository at this point in the history
…bort (#78561)

In the hardening modes that can be used in production (`fast` and
`extensive`), make a failed assertion invoke a trap instruction rather
than calling verbose abort. In the debug mode, still keep calling
verbose abort to provide a better user experience and to allow us to
keep our existing testing infrastructure for verifying assertion
messages. Since the debug mode by definition enables all assertions, we
can be sure that we still check all the assertion messages in the
library when running the test suite in the debug mode.

The main motivation to use trapping in production is to achieve better
code generation and reduce the binary size penalty. This way, the
assertion handler can compile to a single instruction, whereas the
existing mechanism with verbose abort results in generating a function
call that in general cannot be optimized away (made worse by the fact
that it's a variadic function, imposing an additional penalty). See the
[RFC](https://discourse.llvm.org/t/rfc-hardening-in-libc/73925) for more
details. Note that this mechanism can now be completely [overridden at
CMake configuration
time](#77883).

This patch also significantly refactors `check_assertion.h` and expands
its test coverage. The main changes:
- when overriding `verbose_abort`, don't do matching inside the function
-- just print the error message to `stderr`. This removes the need to
set a global matcher and allows to do matching in the parent process
after the child finishes;
- remove unused logic for matching source locations and for using
wildcards;
- make matchers simple functors;
- introduce `DeathTestResult` that keeps data about the test run,
primarily to make it easier to test.

In addition to the refactoring, `check_assertion.h` can now recognize
when a process exits due to a trap.
  • Loading branch information
var-const authored Jan 19, 2024
1 parent 0388ab3 commit 58780b8
Show file tree
Hide file tree
Showing 119 changed files with 483 additions and 354 deletions.
41 changes: 19 additions & 22 deletions libcxx/docs/BuildingLibcxx.rst
Original file line number Diff line number Diff line change
Expand Up @@ -485,28 +485,25 @@ LLVM-specific options
.. _assertion-handler:

Overriding the default assertion handler
==========================================

When the library wants to terminate due to an unforeseen condition (such as
a hardening assertion failure), the program is aborted through a special verbose
termination function. The library provides a default function that prints an
error message and calls ``std::abort()``. Note that this function is provided by
the static or shared library, so it is only available when deploying to
a platform where the compiled library is sufficiently recent. On older
platforms, the program will terminate in an unspecified unsuccessful manner, but
the quality of diagnostics won't be great.

However, vendors can also override that mechanism at CMake configuration time.
When a hardening assertion fails, the library invokes the
``_LIBCPP_ASSERTION_HANDLER`` macro. A vendor may provide a header that contains
a custom definition of this macro and specify the path to the header via the
``LIBCXX_ASSERTION_HANDLER_FILE`` CMake variable. If provided, this header will
be included by the library and replace the default implementation. The header
must not include any standard library headers (directly or transitively) because
doing so will almost always create a circular dependency. The
``_LIBCPP_ASSERTION_HANDLER(message)`` macro takes a single parameter that
contains an error message explaining the hardening failure and some details
about the source location that triggered it.
========================================

When the library wants to terminate due to a hardening assertion failure, the
program is aborted by invoking a trap instruction (or in debug mode, by
a special verbose termination function that prints an error message and calls
``std::abort()``). This is done to minimize the code size impact of enabling
hardening in the library. However, vendors can also override that mechanism at
CMake configuration time.

Under the hood, a hardening assertion will invoke the
``_LIBCPP_ASSERTION_HANDLER`` macro upon failure. A vendor may provide a header
that contains a custom definition of this macro and specify the path to the
header via the ``LIBCXX_ASSERTION_HANDLER_FILE`` CMake variable. If provided,
this header will be included by the library and replace the default
implementation. The header must not include any standard library headers
(directly or transitively) because doing so will almost always create a circular
dependency. The ``_LIBCPP_ASSERTION_HANDLER(message)`` macro takes a single
parameter that contains an error message explaining the hardening failure and
some details about the source location that triggered it.

When a hardening assertion fails, it means that the program is about to invoke
library undefined behavior. For this reason, the custom assertion handler is
Expand Down
21 changes: 14 additions & 7 deletions libcxx/docs/ReleaseNotes/18.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,13 +107,18 @@ Deprecations and Removals
macro is provided to restore the previous behavior, and it will be supported in the LLVM 18 release only.
In LLVM 19 and beyond, ``_LIBCPP_ENABLE_NARROWING_CONVERSIONS_IN_VARIANT`` will not be honored anymore.

- The only supported way to customize the assertion handler that gets invoked when a hardening assertion fails
is now by setting the ``LIBCXX_ASSERTION_HANDLER_FILE`` CMake variable and providing a custom header. See
the documentation on overriding the default assertion handler for details.
- Overriding `__libcpp_verbose_abort` no longer has any effect on library assertions. The only supported way
to customize the assertion handler that gets invoked when a hardening assertion fails is now by setting the
``LIBCXX_ASSERTION_HANDLER_FILE`` CMake variable and providing a custom header. See the documentation on
overriding the default assertion handler for details. The ability to override `__libcpp_verbose_abort` will
be removed in an upcoming release in favor of the new overriding mechanism.

- In safe mode (which is now equivalent to the ``extensive`` hardening mode), a failed assertion will now
generate a trap rather than a call to verbose abort.

- The ``_LIBCPP_AVAILABILITY_CUSTOM_VERBOSE_ABORT_PROVIDED`` macro is not honored anymore in LLVM 18.
Please see the updated documentation about the hardening modes in libc++ and in particular the
``_LIBCPP_VERBOSE_ABORT`` macro for details.
Please see the updated documentation about the hardening modes in libc++ and in particular on
overriding the default assertion handler.

- The headers ``<experimental/deque>``, ``<experimental/forward_list>``, ``<experimental/list>``,
``<experimental/map>``, ``<experimental/memory_resource>``, ``<experimental/regex>``, ``<experimental/set>``,
Expand All @@ -136,13 +141,15 @@ Deprecations and Removals
Upcoming Deprecations and Removals
----------------------------------

- The ability to override ``__libcpp_verbose_abort`` will be removed in an upcoming release.

LLVM 19
~~~~~~~

- The ``LIBCXX_ENABLE_ASSERTIONS`` CMake variable that was used to enable the safe mode will be deprecated and setting
it will trigger an error; use the ``LIBCXX_HARDENING_MODE`` variable with the value ``extensive`` instead. Similarly,
the ``_LIBCPP_ENABLE_ASSERTIONS`` macro will be deprecated (setting it to ``1`` still enables the extensive mode the
LLVM 19 release while also issuing a deprecation warning). See :ref:`the hardening documentation
the ``_LIBCPP_ENABLE_ASSERTIONS`` macro will be deprecated (setting it to ``1`` still enables the extensive mode in
the LLVM 19 release while also issuing a deprecation warning). See :ref:`the hardening documentation
<using-hardening-modes>` for more details.

- The base template for ``std::char_traits`` has been marked as deprecated and will be removed in LLVM 19. If you
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03, c++11, c++14, c++17
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <algorithm>
#include <array>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03, c++11, c++14, c++17
// UNSUPPORTED: !libcpp-hardening-mode=debug
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing
// ADDITIONAL_COMPILE_FLAGS: -D_LIBCPP_DEBUG_STRICT_WEAK_ORDERING_CHECK
// When the debug mode is enabled, this test fails because we actually catch on the fly that the comparator is not
// a strict-weak ordering before we catch that we'd dereference out-of-bounds inside std::sort, which leads to different
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ void my_abort(char const*, ...) {
}

int main(int, char**) {
_LIBCPP_ASSERT(false, "message");
_LIBCPP_VERBOSE_ABORT("%s", "message");
return EXIT_FAILURE;
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ void std::__libcpp_verbose_abort(char const*, ...) {
}

int main(int, char**) {
_LIBCPP_ASSERT(false, "message");
std::__libcpp_verbose_abort("%s", "message");
return EXIT_FAILURE;
}
4 changes: 3 additions & 1 deletion libcxx/test/libcxx/assertions/default_verbose_abort.pass.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@
//===----------------------------------------------------------------------===//

// Test that the default verbose termination function aborts the program.
// XFAIL: availability-verbose_abort-missing

#include <__verbose_abort>
#include <csignal>
#include <cstdlib>

Expand All @@ -19,6 +21,6 @@ void signal_handler(int signal) {

int main(int, char**) {
if (std::signal(SIGABRT, signal_handler) != SIG_ERR)
_LIBCPP_ASSERT(false, "foo");
std::__libcpp_verbose_abort("%s", "foo");
return EXIT_FAILURE;
}
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@

// `check_assertion.h` is only available starting from C++11 and requires Unix headers and regex support.
// UNSUPPORTED: c++03, !has-unix-headers, no-localization
// The ability to set a custom abort message is required to compare the assertion message.
// XFAIL: availability-verbose_abort-missing
// The ability to set a custom abort message is required to compare the assertion message (which only happens in the
// debug mode).
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing
// Note that GCC doesn't support `-Wno-macro-redefined`.
// ADDITIONAL_COMPILE_FLAGS: -U_LIBCPP_HARDENING_MODE -D_LIBCPP_ENABLE_ASSERTIONS=1

Expand Down
1 change: 0 additions & 1 deletion libcxx/test/libcxx/assertions/modes/extensive.pass.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
// UNSUPPORTED: c++03
// `check_assertion.h` requires Unix headers.
// REQUIRES: has-unix-headers
// XFAIL: availability-verbose_abort-missing

#include <cassert>
#include "check_assertion.h"
Expand Down
1 change: 0 additions & 1 deletion libcxx/test/libcxx/assertions/modes/fast.pass.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
// UNSUPPORTED: c++03
// `check_assertion.h` requires Unix headers.
// REQUIRES: has-unix-headers
// XFAIL: availability-verbose_abort-missing

#include <cassert>
#include "check_assertion.h"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@

// `check_assertion.h` is only available starting from C++11 and requires Unix headers and regex support.
// UNSUPPORTED: c++03, !has-unix-headers, no-localization
// The ability to set a custom abort message is required to compare the assertion message.
// XFAIL: availability-verbose_abort-missing
// The ability to set a custom abort message is required to compare the assertion message (which only happens in the
// debug mode).
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing
// ADDITIONAL_COMPILE_FLAGS: -U_LIBCPP_HARDENING_MODE -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE

#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@

// `check_assertion.h` is only available starting from C++11 and requires Unix headers and regex support.
// UNSUPPORTED: c++03, !has-unix-headers, no-localization
// The ability to set a custom abort message is required to compare the assertion message.
// XFAIL: availability-verbose_abort-missing
// The ability to set a custom abort message is required to compare the assertion message (which only happens in the
// debug mode).
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing
// ADDITIONAL_COMPILE_FLAGS: -U_LIBCPP_HARDENING_MODE -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_FAST

#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

// test that array<T, 0>::back() triggers an assertion

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

// test that array<T, 0>::back() triggers an assertion

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

// test that array<T, 0>::operator[] triggers an assertion

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <deque>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <list>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <list>
#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>
#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>
#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>
#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>
#include <cassert>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// UNSUPPORTED: libcpp-hardening-mode=none
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <vector>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_map>
#include <string>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_map>
#include <string>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_map>
#include <string>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_map>
#include <string>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_map>
#include <string>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_map>
#include <string>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_set>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_set>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
// REQUIRES: has-unix-headers
// UNSUPPORTED: c++03
// REQUIRES: libcpp-hardening-mode={{extensive|debug}}
// XFAIL: availability-verbose_abort-missing
// XFAIL: libcpp-hardening-mode=debug && availability-verbose_abort-missing

#include <unordered_set>

Expand Down
Loading

2 comments on commit 58780b8

@maryammo
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@var-const , This commit makes libcxx/containers/views/mdspan/layout_stride/assert.conversion.pass.cpp to unexpectedly pass for PowerPC starting
https://lab.llvm.org/buildbot/#/builders/57/builds/32354.
Looking at why it failed before this commit, the failure was:

# .---command stderr------------
# | EXPECT_DEATH( ([=] { std::layout_stride::mapping<std::extents<char, D, D>> m(offset_map); }()) ) failed! (matcher failed)
# | 
# | child exit code: 2
# | ---------- standard out ----------
# | Failed to match assertion info!
# | msg = "layout_stride::mapping converting ctor: all strides must be greater than 0"
# | line = '*'
# | file = '*'
# | VS
# | /home/maryammo/llvm-workspace/myFork/build/include/c++/v1/__mdspan/layout_stride.h:249 (layout_stride::mapping converting ctor: base offset of mapping must be zero.)
# | 
# | t.tmp.exe: /home/maryammo/llvm-workspace/myFork/llvm-project/libcxx/test/libcxx/containers/views/mdspan/layout_stride/assert.conversion.pass.cpp:76: int main(int, char **): Assertion `(ExpectDeath("([=] { std::layout_stride::mapping<std::extents<char, D, D>> m(offset_map); }())", [&]() { (void)(([=] { std::layout_stride::mapping<std::extents<char, D, D>> m(offset_map); }())); }, AssertionInfoMatcher("layout_stride::mapping converting ctor: all strides must be greater than 0")))' failed.
# `-----------------------------
# error: command failed with exit status: 250

Would you please confirm if your patch should fix that error and if so elaborate on how? Thanks.

@var-const
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryammo Starting from this patch, we no longer check the assertion message in non-debug hardening modes. It doesn't really fix the PowerPC issue, just hides it. It seems like those tests are run in the extensive mode, so that now all that's required for them to pass is to trap, regardless of the exact message. If you were to run those tests in the debug mode, you would still see the PowerPC-specific failures. You might want to update the annotation to:

// XFAIL: libcpp-hardening-mode=debug && target=powerpc{{.*}}le-unknown-linux-gnu

Please sign in to comment.