[safestack] Various Solaris fixes

[rorth]

Even with the -u __safestack_init link order fixed on Solaris, there are still several safestack test issues left:

• While 540fd42 enabled safestack on Solaris in the driver unconditionally, it ignored that Solaris also exists on SPARC and forgot to enable SPARC support for the runtime lib. This patch fixes that.

• The tests fail to link with undefined references to __sanitizer_internal_memset etc. These are from sanitizer_redefine_builtins.h. Definitions live in sanitizer_libc.cpp.o. This patch adds them to the safestack runtime lib as is already the case e.g. for asan and ubsan. Why GNU ld allows the link to complete with those references undefined is beyond me.

• The pthread*.c tests FAIL with

  safestack CHECK failed: /vol/llvm/src/llvm-project/local/compiler-rt/lib/safestack/safestack.cpp:227 size

  The problem is that pthread_attr_init initializes the stacksize attribute to 0, signifying the default. Unless explicitly overridded, it stays that way. I think this is allowed by XPG7. Since safestack cannot deal with this, I set size to the defaults documented in pthread_create(3C). Unfortunately, there's no macro for those values outside of private libc headers.

• The Solaris syscall interface isn't stable. This is not just a theoretical concern, but the syscalls have changed incompatibly several times in the past. Therefore this patch switches the implementations of TgKill (where SYS_lwp_kill doesn't exist on Solaris 11.4 anyway), Mmap, Munmap, and Mprotect to the same _REAL* solution already used in sanitizer_solaris.cpp. Instead of duplicating what's already in sanitizer_common, it seems way better to me to just reuse those implementations, though. A subsequent patch does just that.

With those changes, safestack compiles and all tests PASS, so the tests are re-enabled for good.

Tested on amd64-pc-solaris2.11, sparcv9-sun-solaris2.11, x86_64-pc-linux-gnu, and sparc64-unknown-linux-gnu.

[rorth]

This patch was already approved in [safestack] Various Solaris fixes #98001 , therefore I'll commit it shortly.

[github-actions]

⚠️ C/C++ code formatter, clang-format found issues in your code. ⚠️

You can test this locally with the following command:

git-clang-format --diff d4e46f0e864e37085da0c5e56e4f6f278e2f7aee 97a3888b2da8663395967ee76470b41c146522f9 --extensions cpp,h -- compiler-rt/lib/safestack/safestack.cpp compiler-rt/lib/safestack/safestack_platform.h

View the diff from clang-format here.

diff --git a/compiler-rt/lib/safestack/safestack_platform.h b/compiler-rt/lib/safestack/safestack_platform.h
index 77eeb9cda6..27e29d6ead 100644
--- a/compiler-rt/lib/safestack/safestack_platform.h
+++ b/compiler-rt/lib/safestack/safestack_platform.h
@@ -13,9 +13,6 @@
 #ifndef SAFESTACK_PLATFORM_H
 #define SAFESTACK_PLATFORM_H
 
-#include "safestack_util.h"
-#include "sanitizer_common/sanitizer_platform.h"
-
 #include <dlfcn.h>
 #include <errno.h>
 #include <stdint.h>
@@ -26,6 +23,9 @@
 #include <sys/types.h>
 #include <unistd.h>
 
+#include "safestack_util.h"
+#include "sanitizer_common/sanitizer_platform.h"
+
 #if !(SANITIZER_NETBSD || SANITIZER_FREEBSD || SANITIZER_LINUX || \
       SANITIZER_SOLARIS)
 #  error "Support for your platform has not been implemented"