[libc++] Implements the new version header generator. #97847
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
llvmbot
added
the
libc++
|
@llvm/pr-subscribers-libcxx Author: Mark de Wever (mordante) ChangesThe generator makes a few changes to the output
This code uses the new FTM data structure. Since the contents of this structure are not up-to-date the code is only used in its tests. Full diff: https://github.com/llvm/llvm-project/pull/97847.diff 7 Files Affected:
diff --git a/libcxx/test/libcxx/feature_test_macro/ftm_meta_data.sh.py b/libcxx/test/libcxx/feature_test_macro/ftm_meta_data.sh.py
new file mode 100644
index 00000000000000..5d12a68a40eaa8
--- /dev/null
+++ b/libcxx/test/libcxx/feature_test_macro/ftm_meta_data.sh.py
@@ -0,0 +1,51 @@
+# ===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ===----------------------------------------------------------------------===##
+
+# RUN: %{python} %s %{libcxx-dir}/utils %{libcxx-dir}/test/libcxx/feature_test_macro/test_data.json
+
+import sys
+
+sys.path.append(sys.argv[1])
+from generate_feature_test_macro_components import FeatureTestMacros
+
+
+def test(output, expected):
+ assert output == expected, f"expected\n{expected}\n\noutput\n{output}"
+
+
+ftm = FeatureTestMacros(sys.argv[2])
+test(
+ ftm.ftm_meta_data,
+ {
+ "__cpp_lib_any": {
+ "headers": ["any"],
+ "test_suite_guard": None,
+ "libcxx_guard": None,
+ },
+ "__cpp_lib_barrier": {
+ "headers": ["barrier"],
+ "test_suite_guard": "!defined(_LIBCPP_HAS_NO_THREADS) && (!defined(_LIBCPP_VERSION) || _LIBCPP_AVAILABILITY_HAS_SYNC)",
+ "libcxx_guard": "!defined(_LIBCPP_HAS_NO_THREADS) && _LIBCPP_AVAILABILITY_HAS_SYNC",
+ },
+ "__cpp_lib_format": {
+ "headers": ["format"],
+ "test_suite_guard": None,
+ "libcxx_guard": None,
+ },
+ "__cpp_lib_parallel_algorithm": {
+ "headers": ["algorithm", "numeric"],
+ "test_suite_guard": None,
+ "libcxx_guard": None,
+ },
+ "__cpp_lib_variant": {
+ "headers": ["variant"],
+ "test_suite_guard": None,
+ "libcxx_guard": None,
+ },
+ },
+)
diff --git a/libcxx/test/libcxx/feature_test_macro/implemented_ftms.sh.py b/libcxx/test/libcxx/feature_test_macro/implemented_ftms.sh.py
index 67353fc41e5098..62a3c46b15db3c 100644
--- a/libcxx/test/libcxx/feature_test_macro/implemented_ftms.sh.py
+++ b/libcxx/test/libcxx/feature_test_macro/implemented_ftms.sh.py
@@ -31,7 +31,7 @@ def test(output, expected):
"__cpp_lib_barrier": {
"c++20": "201907L",
"c++23": "201907L",
- "c++26": "201907L",
+ "c++26": "299900L",
},
"__cpp_lib_format": {
"c++20": None,
diff --git a/libcxx/test/libcxx/feature_test_macro/standard_ftms.sh.py b/libcxx/test/libcxx/feature_test_macro/standard_ftms.sh.py
index 43c90b131bff16..231e6072898752 100644
--- a/libcxx/test/libcxx/feature_test_macro/standard_ftms.sh.py
+++ b/libcxx/test/libcxx/feature_test_macro/standard_ftms.sh.py
@@ -31,7 +31,7 @@ def test(output, expected):
"__cpp_lib_barrier": {
"c++20": "201907L",
"c++23": "201907L",
- "c++26": "201907L",
+ "c++26": "299900L",
},
"__cpp_lib_format": {
"c++20": "202110L",
diff --git a/libcxx/test/libcxx/feature_test_macro/test_data.json b/libcxx/test/libcxx/feature_test_macro/test_data.json
index 1f8bbe5d769b5b..18c8836fa5149f 100644
--- a/libcxx/test/libcxx/feature_test_macro/test_data.json
+++ b/libcxx/test/libcxx/feature_test_macro/test_data.json
@@ -23,6 +23,13 @@
"implemented": true
}
]
+ },
+ "c++26": {
+ "299900": [
+ {
+ "implemented": true
+ }
+ ]
}
},
"headers": [
diff --git a/libcxx/test/libcxx/feature_test_macro/version_header.sh.py b/libcxx/test/libcxx/feature_test_macro/version_header.sh.py
new file mode 100644
index 00000000000000..bbb863371f7639
--- /dev/null
+++ b/libcxx/test/libcxx/feature_test_macro/version_header.sh.py
@@ -0,0 +1,73 @@
+# ===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ===----------------------------------------------------------------------===##
+
+# RUN: %{python} %s %{libcxx-dir}/utils %{libcxx-dir}/test/libcxx/feature_test_macro/test_data.json
+
+import sys
+
+sys.path.append(sys.argv[1])
+from generate_feature_test_macro_components import FeatureTestMacros
+
+
+def test(output, expected):
+ assert output == expected, f"expected\n{expected}\n\noutput\n{output}"
+
+
+ftm = FeatureTestMacros(sys.argv[2])
+test(
+ ftm.version_header,
+ """// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP_VERSION
+#define _LIBCPP_VERSION
+
+#include <__config>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+#if _LIBCPP_STD_VER >= 17
+# define __cpp_lib_any 201606L
+# define __cpp_lib_parallel_algorithm 201603L
+# define __cpp_lib_variant 202102L
+#endif // _LIBCPP_STD_VER >= 17
+
+#if _LIBCPP_STD_VER >= 20
+# if !defined(_LIBCPP_HAS_NO_THREADS) && _LIBCPP_AVAILABILITY_HAS_SYNC
+# define __cpp_lib_barrier 201907L
+# endif
+// define __cpp_lib_format 202110L
+# undef __cpp_lib_variant
+# define __cpp_lib_variant 202106L
+#endif // _LIBCPP_STD_VER >= 20
+
+#if _LIBCPP_STD_VER >= 23
+// define __cpp_lib_format 202207L
+#endif // _LIBCPP_STD_VER >= 23
+
+#if _LIBCPP_STD_VER >= 26
+# if !defined(_LIBCPP_HAS_NO_THREADS) && _LIBCPP_AVAILABILITY_HAS_SYNC
+# undef __cpp_lib_barrier
+# define __cpp_lib_barrier 299900L
+# endif
+// define __cpp_lib_format 202311L
+# undef __cpp_lib_variant
+# define __cpp_lib_variant 202306L
+#endif // _LIBCPP_STD_VER >= 26
+
+#endif // _LIBCPP_VERSION
+""",
+)
diff --git a/libcxx/test/libcxx/feature_test_macro/version_header_implementation.sh.py b/libcxx/test/libcxx/feature_test_macro/version_header_implementation.sh.py
new file mode 100644
index 00000000000000..db90206ae770f3
--- /dev/null
+++ b/libcxx/test/libcxx/feature_test_macro/version_header_implementation.sh.py
@@ -0,0 +1,114 @@
+# ===----------------------------------------------------------------------===##
+#
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# ===----------------------------------------------------------------------===##
+
+# RUN: %{python} %s %{libcxx-dir}/utils %{libcxx-dir}/test/libcxx/feature_test_macro/test_data.json
+
+import sys
+
+sys.path.append(sys.argv[1])
+from generate_feature_test_macro_components import FeatureTestMacros
+
+
+def test(output, expected):
+ assert output == expected, f"expected\n{expected}\n\noutput\n{output}"
+
+
+ftm = FeatureTestMacros(sys.argv[2])
+test(
+ ftm.version_header_implementation,
+ {
+ "17": [
+ {
+ "__cpp_lib_any": {
+ "value": "201606L",
+ "implemented": True,
+ "need_undef": False,
+ "condition": None,
+ },
+ },
+ {
+ "__cpp_lib_parallel_algorithm": {
+ "value": "201603L",
+ "implemented": True,
+ "need_undef": False,
+ "condition": None,
+ },
+ },
+ {
+ "__cpp_lib_variant": {
+ "value": "202102L",
+ "implemented": True,
+ "need_undef": False,
+ "condition": None,
+ },
+ },
+ ],
+ "20": [
+ {
+ "__cpp_lib_barrier": {
+ "value": "201907L",
+ "implemented": True,
+ "need_undef": False,
+ "condition": "!defined(_LIBCPP_HAS_NO_THREADS) && _LIBCPP_AVAILABILITY_HAS_SYNC",
+ },
+ },
+ {
+ "__cpp_lib_format": {
+ "value": "202110L",
+ "implemented": False,
+ "need_undef": False,
+ "condition": None,
+ },
+ },
+ {
+ "__cpp_lib_variant": {
+ "value": "202106L",
+ "implemented": True,
+ "need_undef": True,
+ "condition": None,
+ },
+ },
+ ],
+ "23": [
+ {
+ "__cpp_lib_format": {
+ "value": "202207L",
+ "implemented": False,
+ "need_undef": False,
+ "condition": None,
+ },
+ },
+ ],
+ "26": [
+ {
+ "__cpp_lib_barrier": {
+ "value": "299900L",
+ "implemented": True,
+ "need_undef": True,
+ "condition": "!defined(_LIBCPP_HAS_NO_THREADS) && _LIBCPP_AVAILABILITY_HAS_SYNC",
+ },
+ },
+ {
+ "__cpp_lib_format": {
+ "value": "202311L",
+ "implemented": False,
+ "need_undef": False,
+ "condition": None,
+ },
+ },
+ {
+ "__cpp_lib_variant": {
+ "value": "202306L",
+ "implemented": True,
+ "need_undef": True,
+ "condition": None,
+ },
+ },
+ ],
+ },
+)
diff --git a/libcxx/utils/generate_feature_test_macro_components.py b/libcxx/utils/generate_feature_test_macro_components.py
index 3f8ecc26321ee1..a683ac36d63087 100755
--- a/libcxx/utils/generate_feature_test_macro_components.py
+++ b/libcxx/utils/generate_feature_test_macro_components.py
@@ -1909,6 +1909,55 @@ def get_ftms(
return result
+def generate_version_header_dialect_block(data: Dict[str, Any]) -> str:
+ """Generates the contents of the version header for a dialect.
+
+ This generates the contents of a
+ #if _LIBCPP_STD_VER >= XY
+ #endif // _LIBCPP_STD_VER >= XY
+ block.
+ """
+ result = ""
+ for element in data:
+ for ftm, entry in element.items():
+ if not entry["implemented"]:
+ # When a FTM is not implemented don't add the guards
+ # or undefine the (possibly) defined macro.
+ result += f'// define {ftm} {entry["value"]}\n'
+ else:
+ need_undef = entry["need_undef"]
+ if entry["condition"]:
+ result += f'# if {entry["condition"]}\n'
+ if entry["need_undef"]:
+ result += f"# undef {ftm}\n"
+ result += f'# define {ftm} {entry["value"]}\n'
+ result += f"# endif\n"
+ else:
+ if entry["need_undef"]:
+ result += f"# undef {ftm}\n"
+ result += f'# define {ftm} {entry["value"]}\n'
+
+ return result
+
+
+def generate_version_header_implementation(data: Dict[str, Dict[str, Any]]) -> str:
+ """Generates the body of the version header."""
+
+ template = """#if _LIBCPP_STD_VER >= {dialect}
+{feature_test_macros}#endif // _LIBCPP_STD_VER >= {dialect}"""
+
+ result = []
+ for std, ftms in data.items():
+ result.append(
+ template.format(
+ dialect=std,
+ feature_test_macros=generate_version_header_dialect_block(ftms),
+ )
+ )
+
+ return "\n\n".join(result)
+
+
class FeatureTestMacros:
"""Provides all feature-test macro (FTM) output components.
@@ -2060,12 +2109,106 @@ def implemented_ftms(self) -> Dict[str, Dict[str, Any]]:
return get_ftms(self.__data, self.std_dialects, True)
+ @functools.cached_property
+ def ftm_meta_data(self) -> Dict[str, Dict[str, Any]]:
+ """Returns the meta data of the FTMs defined in the Standard.
+
+ The meta data does not depend on the C++ dialect used.
+ The result is a dict with the following contents:
+ - key: Name of the feature test macro.
+ - value: A dict with the following content:
+ * headers: The list of headers that should provide the FTM
+ * test_suite_guard: The condtion for testing the FTM in the test suite.
+ * test_suite_guard: The condtion for testing the FTM in the version header.
+ """
+ result = dict()
+ for feature in self.__data:
+ entry = dict()
+ entry["headers"] = feature["headers"]
+ entry["test_suite_guard"] = feature.get("test_suite_guard", None)
+ entry["libcxx_guard"] = feature.get("libcxx_guard", None)
+ result[feature["name"]] = entry
+
+ return result
+
+ @property
+ def version_header_implementation(self) -> Dict[str, List[Dict[str, Any]]]:
+ """Generates the body of the version header."""
+ result = dict()
+ for std in self.std_dialects:
+ result[get_std_number(std)] = list()
+
+ for ftm, values in self.standard_ftms.items():
+ need_undef = False
+ last_value = None
+ for std, value in values.items():
+ # When a newer Standard does not change the value of the macro
+ # there is no need to redefine it with the same value.
+ if last_value and value == last_value:
+ continue
+ last_value = value
+
+ entry = dict()
+ entry["value"] = value
+ entry["implemented"] = self.implemented_ftms[ftm][std] != None
+ entry["need_undef"] = need_undef
+ entry["condition"] = self.ftm_meta_data[ftm]["libcxx_guard"]
+
+ need_undef = entry["implemented"]
+
+ result[get_std_number(std)].append(dict({ftm: entry}))
+
+ return result
+
+ @property
+ def version_header(self) -> str:
+ """Generates the version header."""
+ template = """// -*- C++ -*-
+//===----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP_VERSION
+#define _LIBCPP_VERSION
+
+#include <__config>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+# pragma GCC system_header
+#endif
+
+{feature_test_macros}
+
+#endif // _LIBCPP_VERSION
+"""
+ return template.format(
+ feature_test_macros=generate_version_header_implementation(
+ self.version_header_implementation
+ )
+ )
+
def main():
produce_version_header()
produce_tests()
produce_docs()
+ # Example how to use the new version header generation function to generate
+ # the file.
+ if False:
+ ftm = FeatureTestMacros(
+ os.path.join(
+ source_root, "test", "libcxx", "feature_test_macro", "test_data.json"
+ )
+ )
+ version_header_path = os.path.join(include_path, "version")
+ with open(version_header_path, "w", newline="\n") as f:
+ f.write(ftm.version_header)
+
if __name__ == "__main__":
main()
|
The generator makes a few changes to the output - removes the synopsis, it did not really show what was implemented correctly. - the output now is clang-format clean. This code uses the new FTM data structure. Since the contents of this structure are not up-to-date the code is only used in its tests.
mordante
force-pushed
the
users/mordante/version_header_generator
branch
from
f40e7af
to
a149e39
Compare
banach-space
pushed a commit
to banach-space/llvm-project
that referenced
this pull request
Aug 7, 2024
The generator makes a few changes to the output - removes the synopsis, it did not really show what was implemented correctly. - the output now is clang-format clean. This code uses the new FTM data structure. Since the contents of this structure are not up-to-date the code is only used in its tests.
kstoimenov
pushed a commit
to kstoimenov/llvm-project
that referenced
this pull request
Aug 15, 2024
The generator makes a few changes to the output - removes the synopsis, it did not really show what was implemented correctly. - the output now is clang-format clean. This code uses the new FTM data structure. Since the contents of this structure are not up-to-date the code is only used in its tests.
cpplearner
reviewed
Sep 15, 2024
| # endif | ||
| // define __cpp_lib_format 202110L | ||
| # undef __cpp_lib_variant | ||
| # define __cpp_lib_variant 202106L |
There was a problem hiding this comment.
The output seems incorrect: this value is marked as "implemented": false in test_data.json, and thus it should not be defined in the generated output.
llvm-project/libcxx/test/libcxx/feature_test_macro/test_data.json
Lines 136 to 142 in a149e39
(I noticed this while working on an up-to-date FTM data file.)
There was a problem hiding this comment.
Thanks for noticing! This PR should fix it: #108843
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The generator makes a few changes to the output
This code uses the new FTM data structure. Since the contents of this structure are not up-to-date the code is only used in its tests.