west: build: also parse common section in yaml file

[nashif]

Parse common section and append to cmake args if available when using
--test-item option of west build.

Signed-off-by: Anas Nashif [email protected]

[PerMac]

Still doesn't work like twister.

• this PR only looks at extra_conf and extra_args. In the PR which this one is a follow up I pointed, that at least sysbuild: true is not propagated and causing failures in west west_commands: Fix parsing of extra args for test-item #60439 (comment). This is not addressed in this PR.

• adding extra_conf in common causes error in twister (twisterlib.error.TwisterRuntimeError: No test cases found at the specified location...) but now works in west. It seems twister is pullin out only from extra_args from common and from those extracting only {"CONF_FILE", "OVERLAY_CONFIG", "DTC_OVERLAY_FILE"}, leaving the rest. Then certain items ("extra_conf_files", "extra_overlay_confs", "extra_dtc_overlay_files") in test section have their own special rules: they can be expanded or replaced depending on their type https://github.com/zephyrproject-rtos/zephyr/blob/main/scripts/pylib/twister/twisterlib/config_parser.py#L200C25-L200C25 I don't see this reproduced in this PR

• In twister conf files are compiled into a single list, in a certain order. Now west is using different order than twister. This can be seen by calling
  west build -v --test-item tests/drivers/build_all/adc/drivers.adc.build -b native_posix --pristine always
  and scripts/twister -s tests/drivers/build_all/adc/drivers.adc.build -p native_posix -v -v. West fails since it will do

/home/maciej/.pyenv/shims/cmake -DWEST_PYTHON=/home/maciej/.pyenv/versions/zephyr38/bin/python -B/home/maciej/zephyrproject/zephyr/build -GNinja -DBOARD=native_posix -DCONFIG_GPIO=y -DCONFIG_GPIO=n -S/home/maciej/zephyrproject/zephyr/tests/drivers/build_all/adc

and twister passes with

/home/maciej/.pyenv/shims/cmake -B/home/maciej/zephyrproject/zephyr/twister-out/native_posix/tests/drivers/build_all/adc/drivers.adc.build -DTC_RUNID=1e7c69b0a6f05bd3c283752e213dc339 -DCONFIG_COMPILER_WARNINGS_AS_ERRORS=y -DEXTRA_GEN_DEFINES_ARGS=--edtlib-Werror -GNinja -S/home/maciej/zephyrproject/zephyr/tests/drivers/build_all/adc -DCONFIG_GPIO=n -DCONFIG_GPIO=y -DBOARD=native_posix

note -DCONFIG_GPIO=y -DCONFIG_GPIO=n vs -DCONFIG_GPIO=n -DCONFIG_GPIO=y

This just shows how tricky it is to get the same with west and twister now, when each tool implements it's own logic for building. I know that the issues I'm poiting can be resolved by furtherly refining logic on the west side. But IMO it will be counter productive. As long as there are two implementations of not-that-trivial logic there will be space for bugs poping up and debugging will be like playing "spot the difference" #60502.

[nashif]

• adding extra_conf in common causes error in twister (twisterlib.error.TwisterRuntimeError: No test cases found at the specified location...) but now works in west.

Does not tell me much, you need to provide more details, how is this related to this change? I tested extra_configs in common section, and it works as expected.

• common sections are supported now
• sysbuild also supported in common section
• item takes priority over common section

[PerMac]

It was just pointing the difference, sorry I was not precise enough with the description and might even not discribed it correctly. Adding extra_configs in yaml's common section is not supported in twister. The schema for testcase.yaml doesn't include it and twister fails on such cases.

ERROR   - /home/maciej/zephyrproject/zephyr/tests/kernel/common: can't load (skipping): SchemaError(msg='Schema validation failed:
 - Key 'extra_configs' was not defined. Path: '/common'.')

I guess it might be just an omission in twister and not a real feature and I see no real risk related to such difference between twister and west.

[nashif]

also added support for extra_conf_files, extra_overlay_confs, extra_dtc_overlay_files.
The code can be optimized a bit, but it does the job.