Device tree generation failed #20

Chert-Nik · 2019-12-13T16:41:11Z

A recent QDT launch ended with following message about device tree creation:

Creating Device Tree for riscv64...
Device Tree for riscv64 isn't created: [Errno 2] No such file or directory: '/usr/local/bin/qemu-system-riscv64'
Making known targets set...
Known targets set was made
Build QEMU Git graph ...
QEMU Git graph was built
Propagation params in graph of commit's description ...
Params in graph of commit's description were propagated

Right now i have following folder organization for my additional qemu repository:

/home/user/qemu_top/qemu_build    //qemu build dirrectory
/home/user/qemu_top/qemu_src      //qemu git repository
/home/user/qemu_top/qdt           //qdt git repository

QDT did not build the device tree for riscv64 because it was looking for qemu-system-riscv64 in '/usr/local/bin/'.
May be it should also look for it in build directory.
(right now it is in the /home/user/qemu_top/qemu_build/riscv64-softmmu/qemu-system-riscv64)

QDT version: 1557a7e

The text was updated successfully, but these errors were encountered:

laerreal · 2019-12-13T18:50:10Z

#21 must fix it up.

Chert-Nik · 2019-12-25T20:22:45Z

Checked last commit in "qtree_from_build" branch.
SHA: a58ade1
First of all the output cluttered with messages of the following form:
Task _build_inclusions consumed 0.070226 sec during iteration between lines /home/chert/tools/qemu/qdt/source/model.py:750 and 750

In my case, machine creation can fail (one device return error in "realize" function via errp argument).
Because of this failure qdt can't build device tree.

Qdt output looks like this:

Creating Device Tree for riscv64...
Task co_init_device_tree consumed 0.136780 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/version_description.py:830 and 830
./build/riscv64-softmmu/qemu-system-riscv64 does not contain .debug_pubtypes section. Provide -gpubnames flag to the compiler
Process ./build/riscv64-softmmu/qemu-system-riscv64 created; pid = 30757
Listening on port 4321
Task co_update_device_tree consumed 61.694818 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py:122 and 140
Remote debugging from host 127.0.0.1
br 0x000000000029e35d (vl.c:3037), handler = on_main
br 0x000000000052db6e (object.c:349), handler = on_type_initialize
br 0x000000000052dbe2 (object.c:349), handler = on_type_initialize
br 0x000000000052ce4d (object.c:137), handler = on_type_register_internal
br 0x000000000052ce53 (object.c:137), handler = on_type_register_internal
br 0x000000000052ce6c (object.c:137), handler = on_type_register_internal
br 0x000000000052def0 (object.c:137), handler = on_type_register_internal
Task co_update_device_tree consumed 1.302885 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py:140 and 147

...
//Some messages during machine intialisation
...

qemu-system-riscv64: Initialization of device uart failed: SceMiParametersNew: (null) | id 7

Child exited with status 1
Device Tree for riscv64 isn't created:

Failure for binary '/usr/local/bin/qemu-system-riscv64':

In coroutine "co_update_device_tree" (Coroutine based task without description):
Traceback (most recent call last):
  File "/home/chert/tools/qemu/qdt/common/co_dispatcher.py", line 166, in poll
    ret = next(generator)
  File "/home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py", line 123, in co_update_device_tree
    dic = create_dwarf_cache(qemu_exec)
  File "/home/chert/tools/qemu/qdt/debug/dic.py", line 254, in create_dwarf_cache
    elf = InMemoryELFFile(exec_file)
  File "/home/chert/tools/qemu/qdt/debug/elf.py", line 31, in __init__
    with open(file_name, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/qemu-system-riscv64'

BDanAnd · 2019-12-26T16:33:10Z

Firstly about: Task _build_inclusions consumed 0.070226 sec during iteration between lines /home/chert/tools/qemu/qdt/source/model.py:750 and 750

This is a normal warning. GUI and useful work occur in one thread. The work is divided into small pieces which should be fast enough the user will not noticed lags. This message informs that the work took a little longer than the current limit for each piece (50 ms). You can just ignore it.

I think it’s a bad idea to build cache for nonstable QEMU version. You should set stable target QEMU version (GUI: File - Select Qemu target version) on which your work is based.

Please configure QEMU with --extra-cflags=-gpubnames and --disable-pie flags for more successful device tree creating.

laerreal · 2019-12-29T20:19:30Z

--extra-cflags="-gpubnames -no-pie" for better assurance

Chert-Nik · 2019-12-29T22:22:24Z

It still fails to build riscv device tree.
QDT commit: a58ade1
Qemu commit: 131b9a05705636086699df15d4a6d328bb2585e8

Qemu configuration command (executed from the directory "build"):
../src/configure --extra-cflags="-gpubnames -no-pie" --target-list=riscv64-softmmu

Error output:

Creating Device Tree for riscv64...
Task co_init_device_tree consumed 0.066251 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/version_description.py:830 and 830
Device Tree for riscv64 isn't created:

Failure for binary '/usr/local/bin/qemu-system-riscv64':

In coroutine "co_update_device_tree" (Coroutine based task without description):
Traceback (most recent call last):
  File "/home/chert/tools/qemu/qdt/common/co_dispatcher.py", line 166, in poll
    ret = next(generator)
  File "/home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py", line 123, in co_update_device_tree
    dic = create_dwarf_cache(qemu_exec)
  File "/home/chert/tools/qemu/qdt/debug/dic.py", line 254, in create_dwarf_cache
    elf = InMemoryELFFile(exec_file)
  File "/home/chert/tools/qemu/qdt/debug/elf.py", line 31, in __init__
    with open(file_name, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/qemu-system-riscv64'

Failure for binary './build/riscv64-softmmu/qemu-system-riscv64':

In coroutine "co_update_device_tree" (Coroutine based task without description):
Traceback (most recent call last):
  File "/home/chert/tools/qemu/qdt/common/co_dispatcher.py", line 166, in poll
    ret = next(generator)
  File "/home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py", line 123, in co_update_device_tree
    dic = create_dwarf_cache(qemu_exec)
  File "/home/chert/tools/qemu/qdt/debug/dic.py", line 254, in create_dwarf_cache
    elf = InMemoryELFFile(exec_file)
  File "/home/chert/tools/qemu/qdt/debug/elf.py", line 31, in __init__
    with open(file_name, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: './build/riscv64-softmmu/qemu-system-riscv64'

laerreal · 2019-12-30T08:25:52Z

Did you build qemu? I.e., is ./build/riscv64-softmmu/qemu-system-riscv64 actually exist?

Chert-Nik · 2020-01-09T09:17:53Z

It fails in both cases.
If the qemu is not built, then the output is the same as in the previous message.

If the qemu is built, then qemu binary is executed, and it falls in qemu monitor and waits there forever.
If i manually close the qemu application, device tree creation fails.
Qdt output:

Creating Device Tree for riscv64...
Task co_init_device_tree consumed 0.123273 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/version_description.py:830 and 830
Task co_update_device_tree consumed 51.279088 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py:122 and 140
Process ./build/riscv64-softmmu/qemu-system-riscv64 created; pid = 15694
Listening on port 4321
Remote debugging from host 127.0.0.1
br 0x000000000029de5d (vl.c:3037), handler = on_main
br 0x000000000052c8ee (object.c:349), handler = on_type_initialize
br 0x000000000052c962 (object.c:349), handler = on_type_initialize
br 0x000000000052bbcd (object.c:137), handler = on_type_register_internal
br 0x000000000052bbd3 (object.c:137), handler = on_type_register_internal
br 0x000000000052bbec (object.c:137), handler = on_type_register_internal
br 0x000000000052cc70 (object.c:137), handler = on_type_register_internal
Task co_update_device_tree consumed 1.403399 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py:140 and 147

At this point, the qemu application is running and waiting in monitor mode.
After i manually close it i get the following error message:

Child exited with status 0
Device Tree for riscv64 isn't created:

Failure for binary '/usr/local/bin/qemu-system-riscv64':

In coroutine "co_update_device_tree" (Coroutine based task without description):
Traceback (most recent call last):
  File "/home/chert/tools/qemu/qdt/common/co_dispatcher.py", line 166, in poll
    ret = next(generator)
  File "/home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py", line 123, in co_update_device_tree
    dic = create_dwarf_cache(qemu_exec)
  File "/home/chert/tools/qemu/qdt/debug/dic.py", line 254, in create_dwarf_cache
    elf = InMemoryELFFile(exec_file)
  File "/home/chert/tools/qemu/qdt/debug/elf.py", line 31, in __init__
    with open(file_name, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/qemu-system-riscv64'

Failure for binary './build/riscv64-softmmu/qemu-system-riscv64':

In coroutine "co_update_device_tree" (Coroutine based task without description):
Traceback (most recent call last):
  File "/home/chert/tools/qemu/qdt/common/co_dispatcher.py", line 166, in poll
    ret = next(generator)
  File "/home/chert/tools/qemu/qdt/qemu/qom_hierarchy.py", line 153, in co_update_device_tree
    '"--extra-cflags=-no-pie" and/or "--disable-pie" to `configure`?'
RuntimeError: No "device" QOM subtree. Did you forget to pass "--extra-cflags=-no-pie" and/or "--disable-pie" to `configure`?

Making known targets set...
Task co_gen_known_targets consumed 0.054718 sec during iteration between lines /home/chert/tools/qemu/qdt/qemu/version_description.py:780 and 785
Known targets set was made
Build QEMU Git graph ..

This line in error message looks suspicious:
Failure for binary '/usr/local/bin/qemu-system-riscv64':

laerreal · 2020-01-13T14:23:50Z

Your compiler duplicated code of some functions. I think, it is not the cause of the problem. But, we never faced this before.
What flags are passed to the compiler? Is it a debug build?

br 0x000000000052c8ee (object.c:349), handler = on_type_initialize
br 0x000000000052c962 (object.c:349), handler = on_type_initialize
br 0x000000000052bbcd (object.c:137), handler = on_type_register_internal
br 0x000000000052bbd3 (object.c:137), handler = on_type_register_internal
br 0x000000000052bbec (object.c:137), handler = on_type_register_internal
br 0x000000000052cc70 (object.c:137), handler = on_type_register_internal

BDanAnd · 2020-01-13T14:49:13Z

Firstly please configure with --prefix=path flag to ensure that a fresh build is called.
Secondly please try to build same version of qemu which is indicated in the script field target_version.

laerreal · 2020-01-14T17:05:04Z

All required breakpoints are set but not reached. Possible causes are:

PIE (i.e. addresses are not correspond to actual code locations)
Qemu binary ./build/riscv64-softmmu/qemu-system-riscv64 is for different version (git SHA1) then current HEAD.

Breakpoint positions (file:line) are defined relative to specific version (Git SHA1).
QDT reads current HEAD from the repository and computes deltas for line numbers
according to insertions/deletions above each breakpoint position.
Then it uses DWARF from the binary to get addresses for given positions.
If you checked out different SHA1 (HEAD was moved) without rebuilding of the binary,
new breakpoint positions may correspond to different addresses.

Failure for binary '/usr/local/bin/qemu-system-riscv64' is not suspicious. QDT just trying to
use binaries from several locations.
If all attempts have failed, then it outputs causes of all failures.

laerreal · 2020-01-14T17:41:53Z

I used such flags to configure:

--prefix=/home/real/work/qemu/debug
--enable-debug
--target-list=riscv64-softmmu
--enable-vte
--disable-sdl
--disable-pie
--extra-cflags="-Wno-error -no-pie"

laerreal · 2020-01-14T17:49:36Z

There is yet another cause.
You can set target Qemu version for QDT.
In this case QDT ignores HEAD of the Qemu repository and assumes target version you set.
The binary must be built for that target version.

This is a bug.
Actually, there is no known way to get Git version a Qemu binary was built from.
--version CLI key output (i.e. vX.Y.Z) is not accurate enough.

Chert-Nik · 2020-01-15T00:53:25Z

After spending 5 hours rerunning qdt again and again (The definition of insanity meme) with different qemu configarition options i get the following list of configurations, which leads to successfull device tree build:

../src/configure --target-list=riscv64-softmmu --prefix=/home/chert/tools/qemu/debug/ --enable-debug --target-list=riscv64-softmmu --disable-sdl --disable-pie --extra-cflags="-gpubnames -Wno-error -no-pie"

../src/configure --target-list=riscv64-softmmu --enable-debug --target-list=riscv64-softmmu --disable-sdl --disable-pie --extra-cflags="-gpubnames -Wno-error -no-pie"

../src/configure --target-list=riscv64-softmmu --enable-debug --target-list=riscv64-softmmu --disable-pie --extra-cflags="-gpubnames -Wno-error -no-pie"

../src/configure --target-list=riscv64-softmmu --enable-debug --target-list=riscv64-softmmu --disable-pie --extra-cflags="-gpubnames"

Last one gives a warning: "./build/riscv64-softmmu/qemu-system-riscv64 does not contain .debug_pubtypes section. Provide -gpubnames flag to the compiler"
But device tree is built successfully.

../src/configure --target-list=riscv64-softmmu --enable-debug --target-list=riscv64-softmmu --disable-pie

There are also a few configurations (but not all of those that I tried) that lead to tree generation failure:

../src/configure --target-list=riscv64-softmmu --target-list=riscv64-softmmu --disable-sdl --disable-pie --extra-cflags="-gpubnames -Wno-error -no-pie"
../src/configure --target-list=riscv64-softmmu --target-list=riscv64-softmmu --disable-pie --extra-cflags="-gpubnames -Wno-error -no-pie"
../src/configure --target-list=riscv64-softmmu --enable-debug --target-list=riscv64-softmmu
../src/configure --target-list=riscv64-softmmu --target-list=riscv64-softmmu

In all of the above attempts, I built the qemu after configuration with simple make command without additional arguments.
I also did not execute the make install command.
After successfully building the qemu, I launched the qdt with the command:
python3 qdt/qemu_device_creator.py -b ./build basic-device.py
Where basic-device.py looks as simple as:

uart_desc = SysBusDeviceDescription(
    # name =
    "UART_try",
    # directory =
    "char"
)
p = QProject([uart_desc])

In all successful and unsuccessful attempts, the device boilerplate was added to the qemu source files.
As a compiler i use default ubuntu 16.04 gcc (yes it is quite outdated)
gcc (Ubuntu 5.5.0-12ubuntu1~16.04) 5.5.0 20171010

This was referenced Jan 15, 2020

Output cluttered with messages "Task <name> consumed ..." #24

Closed

QDT temporary files left in tmp directory #26

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Device tree generation failed #20

Device tree generation failed #20

Chert-Nik commented Dec 13, 2019

laerreal commented Dec 13, 2019

Chert-Nik commented Dec 25, 2019 •

edited

Loading

BDanAnd commented Dec 26, 2019

laerreal commented Dec 29, 2019

Chert-Nik commented Dec 29, 2019

laerreal commented Dec 30, 2019 •

edited

Loading

Chert-Nik commented Jan 9, 2020 •

edited

Loading

laerreal commented Jan 13, 2020 •

edited

Loading

BDanAnd commented Jan 13, 2020

laerreal commented Jan 14, 2020 •

edited

Loading

laerreal commented Jan 14, 2020

laerreal commented Jan 14, 2020

Chert-Nik commented Jan 15, 2020

Device tree generation failed #20

Device tree generation failed #20

Comments

Chert-Nik commented Dec 13, 2019

laerreal commented Dec 13, 2019

Chert-Nik commented Dec 25, 2019 • edited Loading

BDanAnd commented Dec 26, 2019

laerreal commented Dec 29, 2019

Chert-Nik commented Dec 29, 2019

laerreal commented Dec 30, 2019 • edited Loading

Chert-Nik commented Jan 9, 2020 • edited Loading

laerreal commented Jan 13, 2020 • edited Loading

BDanAnd commented Jan 13, 2020

laerreal commented Jan 14, 2020 • edited Loading

laerreal commented Jan 14, 2020

laerreal commented Jan 14, 2020

Chert-Nik commented Jan 15, 2020

Chert-Nik commented Dec 25, 2019 •

edited

Loading

laerreal commented Dec 30, 2019 •

edited

Loading

Chert-Nik commented Jan 9, 2020 •

edited

Loading

laerreal commented Jan 13, 2020 •

edited

Loading

laerreal commented Jan 14, 2020 •

edited

Loading