Semihosting broken in v1.10

[pavel-kirienko]

I was testing the latest release v1.10 to see if #970 is resolved there. While doing so, I discovered that the semihosting support appears to be broken there. Testing has been conducted with arm-none-eabi GDB versions 10.3 and 12.3 with identical results. The target MCU is STM32F446 connected via SWD. The BMP hardware is the native probe.

First, I was able to successfully load and run ordinary firmware that does not use semihosting. The whole stack operated as intended without noticeable issues.

When I attempted to run a different firmware image that does make use of semihosting, I discovered that some of the semihosting calls were not being executed by the host and that at least some attempts to execute a semihosting call caused the debugger to stall code execution for somewhere between a few hundred ms up to seconds. Shortly after starting code execution, GDB prints the following message and then stops completely:

warning: Invalid remote reply:

Sic! There is nothing after the colon. Pressing Ctrl+C results in "Disconnected from target." being printed.

Enabling remote protocol debugging revealed the following:

(gdb) cont  # At this point the program is in the reset state.
Continuing.
[remote] Sending packet: $c#63
[remote] Received Ack
[remote] wait: enter
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: Funlink,080559D4/11
  [remote] Sending packet: $Hg0#df
  [remote] Received Ack
  [remote] Packet received: OK
  [remote] Sending packet: $m80559d4,11#9e
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $m80559d4,11#9e
  [remote] Received Ack
  [remote] Packet received: <REDACTED, the value is correct>
  [remote] Sending packet: $F-1,2#02
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $F-1,2#02
  [remote] Received Ack
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: Fwrite,00000002,080575DC,00000007
  [remote] Sending packet: $m80575dc,7#a0
  [remote] Received Ack
  [remote] Packet received: 4153534552543A
ASSERT:  [remote] Sending packet: $F7#7d
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $F7#7d
  [remote] Received Ack
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: Fwrite,00000002,08054A70,00000029
  [remote] Sending packet: $m8054a70,29#9d
  [remote] Received Ack
  [remote] Packet received: <REDACTED, the value is correct>
[remote] Sending packet: $F29#b1
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $F29#b1
  [remote] Received Ack
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: Fwrite,00000002,08054F20,00000001
  [remote] Sending packet: $m8054f20,1#63
  [remote] Received Ack
  [remote] Packet received: 3A
[remote] Sending packet: $F1#77
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $F1#77
  [remote] Received Ack
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: Fwrite,00000002,20007795,00000003
  [remote] Sending packet: $m20007795,3#6a
  [remote] Received Ack
  [remote] Packet received: 323234
[remote] Sending packet: $F3#79
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $F3#79
  [remote] Received Ack
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: Fwrite,00000002,080574FC,00000002
  [remote] Sending packet: $m80574fc,2#9c
  [remote] Received Ack
  [remote] Packet received: 0D0A

  [remote] Sending packet: $F2#78
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] putpkt_binary: Packet instead of Ack, ignoring it
  [remote] Sending packet: $F2#78
  [remote] Received Ack
[remote] wait: exit
[remote] wait: enter
  [remote] Packet received: 
warning: Invalid remote reply: 
[remote] wait: exit
[remote] pass_ctrlc: enter
  [remote] interrupt: enter
  [remote] interrupt: exit
[remote] pass_ctrlc: exit
[remote] pass_ctrlc: enter
[remote] pass_ctrlc: exit
Disconnected from target.

For comparison, here is the log captured with BMP firmware v1.9.1 with the same firmare using GDB v10.3 (not possible to test it with GDB v12.3 because of #929), where everything is functioning correctly:

(gdb) cont
Continuing.
Sending packet: $c#63...Ack
Packet received: Funlink,080559D4/11
Sending packet: $Hg0#df...Ack
Packet received: OK
Sending packet: $m80559d4,11#9e...Ack
Packet received: <REDACTED>
Sending packet: $F-1,2#02...Ack
Packet received: Fwrite,00000002,200077F4,00000017
Sending packet: $m200077f4,17#cb...Ack
Packet received: <REDACTED>
Started at 0.000088766
Sending packet: $F17#ae...Ack
Packet received: Fopen,08055A7C/14,00000602,000001A4
Sending packet: $m8055a7c,14#cb...Ack
Packet received: <REDACTED>
Sending packet: $F3#79...Ack
Packet received: Fwrite,00000003,08055A90,00000211
Sending packet: $m8055a90,200#c7...Ack

The connection sequence I used is as follows:

tar ext <PORT>
monitor connect_rst enable
monitor auto_scan
attach 1
load

[ALTracer]

Confirming there is a bug in BMF at current main, but not BMDA. I've been investigating this for the last few days and my findings are inconclusive. I managed to enable logging in blackpill-f411ce platform thanks to its big internal flash and uncrippled ENABLE_DEBUG=1 build setting (skips rdimon entirely), and observed all syscalls with arguments on debug_serial. What's worse, syscalls appeared in pairs with hiccups, leading me to believe that some syscalls or packets get skipped or lost.

I tested BMDA semihosting on Linux and Windows 10 via JLink, but also recent BMF, and that behaves properly due to its different implementation -- it does not rely on hostio code, which I at the moment assume is suspect. I used the sketch and library for semihosting.

[ALTracer]

@pavel-kirienko Please revisit after building fresh firmware -- there should be no more missed calls and 2-second timeouts.
I'm not sure which gdb version started requiring fake threads, and which BMP branch fakes them as such, but semihosting behaviour shouldn't change much.