Simpleperf error when more than 6 events

[ghost]

Hi, so I'm using the simpleperf stat command on WA, but when I have 7 events, there seems to be an error in the parsing stage. Only occurs when --csv is in the option string.

  perf:
    cleanup_assets: true
    command: stat
    events:
    - r11
    - r8
    - r13
    - r12
    - r19
    - r1D
    - r1B
    force_install: false
    labels: ['simpleperf']
    optionstring: '-a --interval 10 --per-core --interval-only-values --use-devfreq-counters --csv'
    perf_type: simpleperf
    report_option_string:
    report_sample_options:
    run_report_sample: false

Here's the return error:

2023-09-15 15:26:08,535 ERROR    instrument:         Error in instrument perf
2023-09-15 15:26:08,535 DEBUG        signal:         Sending error-logged from <ErrorSignalHandler (DEBUG)>
2023-09-15 15:26:08,535 ERROR    instrument:           File "/home/hsc/miniconda3/lib/python3.11/site-packages/wlauto-3.4.0.dev1+05eab42f-py3.11.egg/wa/framework/instrument.py", line 272, in __call__
2023-09-15 15:26:08,535 ERROR    instrument:             self.callback(context)
2023-09-15 15:26:08,535 ERROR    instrument:           File "/home/hsc/miniconda3/lib/python3.11/site-packages/wlauto-3.4.0.dev1+05eab42f-py3.11.egg/wa/instruments/perf.py", line 160, in update_output
2023-09-15 15:26:08,535 ERROR    instrument:             self._process_simpleperf_output(context)
2023-09-15 15:26:08,535 ERROR    instrument:           File "/home/hsc/miniconda3/lib/python3.11/site-packages/wlauto-3.4.0.dev1+05eab42f-py3.11.egg/wa/instruments/perf.py", line 173, in _process_simpleperf_output
2023-09-15 15:26:08,535 ERROR    instrument:             self._process_simpleperf_stat_output(context)
2023-09-15 15:26:08,535 ERROR    instrument:           File "/home/hsc/miniconda3/lib/python3.11/site-packages/wlauto-3.4.0.dev1+05eab42f-py3.11.egg/wa/instruments/perf.py", line 251, in _process_simpleperf_stat_output
2023-09-15 15:26:08,535 ERROR    instrument:             self._process_simpleperf_stat_from_csv(stat_file, context, label)
2023-09-15 15:26:08,535 ERROR    instrument:           File "/home/hsc/miniconda3/lib/python3.11/site-packages/wlauto-3.4.0.dev1+05eab42f-py3.11.egg/wa/instruments/perf.py", line 265, in _process_simpleperf_stat_from_csv
2023-09-15 15:26:08,535 ERROR    instrument:             context.add_metric('{}_{}'.format(label, row[1]), row[0], 'count', classifiers=classifiers)
2023-09-15 15:26:08,535 ERROR    instrument:                                                      ~~~^^^
2023-09-15 15:26:08,535 ERROR    instrument:         
2023-09-15 15:26:08,535 DEBUG        signal:         Sending error-logged from <ErrorSignalHandler (DEBUG)>
2023-09-15 15:26:08,535 ERROR    instrument:         IndexError(list index out of range)
2023-09-15 15:26:08,536 INFO      processor:         Processing using "csv"

Couldn't quite figure out what the issue is. The moment I only have 6 events, it works just fine.
Thanks!

[marcbonnici]

Hi,

Thanks for reporting this.

I suspect this means that your device only has 6 hardware counters available. When enabling more events than counters available I'm seeing additional messages in the raw simpleperf.out file. E.g.

simpleperf W cmd_stat.cpp:865] Percentages < 100% means some events only run a subset of enabled time,
probably caused by hardware counter multiplexing (less counters than events).
Try --use-devfreq-counters if on a rooted device.

These messages are causing the parsing to fail so handling this scenario need to be fixed, however to prevent reduced accuracy I'd suggest limiting the number of events enabled if possible.

[hosunhc]

Hmmmm, I think my device does allow 6 counters at the same time + the cycle counter which is 0x11 and can always be measured outside of the six. Is there any way I can just force the parse despite the limit? Or is that something that needs to be figured out on the simpleperf binary end?

[marcbonnici]

Although simpleperf can handle this, the main issue with the parsing is that these error messages are being printed in the middle of the output data which is the part of the reason the parsing is falling over.
We need to investigate if we can prevent these additional prints or remove them from the output file properly.

As a quick workaround we can try skipping the corrupted lines but this will mean that some entries will be dropped so we'll need to find a proper solution to this. In case it helps you I have pushed the workaround to my branch which can be found here [1]

[1] marcbonnici@fc620f8