Matching extracted data with screenshots

[js0nwu]

Hi,

This is a really cool tool for extracting the behavior of UIs. Thank you for releasing the code and the dataset. I was wondering how I could create something similar to Figure 1 in your paper [1], where the extracted metadata is aligned with visual elements on the screenshot. It looks like the released dataset on Zenodo [2] consists of json files that aren’t paired with images.

Thanks in advance

[1] https://arxiv.org/pdf/2105.03144.pdf
[2] https://zenodo.org/record/5084655