fMBT GUI Test Interface FAQ

Questions on matching bitmaps:

• How can I save reference bitmaps from screenshots?
• Why locating a bitmap fails?
• How can I find suitable OIR parameters for matching a bitmap?
• How can I avoid repeating OIR parameters on every Bitmap method call?
• How to deal with impossible-to-detect regions?
• How can I get coordinates of a bitmap?
• What is the most efficient way to detect which screen the device under test is showing?

How can I save reference bitmaps from screenshots?

fmbt-scripter helps with that. Move the text cursor is on a bitmap filename, such as "window-close.png" on the editor, click the Select button, and select an area from the screenshot on the left.

Why locating a bitmap fails?

Compared screenshot and reference bitmap do not match. For instance, transparency, scaling, antialiasing and slight palette changes are difficult or impossible to see with bare eye, but may cause recognition of the bitmap fail.

Bitmap methods (tapBitmap, ...) require 100 % exact match by default. This can be relaxed by adjusting OIR (optical image recognition) parameters. For instance,

>>> d.tapBitmap("ref-image.png", colorMatch=0.8)

allows 20 % difference on each color channel value on pixel-to-pixel comparisons. Even a bigger threshold may be needed to cope with partial transparency.

How can I find suitable OIR parameters for matching a bitmap?

fMBT's default OIR engine has a method that searches for OIR parameters that enable finding a reference bitmap on a screenshot. Example:

>>> d.oirEngine().adjustParameters(d.screenshot(), "ref-bitmap.png")

You can give adjustParameters a range of colorMatch, bitmapPixelSize, screenshotPixelSize, and scale values from which it should find a working combination. For more information, see

>>> help(d.oirEngine())
>>> help(d.oirEngine().adjustParameters)

How can I avoid repeating OIR parameters on every Bitmap method call?

OIR parameters can be provided in .fmbtoirrc files. Parameters given in the file apply to all reference bitmaps in the same directory. An example of a pretty liberal .fmbtoirrc:

colorMatch = 0.8
bitmapPixelSize = 2
screenshotPixelSize = 2

OIR parameters given in Bitmap method calls in test scripts will override parameters read from the .fmbtoirrc file.

How to deal with impossible-to-detect regions?

Some graphics are hard or impossible to detect because of transparency, for instance. You can still avoid hard coding coordinates to test script, if you use OIR "forced matching results". In that case coordinates are stored to bitmap.png.fmbtoir.loc.

You can create these files with fmbt-scripter by clicking the Select button twice and then selecting the region.

How can I get coordinates of a bitmap?

Coordinates of a bitmap are available through found Item instances. Example:

>>> allItems = d.screenshot().findItemsByBitmap("ref-image.png")
>>> firstItem = allItems[0]
>>> print "(x1, y1, x2, y2) ==", firstItem.bbox()
>>> print "center (x, y) ==", firstItem.coords()

What is the most efficient way to detect which screen the device under test is showing?

waitAnyBitmap(list-of-reference-bitmaps[, waitTime=n][, pollDelay=n]) waits until any of the reference bitmaps appears on the display, or waitTime expires. It returns a list of bitmaps that contains all reference bitmaps that are visible in the latest screenshot.

Example:

>>> d.tapBitmap("window-close.png")
>>> whatISee = d.waitAnyBitmap(["desktop.png", "browser.png", "camera.png"], waitTime=1.0)
>>> if "desktop.png" in whatISee:
...