GUI testing

fMBT provides utilities for GUI testing:

• Python libraries that implement similar GUI test interface for multiple platforms
  • fmbtandroid for Android phones, tablets and emulator
  • fmbtchromiumos for Chromium OS devices (available in fMBT devel branch at the moment)
  • fmbttizen for Tizen mobile and emulator (X server) and Tizen IVI (Wayland + Weston)
  • fmbtvnc for anything that runs VNC servers
  • fmbtwindows for Windows desktops, laptops and tablets
  • fmbtx11 for X servers
• fmbt-scripter - a tool for
  • capturing reference bitmaps from screenshots from Android, Chromium OS, Tizen, VNC, Windows and X
  • editing and debugging Python GUI test scripts.

GUI test interface

GUI test interfaces for multiple platforms have a lot in common, but also some platform-specific extensions.

While this document gives only an overview, you will get full parameter and return value documentation for each GUI test interfaces with Python:

• python -c 'import fmbtandroid; help(fmbtandroid.Device)'
• python -c 'import fmbtchromiumos; help(fmbtchromiumos.Device)'
• python -c 'import fmbttizen; help(fmbttizen.Device)'
• python -c 'import fmbtvnc; help(fmbtvnc.Screen)'
• python -c 'import fmbtwindows; help(fmbtwindows.Device)'
• python -c 'import fmbtx11; help(fmbtx11.Screen)'

If you want help from a single method, use python -c 'import fmbtandroid; help(fmbtandroid.Device.refreshScreenshot)', for instance.

Receiving screen contents

• refreshScreenshot fetches the latest screenshot from the device under test. <act>Bitmap and <act>OcrText methods (like tapBitmap, swipeOcrText) use most recently fetched screenshot.
• refreshView fetches UI elements currently on the screen. <act>Text methods (like tapText) use most recently fetched UI element information.
• Screenshots are automatically stored in the "screenshots" directory under the working directory, unless something else is specified with setScreenshotDir and setScreenshotSubdir.
• Disk space needed by screenshots can be controlled with setScreenshotLimit and setScreenshotArchiveMethod.
• screenshot().save(filename) saves the latest screenshot to given file.

	android	chromiumos	tizen	vnc	windows	x11
refreshScreenshot	Y	Y	Y	Y	Y	Y
refreshView	Y

Verifying screen contents

• verifyBitmap and verifyOcrText return True if a bitmap or a text is recognized from latest screenshot.
• screenshot().findItemsByBitmap and screenshot().findItemsByOcr return full list of items matching a bitmap or a text. Full documentation on OIR and OCR parameters can be found with help(obj.oirEngine()) and help(obj.ocrEngine()).
• verifyText returns True if a UI element with given text is found from latest UI element information.
• view().findItemsByClass, view().findItemsById, view().findItemsByText return full list of UI elements matching given criteria.
• Items found with findItemsBy... can be acted upon with tapItem and swipeItem.

	android	chromiumos	tizen	vnc	windows	x11
verifyBitmap verifyOcrText	Y	Y	Y	Y	Y	Y
verifyText	Y
screenshot	Y	Y	Y	Y	Y	Y
view	Y
ocrEngine	Y	Y	Y	Y	Y	Y
oirEngine	Y	Y	Y	Y	Y	Y

Sending touch or mouse input

• tap<what> sends touch down and touch up (or mouse button down, mouse button up) events on given location. The location can be given as absolute coordinates or unity coordinates, or it can be based on a bitmap, text recognized by OCR, or a text from UI elements.
• platforms that support both touch and mouse events will get a touch event by default. When called with optional argument button=X, a mouse event with button X is sent. If button=0, then only mouse move is sent, resulting in hovering over given location.

	android	chromiumos	tizen	vnc	windows	x11
drag	touch	mouse	touch mouse	mouse	touch mouse	mouse
swipe, swipeBitmap, swipeOcrText	touch	mouse	touch mouse	mouse	touch mouse	mouse
swipeText	touch
tap, tapBitmap, tapOcrText	touch	mouse	touch mouse	mouse	touch mouse	mouse
tapText	touch

Sending keyboard input

• pressKey sends a key press and release.
• type types given text.
• keyNames returns list of key names supported by pressKey.

	android	chromiumos	tizen	vnc	windows	x11
keyNames	Y			Y	Y
pressKey	Y	Y	Y	Y	Y	Y
type	Y	Y	Y	Y	Y	Y

Executing commands remotely

• shellSOE executes shell command and returns triplet: (Status, Output, Error), that is
  1. child process exit status
  2. what was printed to standard output
  3. what was printed to standard error

	android	chromiumos	tizen	vnc	windows	x11
shellSOE	Y	Y	Y		Y

Logging test runs

• enableVisualLog starts tracing all GUI test interface method calls, return values, exceptions, captured screenshots and reference bitmaps. The log is written to given HTML file during the test run.
• fmbtlogger.text and fmbtlogger.csv log method calls, return values and exceptions of any Python object, including instances of GUI test interfaces. The log format is plain text or CSV. See python -c 'import fmbtlogger; help(fmbtlogger.text)' for more information.

	android	chromiumos	tizen	vnc	windows	x11
enableVisualLog	Y	Y	Y	Y	Y	Y

fmbtandroid

Prerequisites:

• adb from Android SDK (platform tools)
• Enable USB debugging from device settings.

Example:

import fmbtandroid
d = fmbtandroid.Device()
d.swipe((0.5, 0.85), "east") # open lock screen

fmbtchromiumos

Prerequisites:

• Enable ssh login to the Chromium OS device without password.

  Example with Qemu:

  • Launch Qemu with port forwarding, for instance

  ... -net nic,model=virtio -net user,hostfwd=tcp::9222-:22.

  • Press Ctrl-Alt-T in the browser. When in crosh> prompt, enter shell.
  • Copy your ssh public key: scp [email protected] 'cat .ssh/id_rsa.pub' > ~/.ssh/authorized_keys.

Chromium OS in Qemu example:

import fmbtchromiumos
d = fmbtchromiumos.Device("ssh -p 9222 chronos@localhost")
print d.shellSOE("echo Welcome, $(whoami); pwd")[1]

fmbttizen

Prerequisites:

• sdb from Tizen SDK for testing a Tizen mobile device
• ssh for testing Tizen IVI

Tizen mobile example:

import fmbttizen
d = fmbttizen.Device()
d.pressPower()

Tizen IVI example:

import fmbttizen
d = fmbttizen.Device(loginCommand="ssh root@ivi")
d.refreshScreenshot().save("current-screen.png")

fmbtvnc

Prerequisites:

• Install vncdotool: https://github.com/sibson/vncdotool

Example:

import fmbtvnc
d = fmbtvnc.Screen("IP-ADDRESS-OF-VNC-SERVER")
d.tap((0.5, 0.5), button=0) # hover mouse on the middle of the screen

fmbtwindows

Prerequisites for Windows device:

• Install Python 2.X.
• Copy fMBT's pythonshare directory to Windows.
• In the pythonshare directory, run c:\python27\python.exe setup.py install
• Launch pythonshare-server to which fmbtwindows can connect to:

cd \python27\scripts
python pythonshare-server --interface=all --password=xxxxxxxx

Example:

import fmbtwindows
d = fmbtwindows.Device("IP-ADDRESS-OF-THE-DEVICE", password="xxxxxxxx")
d.type("Hello Windows")

fmbtx11

Example:

import fmbtx11
d = fmbtx11.Screen(":2")
d.enableVisualLog("calls.html")
d.refreshScreenshot()