Quality/TestPackages

DRAFT as of 15. Oct. 2012

There are in several places infos about how to package tests in Mer for automated testing. This page tries to consolidate the info and define a standard. Also to point form the different places to this page.

The DRAFT shall be discussed at the next QA irc meeting and get it's approval and this note will be removed.

Work on going

this page is at the moment under construction! so please don't mess with it to much :-) consult kontio in IRC #mer

Pages/Sections which will point to this page

Packaging_guidelines#Test_Packages text will be replaced and point to this page

Quality/Development#Guidelines first part till OBS will be replaced with a pointer to this page

Guidelines

TODO split the packaging part form tests.xml guidelines!

Test package is a RPM package that includes scripts, binaries and/or libraries for testing a feature or function.

• Test package's name must use <packagename>[-*]-tests format
  • Interfixes shall be used if it makes sense to split up the test package
  • The following interfixes shall be used: unit, benchmark, functional, feature

    eg. bluez-tests or bluez-unit-tests

  • if a binary and/or library is used in more than one <packagename>*-tests package, then it shall go to a package named:

    <packagename>-testutils and the <packagename>*-tests packages shall depend on it

• Doesn't have to depend on testrunner-lite
• Test-definition is not required, however recommended since without no automated testing is possible
  • Test package can have not approved or failing test cases

    in that case mark the test case as insignificant:
    <case name="not-so-important-test" insignificant="true">
    see also Test Definition HowTo

  • Test package's test-definition should be installed to /opt/tests/<packagename>/test-definition/
    • if upstream installs it due to historical reasons (e.g. Harmattan) to /usr/share/<packagename>/ no change is needed
• Test package's files should be installed to /opt/tests/<packagename>/
• If test package provides some common test data (audio, video, image) those files should be installed to /opt/tests/<packagename>/{data, audio, video, image, text etc}/,
  use common sense!
• If tests sets require special hardware then the hwiddetect feature of testrunner-lite shall be used, see filtering based on Hardware Identifier

  Mer provides a hwiddetect binary: /usr/bin/hwiddetect add the following to your tests.xml: TODO: the binary does not yet exist, needs to be created! the following list needs to be updated accordingly
  <hwiddetect>/usr/bin/hwiddetect</hwiddetect>
  The program can identify the following hardware: N9, N900, N950, x86-VM (Virtual Machine), RasperryPi TODO: which other hardware adoptions are available for Mer?

• It can be expected that tests are run with eat-device package setup, this means that: TODO: link to eat-device description
  • the tests are run as non-root TODO: what shall be done when root permissions are needed for the tests to run?
  • DISPLAY and DBUS_SESSION_BUS_ADDRESS environment variables are exported
  • do not hard code any user paths, use $HOME> and $XDG_* environment variables instead