The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)
Talk:Quality/Test processes
From Mer Wiki
(Difference between revisions)
| Line 12: | Line 12: | ||
* Metadata files are stored in version control | * Metadata files are stored in version control | ||
** Can be automatically updated to a database | ** Can be automatically updated to a database | ||
| − | |||
* Testing changed packages first, then integration testing? | * Testing changed packages first, then integration testing? | ||
| + | |||
| + | This requires lots of changes and work. We could use the current tools for simple testing and start developing the new process on the side. | ||
Revision as of 17:56, 23 April 2012
New test process
Some notes from #mer channel at 23.4.2012 from 12:34 onwards:
- Separating test-definition from the test packaging
- New test-definition/metadata defines test case
- Same information than in the current test-definition
- Architectural domain and subdomain/feature
- HW (chroot or device or both)
- Defines requirements, eg. needs 3G network or test data
- Defines provides, eg. provides 3G network
- Defines dependencies to packages 0-*
- Same information than in the current test-definition
- Metadata files are stored in version control
- Can be automatically updated to a database
- Testing changed packages first, then integration testing?
This requires lots of changes and work. We could use the current tools for simple testing and start developing the new process on the side.
Test mapping / Test plan configuration
- Test request comes in. The request has parameters such as stage and "changed packages" list
- Testplanner reads the configuration/mapping file and creates test plan(s)
- Kickstart files are delivered to image builder and test plans for test automation
Mapping/configuration file:
name: Release testing
devices:
- chroot
ots-device: mer-chroot
kickstart: chroot-release.ks
- virtual
ots-device: mer-virtual-i686
kickstart: virtual-image-core.ks
- n900
ots-device: mer-n900
kickstart: nemo-n900.ks
default options:
- chroot:
filter: "hardware=chroot"
- all:
test package suffix: "-tests"
default packages:
- chroot:
- mer-core-chroot-tests
- all:
- mer-core-sanity-tests
packages:
- alc:
filter: "name=unittests"
versio: < 1.5.0
device: [virtual n900]
- bluez
- qt5base:
test packages:
- mwts-location-tests
filter: "type=functional"
versio: 2.3.1
device: [virtual n900]
- qt5base-unittests
Description:
- devices: a list of devices for the tests are executed
- ots-device: device name in the OTS (test automation)
- kickstart: mapping kickstart files to devices
- default options: default options for devices
- filter: filter options to testrunner-lite
- test package suffix: if test package(s) are not defined for a package, then trying to use "original package name + suffix" test package. See the bluez package below.
- default packages: a list of test packages that are always executed
- packages: the actual package mapping
- filter: filter options to testrunner-lite
- version: package version to use
- device: on which devices to execute the test
- test packages: list of test packages and their configurations
