The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)
Talk:Quality/Test processes
From Mer Wiki
(Difference between revisions)
(→Test mapping / Test plan configuration) |
(→Test mapping / Test plan configuration) |
||
| Line 90: | Line 90: | ||
- package: bluez | - package: bluez | ||
tests: | tests: | ||
| + | - check interface | ||
- transfer 1Mib file | - transfer 1Mib file | ||
| Line 100: | Line 101: | ||
** kickstart: mapping kickstart files to devices | ** kickstart: mapping kickstart files to devices | ||
* default options: default options for devices | * default options: default options for devices | ||
| − | |||
| − | |||
* default packages: a list of test packages that are always executed | * default packages: a list of test packages that are always executed | ||
* packages: the actual package mapping | * packages: the actual package mapping | ||
| − | |||
** version: package version to use | ** version: package version to use | ||
** device: on which devices to execute the test | ** device: on which devices to execute the test | ||
| − | |||
Revision as of 17:53, 25 April 2012
New test process
Some notes from #mer channel at 23.4.2012 from 12:34 onwards:
- Separating test-definition from the test packaging
- New test-definition/metadata defines test case
- Same information than in the current test-definition
- Architectural domain and subdomain/feature
- HW (chroot or device or both)
- Defines requirements, eg. needs 3G network or test data
- Defines provides, eg. provides 3G network
- Defines dependencies to packages 0-*
- Same information than in the current test-definition
- Metadata files are stored in version control
- Can be automatically updated to a database
- Testing changed packages first, then integration testing?
- This requires lots of changes to existing tools. We could take the current tools in use before developing them further.
- New testrunner could be written in Python or similar language
New test-definition format:
version: 1.0
name: bluetooth tests
domain: communications
feature: bluetooth
test cases:
- name: check interface
timeout: 60
test type: functional
dependencies: [bluez]
environment: hardware
- name: create l2cap connection
timeout: 120
test type: functional
dependencies: [bluez]
environment: hardware
provides: bluetooth-connection-l2cap
- name: close l2cap connection
timeout: 120
test type: functional
dependencies: [bluez]
environment: hardware
provides: bluetooth-connection-l2cap-close
- name: transfer 1Mib file
timeout: 600
test type: functional
dependencies: [bluez, mer-testdata-files]
environment: hardware
requires-before: bluetooth-connection-l2cap
requires-after: bluetooth-connection-l2cap-close
Test mapping / Test plan configuration
- Test request comes in. The request has parameters such as stage and "changed packages" list
- Testplanner reads the configuration/mapping file and creates test plan(s)
- Kickstart files are delivered to image builder and test plans for test automation
Mapping/configuration file:
name: Release testing
devices:
- chroot
ots-device: mer-chroot
kickstart: chroot-release.ks
- virtual
ots-device: mer-virtual-i686
kickstart: virtual-image-core.ks
- n900
ots-device: mer-n900
kickstart: nemo-n900.ks
default options:
- chroot:
filter: "hardware=chroot"
default packages:
- chroot:
- mer-core-chroot-tests
- all:
- mer-core-sanity-tests
packages:
- package: bluez
tests:
- check interface
- transfer 1Mib file
Description:
- devices: a list of devices for the tests are executed
- ots-device: device name in the OTS (test automation)
- kickstart: mapping kickstart files to devices
- default options: default options for devices
- default packages: a list of test packages that are always executed
- packages: the actual package mapping
- version: package version to use
- device: on which devices to execute the test
