The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)
Talk:Quality/Test processes
From Mer Wiki
(Difference between revisions)
(→Test selection) |
|||
| (2 intermediate revisions by one user not shown) | |||
| Line 1: | Line 1: | ||
| − | + | = New test process = | |
| − | Some notes from #mer channel at [http://mer.bfst.de/logs/%23mer/%23mer.2012-04-23.log.html#t2012-04-23T12:34:18 23.4.2012] from 12:34 onwards: | + | == Test definition == |
| − | * Separating test | + | Some notes from #mer channel at [http://mer.bfst.de/logs/%23mer/%23mer.2012-04-23.log.html#t2012-04-23T12:34:18 23.4.2012] from 12:34 onwards and from the QA meeting [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-04-26-12.00.html 2012-04-26]. |
| − | * New test | + | |
| + | * Separating test definition from the test packaging | ||
| + | * New test definition defines test cases and test sets | ||
| + | ** Test's steps are in own file, makes test management easier | ||
** Same information than in the current test-definition | ** Same information than in the current test-definition | ||
*** Architectural domain and subdomain/feature | *** Architectural domain and subdomain/feature | ||
| Line 10: | Line 13: | ||
** Defines provides, eg. provides 3G network | ** Defines provides, eg. provides 3G network | ||
** Defines dependencies to packages 0-* | ** Defines dependencies to packages 0-* | ||
| − | * | + | * Test definition files are stored in version control |
| − | + | ||
| − | + | ||
<br /> | <br /> | ||
| − | + | * This requires lots of changes to existing tools. We could take the current tools in use before developing them further. | |
| − | + | ||
New test-definition format: | New test-definition format: | ||
| Line 71: | Line 71: | ||
</pre> | </pre> | ||
| − | == Test | + | == Test selection == |
[[Image:test_mapping.png]] | [[Image:test_mapping.png]] | ||
| − | # | + | # New test request comes in. The request has parameters such as stage and "changed packages" list |
| − | # | + | # Get from the stage's test selection file devices and test sets/cases |
| + | # Get from test definitions requirements, packages and dependencies for test sets/cases | ||
# Kickstart files are delivered to image builder and test plans for test automation | # Kickstart files are delivered to image builder and test plans for test automation | ||
| − | + | Test selection: | |
<pre> | <pre> | ||
name: Release testing | name: Release testing | ||
Latest revision as of 05:52, 2 May 2012
[edit] New test process
[edit] Test definition
Some notes from #mer channel at 23.4.2012 from 12:34 onwards and from the QA meeting 2012-04-26.
- Separating test definition from the test packaging
- New test definition defines test cases and test sets
- Test's steps are in own file, makes test management easier
- Same information than in the current test-definition
- Architectural domain and subdomain/feature
- HW (chroot or device or both)
- Defines requirements, eg. needs 3G network or test data
- Defines provides, eg. provides 3G network
- Defines dependencies to packages 0-*
- Test definition files are stored in version control
- This requires lots of changes to existing tools. We could take the current tools in use before developing them further.
New test-definition format:
version: 1.0
name: bluetooth tests
domain: communications
feature: bluetooth
test cases:
- name: &check_interface
timeout: 60
test type: functional
dependencies: [bluez]
environment: hardware
- name: &create_l2cap_connection
timeout: 120
test type: functional
dependencies: [bluez]
environment: hardware
provides: bluetooth-connection-l2cap
- name: &close_l2cap_connection
timeout: 120
test type: functional
dependencies: [bluez]
environment: hardware
provides: bluetooth-connection-l2cap-close
- name: &transfer_1Mib_file
timeout: 600
test type: functional
dependencies: [bluez, mer-testdata-files]
environment: hardware
requires-before: bluetooth-connection-l2cap
requires-after: bluetooth-connection-l2cap-close
- name: &transfer_10Mib_file
timeout: 600
test type: functional
dependencies: [bluez, mer-testdata-files]
environment: hardware
requires-before: bluetooth-connection-l2cap
requires-after: bluetooth-connection-l2cap-close
test sets:
- name: sanity testing
- test: &check_interface
- test: &create_l2cap_connection
- test: &close_l2cap_connection
- test: &transfer_1Mib_file
[edit] Test selection
- New test request comes in. The request has parameters such as stage and "changed packages" list
- Get from the stage's test selection file devices and test sets/cases
- Get from test definitions requirements, packages and dependencies for test sets/cases
- Kickstart files are delivered to image builder and test plans for test automation
Test selection:
name: Release testing
devices:
- chroot
ots-device: mer-chroot
kickstart: chroot-release.ks
- virtual
ots-device: mer-virtual-i686
kickstart: virtual-image-core.ks
- n900
ots-device: mer-n900
kickstart: nemo-n900.ks
default options:
- chroot:
filter: "hardware=chroot"
default packages:
- chroot:
- mer-core-chroot-tests
- all:
- mer-core-sanity-tests
packages:
- package: bluez
- set: sanity testing
- test: transfer_10Mib_file
Description:
- devices: a list of devices for the tests are executed
- ots-device: device name in the OTS (test automation)
- kickstart: mapping kickstart files to devices
- default options: default options for devices
- default packages: a list of test packages that are always executed
- packages: the actual package mapping
- version: package version to use
- device: on which devices to execute the test
