The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)

Quality

From Mer Wiki
(Difference between revisions)
Jump to: navigation, search
(→‎Metrics)
m (→‎QA tools)
 
(23 intermediate revisions by one user not shown)
Line 1: Line 1:
 
= Quality =
 
= Quality =
  
−
== Metrics ==
 
  
−
{| border="0" cellspacing="0" cellpadding="5" style="text-align:center;"
 
−
! colspan="6" | '''Burn-down chart:'''
 
−
|-
 
−
| '''Release''' || '''Changed packages''' || '''Added packages''' || '''Fixes''' || '''Bugs/tasks reported*''' || '''Bugs/tasks resolved*'''
 
−
|-
 
−
| 0.20120209.1 || 67 || 0 || 11 || 105 || 39
 
−
|-
 
−
| 0.20120315.1 || 36 || 0 || 9 || 27 || 15
 
−
|-
 
−
| 0.20120405.1 || 13 || 2 || 1 || 36 || 2
 
−
|-
 
−
| 0.20120419.2 || 14 || 4 || 13 || 30 || 12
 
−
|-
 
−
| 0.20120517.1 || 15 || 2 || 18 || 9 || 19
 
−
|-
 
−
| 0.20120614.1 || 24 || 9 || 16 || 38 || 13
 
−
|-
 
−
! Total:
 
−
| 169 || 17 || 68 || 245 || 100
 
−
|-
 
−
|}
 
−
 
−
(*) From Bugzilla, other values are from releases changelogs
 
−
 
−
 
−
{| border="0" cellspacing="0" cellpadding="2"
 
−
! colspan="5" | '''Open bugs and tasks:'''
 
−
|-
 
−
! Product:
 
−
| colspan="4" | Mer Core
 
−
|-
 
−
! Status:
 
−
| colspan="4" | NEEDINFO, UNCONFIRMED, NEW, ASSIGNED, TRIAGEDUPSTREAM, REOPENED
 
−
|-
 
−
| |
 
−
| colspan="2" | '''By priority:'''
 
−
| colspan="2" | '''By severity:'''
 
−
|-
 
−
| || High: || 9 || Critical || 1
 
−
|-
 
−
| || Normal: || 43 || Major || 4
 
−
|-
 
−
| || Low: || 86 || Normal || 37
 
−
|-
 
−
| || Undecided: || 5 || Trivial || 4
 
−
|-
 
−
| colspan="3" | || Echancement || 1
 
−
|-
 
−
| colspan="3" | || Task || 96
 
−
|-
 
−
! Total:
 
−
| colspan="4" | 143 (47 bugs and 96 tasks)
 
−
|-
 
−
|}
 
−
 
−
 
−
 
−
 
−
{| border="0" cellspacing="0" cellpadding="2"
 
−
! colspan="5" | '''Closed bugs:'''
 
−
|-
 
−
! Product:
 
−
| colspan="4" | Mer Core
 
−
|-
 
−
! Status:
 
−
| colspan="4" | RESOLVED, RELEASED, VERIFIED, CLOSED
 
−
|-
 
−
| |
 
−
| colspan="2" | '''By resolution:'''
 
−
| colspan="2" | '''By priority:'''
 
−
|-
 
−
| || Fixed: || 114 || High: || 35
 
−
|-
 
−
| || Dupllicate: || 6 || Normal: || 37
 
−
|-
 
−
| || Invalid: || 4 || Low: || 15
 
−
|-
 
−
| || || || Undecided: || 37
 
−
|-
 
−
| |
 
−
| colspan="4" | '''By Assignee:'''
 
−
|-
 
−
| || Sage || 39
 
−
|-
 
−
| || not-taken || 31
 
−
|-
 
−
| || need-triage || 22
 
−
|-
 
−
| || Stskeeps || 14
 
−
|-
 
−
| || lbt || 6
 
−
|-
 
−
| || letters.random13 || 4
 
−
|-
 
−
| || arnaud || 3
 
−
|-
 
−
| || Martin Brook || 2
 
−
|-
 
−
| || Robin || 2
 
−
|-
 
−
| || Benjamin Federau || 1
 
−
|-
 
−
! Total:
 
−
| colspan="4" | 124 (51 bugs and 71 tasks)
 
−
|-
 
−
|}
 
−
 
−
 
−
 
−
{| border="0" cellspacing="0" cellpadding="2"
 
−
! colspan="5" | '''Mean-time of fixing bugs/tasks, [days]'''
 
−
|-
 
−
! Product:
 
−
| colspan="4" | Mer Core
 
−
|-
 
−
! Status:
 
−
| colspan="4" | RESOLVED, RELEASED, VERIFIED, CLOSED
 
−
|-
 
−
|  || '''Priority:''' || '''Mean-time:''' || '''Min:''' || '''Max:'''
 
−
|-
 
−
| '''Bugs''' || || || ||
 
−
|-
 
−
| || High: || 22.2 || 0.3 || 81.5
 
−
|-
 
−
| || Normal: || 55.3 || 2.7 || 136.7
 
−
|-
 
−
| || Low: || - || - || -
 
−
|-
 
−
| || Undecided: || 11.5 || 0.0 || 49.0
 
−
|-
 
−
| '''Tasks''' || || || ||
 
−
|-
 
−
| || High: || 46.1 || 0.1 || 108.2
 
−
|-
 
−
| || Normal: || 39.1 || 7.0 || 126.9
 
−
|-
 
−
| || Low: || 37.4 || 9.3 || 97.0
 
−
|-
 
−
| || Undecided: || 20.4 || 0.0 || 80.0
 
−
|-
 
−
|}
 
−
 
−
 
−
 
−
Updated on 18.07.2012
 
−
 
−
== QA Processes ==
 
−
 
−
* [[Quality/Terminology|Terminology/Glossary]]
 
 
* [[Quality/Test coverage|Test coverage]]
 
* [[Quality/Test coverage|Test coverage]]
 +
* [[Quality/Metrics|Metrics]]
 
* [[Quality/Test processes|Test processes]]
 
* [[Quality/Test processes|Test processes]]
 
* [[Quality/Development|Development]]
 
* [[Quality/Development|Development]]
 +
* [[Quality/Terminology|Terminology/Glossary]]
  
 
== QA tools ==
 
== QA tools ==
Line 162: Line 14:
  
 
HowTo's
 
HowTo's
 +
* [[Quality/Tutorials/Testing_with_SDK_to_VM]]
 
* [[Quality/ExecuteTests|Execute tests]]
 
* [[Quality/ExecuteTests|Execute tests]]
 
* [[Quality/TestDefinitionHowTo|Writing test-definition XMLs]]
 
* [[Quality/TestDefinitionHowTo|Writing test-definition XMLs]]
Line 167: Line 20:
 
== ToDo ==
 
== ToDo ==
  
−
1. '''Test automation'''
+
1. '''Test automation and tools'''
 
# Automatic smoke testing
 
# Automatic smoke testing
−
## Create test automation virtual image (with eat, SSH and static IP)
+
## Create test automation virtual image ([[Quality/ExecuteTests | HowTo]])
−
## Setup OTS environment
+
## Setup test automation environment (Yunta from Jolla is working on this)
−
### Create "flasher" for OTS
+
## Create smoke test plan ([[Quality/Test coverage|Test plans]])
−
## Create smoke test plan
+
## QA-reports for test result reporting ([https://bugs.merproject.org/show_bug.cgi?id=244 MER-244], phaeron)
−
## Test result reporting (QA-reports, see task [https://bugs.merproject.org/show_bug.cgi?id=244 Mer-244])
+
# testrunner-ui refactorin ([https://bugs.merproject.org/show_bug.cgi?id=436 MER-436], timoph)
−
# Review OTS and how it fits to the BOSS
+
# TDriver in Qt5 ([https://bugs.merproject.org/show_bug.cgi?id=439 MER-439], Paimen)
−
# Design to identify test constraints [https://bugs.merproject.org/show_bug.cgi?id=317 Mer-317]
+
# TDriver with QtQuick 2.0 ([https://bugs.merproject.org/show_bug.cgi?id=440 MER-440], Paimen)
−
# Netconsole for debugging, http://www.mjmwired.net/kernel/Documentation/networking/netconsole.txt  
+
# Design to identify test constraints ([https://bugs.merproject.org/show_bug.cgi?id=317 MER-317], not taken)
 +
# Netconsole for debugging, http://www.mjmwired.net/kernel/Documentation/networking/netconsole.txt (not taken)
  
 
2. ''' Test coverage '''
 
2. ''' Test coverage '''
−
# Create a coverage chart/table for core packages
+
# Create a coverage chart/table for core packages ([[Quality/Test coverage|Test coverage]])
  
 
3. ''' Mer Dashboard '''
 
3. ''' Mer Dashboard '''
−
# Mer dashboard [https://bugs.merproject.org/show_bug.cgi?id=429 Mer-429]
+
# Mer dashboard ([https://bugs.merproject.org/show_bug.cgi?id=429 Mer-429], kyyberi and lbt)
 
# http://www.youtube.com/watch?v=urmp-fmjRro
 
# http://www.youtube.com/watch?v=urmp-fmjRro
  
Line 189: Line 43:
 
# Create a documentation how to contribute to the QA tests
 
# Create a documentation how to contribute to the QA tests
  
−
[https://bugs.merproject.org/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&product=Mer%20QA%20Tools List of all bugs and tasks related to QA.]
+
[https://bugs.merproject.org/buglist.cgi?query_format=advanced&bug_status=NEEDINFO&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=TRIAGEDUPSTREAM&bug_status=REOPENED&product=Mer%20QA List of all bugs and tasks in QA area.]
  
 
== Contact ==
 
== Contact ==
Line 200: Line 54:
 
=== Future topics ===
 
=== Future topics ===
  
−
* Bug metrics and QA ToDo list (19.7.2012)
+
* Test coverage
−
* Test coverage (26.7.2012)
+
 
  
 
=== Minutes ===
 
=== Minutes ===
  
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-07-12-12.00.html Meeting minutes 2012-07-12, Status update]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-09-27-12.00.html Meeting minutes 2012-09-27, Status update and QA roles changes]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-07-05-12.20.html Meeting minutes 2012-07-05, Status update]
+
* [http://epeli.org/mer/mer-qa-meetin-2012-09-13.txt Meeting minutes 2012-09-13, Status update]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-06-21-12.00.html Meeting minutes 2012-06-21, Acceptance criteria for the tools and tests]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-08-09-12.03.html Meeting minutes 2012-08-09, Status update]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-06-14-12.01.html Meeting minutes 2012-06-14, Status update]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-08-02-12.00.html Meeting minutes 2012-08-02, Mer Dashboard]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-05-31-12.00.html Meeting minutes 2012-05-31, Test automation image]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-07-26-12.00.html Meeting minutes 2012-07-26, OTS replacement]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-05-24-12.00.html Meeting minutes 2012-05-24, QA tools and OBS]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-07-19-12.00.html Meeting minutes 2012-07-19, Bug metrics and QA ToDo]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-05-10-12.00.html Meeting minutes 2012-05-10, Status update]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-07-12-12.00.html Meeting minutes 2012-07-12, Status update]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-05-03-12.16.html Meeting minutes 2012-05-03, QA process] and [http://epeli.org/mer/mer-qa-meetin-2012-05-03.txt full log]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-07-05-12.20.html Meeting minutes 2012-07-05, Status update]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-04-26-12.00.html Meeting minutes 2012-04-26, Test constraints and mapping]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-06-21-12.00.html Meeting minutes 2012-06-21, Acceptance criteria for the tools and tests]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-04-19-11.00.html Meeting minutes 2012-04-19, Test package mapping]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-06-14-12.01.html Meeting minutes 2012-06-14, Status update]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-04-12-12.01.html Meeting minutes 2012-04-12, QA process, tools and tests]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-05-31-12.00.html Meeting minutes 2012-05-31, Test automation image]
−
* [http://mer.bfst.de/meetings/mer-meeting/2012/mer-meeting.2012-04-03-11.00.html Meeting minutes 2012-04-03, Initial meeting]
+
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-05-24-12.00.html Meeting minutes 2012-05-24, QA tools and OBS]
 +
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-05-10-12.00.html Meeting minutes 2012-05-10, Status update]
 +
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-05-03-12.16.html Meeting minutes 2012-05-03, QA process] and [http://epeli.org/mer/mer-qa-meetin-2012-05-03.txt full log]
 +
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-04-26-12.00.html Meeting minutes 2012-04-26, Test constraints and mapping]
 +
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-04-19-11.00.html Meeting minutes 2012-04-19, Test package mapping]
 +
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-04-12-12.01.html Meeting minutes 2012-04-12, QA process, tools and tests]
 +
* [http://merproject.org/meetings/mer-meeting/2012/mer-meeting.2012-04-03-11.00.html Meeting minutes 2012-04-03, Initial meeting]
  
 
[[Category:QA]]
 
[[Category:QA]]

Latest revision as of 12:38, 8 October 2012

Contents

[edit] Quality

[edit] QA tools

QA tools used in Mer:

HowTo's

[edit] ToDo

1. Test automation and tools

  1. Automatic smoke testing
    1. Create test automation virtual image ( HowTo)
    2. Setup test automation environment (Yunta from Jolla is working on this)
    3. Create smoke test plan (Test plans)
    4. QA-reports for test result reporting (MER-244, phaeron)
  2. testrunner-ui refactorin (MER-436, timoph)
  3. TDriver in Qt5 (MER-439, Paimen)
  4. TDriver with QtQuick 2.0 (MER-440, Paimen)
  5. Design to identify test constraints (MER-317, not taken)
  6. Netconsole for debugging, http://www.mjmwired.net/kernel/Documentation/networking/netconsole.txt (not taken)

2. Test coverage

  1. Create a coverage chart/table for core packages (Test coverage)

3. Mer Dashboard

  1. Mer dashboard (Mer-429, kyyberi and lbt)
  2. http://www.youtube.com/watch?v=urmp-fmjRro

4. Documentation

  1. Create Category:About pages for tools and processes
  2. Create a documentation how to contribute to the QA tests

List of all bugs and tasks in QA area.

[edit] Contact

[edit] Meetings

[edit] Future topics

  • Test coverage


[edit] Minutes

Retrieved from "https://wiki.merproject.org/wiki/index.php?title=Quality&oldid=2581"
Personal tools