Process

Revision as of 22:42, 22 October 2011

Mer Process

The objective of the Mer change control processes is to ensure that each change is subjected to sensible QA checks - both automated and manual and to encourage Vendor participation in the QA process.

We will:

• Verify every Mer release through continuous integration and QA
• Provide a easy contribution model with fast feedback on changes
• Have a distributed model of performing QA checks, so we can easily scale operations and support a variety of hardware

Summary

A contributor makes and submits a change which triggers the process.

When a change is proposed it must pass the following quality gates: (Each of these tests is described in more detail later.)

Core Checks

• Packaging checks
• Build check
• ABI Checks
• Trial build

Code Review (48hr window)

The change is available for manual review.

Vendor Checks: Early and Late (48hr window)

The change is now ready for automated review by vendors; first early notifications are sent, then after a while, late notifications.

• Image creation
• Device flashing
• Functional (OTS) tests

Manual testing (48hr window)

With such a short window there is limited opportunity for manual testing.

Maintainer Review

After 48hrs the maintainer looks at the submitted reviews and makes an accept/decline decision.

CI Automation

In order to verify that no obviously conflicting changes have occured the change is queued for automated acceptance.

Issues

Handling conflicts?

Change Controlled Code

We have:

• A large amount of git repositories, with branches "master" (Trunk) and potentially release branches (Mer_1.0, Mer_2.0)
• Each of these repositories checks out to be a .spec package, with tarball included, including _meta (OBS meta information) and _attribute (attributes)
• "Core" git repository, which contains .xml files and other OBS configuration along with pointers to specific git repositories and commit IDs representing current state of Mer Core

Contributor Process (abstracted)

When a contributor identifies that a change is required in a package they should:

1. Clone the git repository for the package
2. Make changes to the package
3. Commit with a signed-off-by in the commit message
4. Submit the change to the review process

They should now observe reviews and correct problems. This process is repeated until the change is approved and merged to the master git repository or until the proposal is rejected.

Reviewer process (abstracted)

1. Read reviews of code changes on review site
2. Perform own review and comment on review site

Maintainer process (abstracted)

1. Read manual and automatic reviews
2. Optionally do own review
3. Approve/decline contribution on site
4. Merge code change (manually or automatically)

Core Checks process (Automated)

When a proposed change comes in the following automated core checks are carried out and reported as 'reviews' of the change.

Packaging checks

Check if the proposed change is acceptable according to packaging guidelines.

Single-build test

Test if the proposed change builds on it's own or if the appropriate sha1 has been built on a public OBS.

Dependency-build test

Test if the proposed change will break other packages in the Core by compiling the package and everything that depends on it (directly and indirectly).

Following a dependency build there will be fresh packages to permit additional checks:

ABI checks

Test if the changed package causes any kind of breakage in ABI, utilizing http://forge.ispras.ru/projects/abi-compliance-checker

Vendor notification process (Automated)

A notice is sent over various channels (eg email, AMQP, rss) to notify the vendor community that a change is available to be checked. The results of the dependency-build test are provided and the vendor can carry out vendor specific checks.

The objective of this notification step is to ensure that changes do not cause regressions in vendor devices; or that, if applied to Core, vendors will be aware of problems that arise)

This is a 2-stage process; certain early notification vendors are told about the change right away, if there are no bad reviews within 12 hours, we notify late vendors. Reviews must be provided within the 48 hr review window to be considered.

It is possible that Mer will operate a sample reference vendor using a 'virtual device' such as kvm.

[Question: What's the criteria for *not* sending late notifications? Is this a veto?] [Question: What kinds of checks should be allowed in the early notification review? Veto is quite powerful so it should probably be limited to "Doesn't build/boot"]

Typical gates here are:

Image creation test

Test if the repository result of the dependency-build test will create an 'image' succesfully, based on the vendor kickstart files.

Upgradeability check

Take packages from dependency-build test and attempt an upgrade on a suitable image (eg last official release), report back as code reviews.

Device flash and boot

Verify that any image is flashable and actually boots on a device to a certain level.

Functional (OTS) tests

If any functional or regression tests can be run then any failures may be provided.

Implementation DRAFT

Current implementation, OBS, Gerrit, BOSS

Contributor setup

1. Register account on http://review.merproject.org 2. Add SSH keys and set user name, real name

Contribution model (gerrit)

1. Clone git repository for package from git://username@review.merproject.org:29418/packagepath

2. Do changes to package

3. Commit with signed-off-by in commit message, git commit -s -m "Message"

4. Submit patch to gerrit: git push origin HEAD:refs/for/master

5. Receive reviews and correct problems at http://review.merproject.org, until approved and merged to git repository

Reviewer model (gerrit)

1. Read automatic reviews of code change on http://review.merproject.org

2. Perform own review and comment on review site, use grading:

                              -1 I would prefer that you didn't submit this
                               0 No score
                              +1 Looks good to me, but someone else must approve

Maintainer model (gerrit)

1. Read manual and automatic reviews

2. Optionally do own review, use +2 / -2 rating:

                              -2 Do not submit
                              +2 Looks good to me, approved

3. Approve/decline contribution on site (Submit vs ..?)

4. Code change gets merged automatically

Automation

Documentation for gerrit hooks: http://gerrit-documentation.googlecode.com/svn/Documentation/2.2.0/config-hooks.html

Gerrit hook: patchset-created

patchset-created --change <change id> --change-url <change url> --project <project name> --branch <branch> --uploader <uploader> --commit <sha1> --patchset <patchset id>

Things to make life easier

If the OBS can identify the sha1 of a package then a sha1 built in a public OBS against the correct repos is acceptable