Revision as of 01:05, 14 June 2012

Mer provides the Mer Delivery System (MDS) to make it easy for vendors to track upstream Mer releases.

Overview

MDS is a small collection of tools that both supports distributing Mer binary releases and OBS build targets to vendors and accessing the local copy of the release from a private vendor-managed OBS.

Operations

MDS tools

Installation

The MDS is installed in the root of the data area which will store the Mer releases (several Gb). This may be a dedicated VM.

Ensure the system has the following installed:

• git
• python
• make
• smmap: http://pypi.python.org/pypi/smmap
• async: http://pypi.python.org/pypi/async
• gitdb: http://pypi.python.org/pypi/gitdb
• gitpython homepage: http://pypi.python.org/pypi/GitPython/0.3.2.RC1
• gitpython install guide: http://packages.python.org/GitPython/0.3.2/intro.html#installing-gitpython
• cpio

On Debian this can be done as follows (although pip is not a preferred Debian solution):

aptitude install python-dev build-essential python-pip ca-certificates git git-core libcurl3-gnutls liberror-perl libidn11
pip install smmap
pip install async
pip install gitdb
pip install GitPython

Under OpenSUSE11.4, it is a better way to install smmap/async/gitdb/GitPython as following:

zypper install python-pip

pip install smmap

pip install async

pip install gitdb

pip install GitPython

Get the release tools

mkdir -p /srv/mer/mds/
cd /srv/mer/mds/
git clone http://review.merproject.org/p/mer/release-tools

Update

The tools are updated periodically - simply run

git pull

Eventually:

git remote update

to update and then

git checkout vXXX

to obtain the version required.

Mer release(s)

Setup

This step will pull (using rsync) all the git repositories for the Mer packages (nb: this is a huge download).

It will then build a list of the packages, create a set of mappings and prepare the OBS event stream.

Finally the OBS projects are cloned and extracted ready for the API service.

Note: You might need to set rsync proxy:

export RSYNC_PROXY=proxy.company.com:8080

cd release-tools; make; make update

Note: after doing "make", you "make update" grabs the rpm binaries.

Update

An update simply requires:

cd release-tools; make update

This will re-synchnronise the package git repos, the OBS projects and reset the 'latest' link.

It can be run periodically to ensure that up-to-date changes are available.

There is no mechanism at the moment to specify that certain releases should be made available.

You might have to, after 'make update' to go into obs-project/Core and do 'git checkout db08d70258b5a53689fcee3d3deb94926015ff26', as to get the right version of core.

Setup an API service for Mer releases

MDS runs a simple http server that emulates the OBS api for a static release.

To start this 'fake obs server' (hence the original name for the packages):

python tools/fakeobs.py 8001 &

Note: You should do this under "release-tools" directory, since fakeobs.py will read mappings.xml under "release-tools" directory.

Add the MDS project as a remote link

A local OBS can now add a 'remote link' to the MDS API service (FakeOBS) port 8001, /public, note, http, not-https. So assuming the MDS is running on a machine called 'mer' then this would work.

<project name="MerDS">
 <title>Mer Delivery System</title>
 <description>A 'remote link' to the MDS API service running against a local copy of Mer
 </description>
 <remoteurl>http://mer:8001/public</remoteurl>
 <person userid="Admin" role="maintainer"/>
 <person userid="Admin" role="bugowner"/>
</project>

MDS Build Targets

Mer supports a number of build targets for various release points and architectures.

These are organised as:

<link>:<area>:<arch>:<release>

So the build target:

MerDS:Core:armv6l:0.20120120.1

would build against the collection of packages that forms 'Core' built for the armv6l architecture.

Use MDS repos as a build target

Once your project is set-up, you can add Mer build targets to the OBS and they should start building automatically. Given a setup as above then a project meta would contain repositories similar to this:

<project name="MyMerUx">
  <title>MyMerUx</title>
  <description>
  A UX building against Mer
  </description>
  <person userid="Admin" role="maintainer"/>
  <person userid="Admin" role="bugowner"/>
  <repository name="MyMerUX_Mer_Core_i586">
    <path repository="Core_i586" project="MerDS:Core:i586"/>
    <arch>i586</arch>
  </repository>
</project>

Running under Systemd

MDS can (and should) be run as a service. This sample systemd service file may be useful:

in /etc/systemd/system/mds.service

[Unit]
Description=Mer Delivery System
After=multi-user.target

[Service]
User=mds
Group=mds
WorkingDirectory=/srv/mer/mds/release-tools/
ExecStart=/usr/bin/python /srv/mer/mds/release-tools/tools/fakeobs.py 8001

[Install]
WantedBy=multi-user.target

Then:

systemctl enable mds.service
systemctl start mds.service
systemctl status mds.service

Limitations

There is no way to limit MDS to only certain architectures.

There is no way to see the list of build target releases or architectures.

You can log a bug (against the .Release-tools component) or show up in #mer and poke Stskeeps if there's any questions.

MDS2

DRAFT

Mer Delivery System 2.0 is a new approach to the MDS idea. Instead of utilizing checked out (in the git sense) project cores to know what OBS projects & packages it has to present, it now has the ability to access any commit/tag/git reference inside a git repository.

mappings.xml

Instead of a

 <mapping project="Core:i586" path="obs-projects/Core/i586" binaries="obs-repos/Core:i586:latest" reponame="Core_i586" /> 

a different form is used:

 <mapping project="Core" path="packages-git/mer/mer-core" />

When querying MDS through it's OBS API, it will now respond to requests for Core:GIT_REFERENCE:SUBDIR. This means, to find which packages to respond with, it will look in mappings.xml for "Core", find the GIT_REFERENCE commit in packages-git/mer/mer-core and extract packages.xml from subdir/ in the tree.