The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)


Mer Delivery System

From Mer Wiki
(Difference between revisions)
Jump to: navigation, search
(debian install notes)
(Installation)
 
(10 intermediate revisions by 4 users not shown)
Line 35: Line 35:
 
  pip install gitdb
 
  pip install gitdb
 
  pip install GitPython
 
  pip install GitPython
 +
 +
Under OpenSUSE11.4, it is a better way to install smmap/async/gitdb/GitPython as following:
 +
zypper install python-pip
 +
pip install smmap
 +
pip install async
 +
pip install gitdb
 +
pip install GitPython
 +
  
 
Get the release tools
 
Get the release tools
 +
mkdir -p /srv/mer/mds/
 +
cd /srv/mer/mds/
 
  git clone http://review.merproject.org/p/mer/release-tools
 
  git clone http://review.merproject.org/p/mer/release-tools
  
Line 66: Line 76:
 
  export RSYNC_PROXY=proxy.company.com:8080
 
  export RSYNC_PROXY=proxy.company.com:8080
  
  cd release-tools; make
+
  cd release-tools; make; make update
Note: after doing "make", you should do "make update" to grab the rpm binaries.
+
 
 +
Note: after doing "make", you "make update" grabs the rpm binaries.
  
 
=== Update ===
 
=== Update ===
Line 155: Line 166:
 
  User=mds
 
  User=mds
 
  Group=mds
 
  Group=mds
  WorkingDirectory=/data/release-tools/
+
  WorkingDirectory=/srv/mer/mds/release-tools/
  ExecStart=/usr/bin/python /data/release-tools/tools/fakeobs.py 8001
+
  ExecStart=/usr/bin/python /srv/mer/mds/release-tools/tools/fakeobs.py 8001
 
   
 
   
 
  [Install]
 
  [Install]
Line 174: Line 185:
  
  
You can [https://bugs.merproject.org/enter_bug.cgi?product=Mer%20Core
+
You can [https://bugs.merproject.org/enter_bug.cgi?product=Mer%20Core log a bug] (against the .Release-tools component) or show up in #mer and
log a bug] (against the .Release-tools component) or show up in #mer and
+
 
poke Stskeeps if there's any questions.
 
poke Stskeeps if there's any questions.
  
 
[[Category:mds]][[Category:obs]]
 
[[Category:mds]][[Category:obs]]
 +
 +
= MDS2 =
 +
 +
'''DRAFT'''
 +
 +
Mer Delivery System 2.0 is a new approach to the MDS idea. Instead of utilizing checked out (in the git sense) project cores to know what OBS projects & packages it has to present, it now has the ability to access any commit/tag/git reference inside a git repository.
 +
 +
== mappings.xml ==
 +
 +
Instead of a
 +
  <mapping project="Core:i586" path="obs-projects/Core/i586" binaries="obs-repos/Core:i586:latest" reponame="Core_i586" />
 +
a different form is used:
 +
  <mapping project="Core" path="packages-git/mer/mer-core" />
 +
 +
When querying MDS through it's OBS API, it will now respond to requests for Core:GIT_REFERENCE:SUBDIR. This means, to find which packages to respond with, it will look in mappings.xml for "Core", find the GIT_REFERENCE commit in packages-git/mer/mer-core and extract packages.xml from subdir/ in the tree.

Latest revision as of 01:08, 14 June 2012

Mer provides the Mer Delivery System (MDS) to make it easy for vendors to track upstream Mer releases.

Contents

[edit] Overview

MDS is a small collection of tools that both supports distributing Mer binary releases and OBS build targets to vendors and accessing the local copy of the release from a private vendor-managed OBS.

[edit] Operations

[edit] MDS tools

[edit] Installation

The MDS is installed in the root of the data area which will store the Mer releases (several Gb). This may be a dedicated VM.

Ensure the system has the following installed:

On Debian this can be done as follows (although pip is not a preferred Debian solution):

aptitude install python-dev build-essential python-pip ca-certificates git git-core libcurl3-gnutls liberror-perl libidn11
pip install smmap
pip install async
pip install gitdb
pip install GitPython

Under OpenSUSE11.4, it is a better way to install smmap/async/gitdb/GitPython as following:

zypper install python-pip
pip install smmap
pip install async
pip install gitdb
pip install GitPython


Get the release tools

mkdir -p /srv/mer/mds/
cd /srv/mer/mds/
git clone http://review.merproject.org/p/mer/release-tools

[edit] Update

The tools are updated periodically - simply run

git pull

Eventually:

git remote update

to update and then

git checkout vXXX

to obtain the version required.

[edit] Mer release(s)

[edit] Setup

This step will pull (using rsync) all the git repositories for the Mer packages (nb: this is a huge download).

It will then build a list of the packages, create a set of mappings and prepare the OBS event stream.

Finally the OBS projects are cloned and extracted ready for the API service.

Note: You might need to set rsync proxy:

export RSYNC_PROXY=proxy.company.com:8080
cd release-tools; make; make update

Note: after doing "make", you "make update" grabs the rpm binaries.

[edit] Update

An update simply requires:

cd release-tools; make update

This will re-synchnronise the package git repos, the OBS projects and reset the 'latest' link.

It can be run periodically to ensure that up-to-date changes are available.

There is no mechanism at the moment to specify that certain releases should be made available.

You might have to, after 'make update' to go into obs-project/Core and do 'git checkout db08d70258b5a53689fcee3d3deb94926015ff26', as to get the right version of core.

[edit] Setup an API service for Mer releases

MDS runs a simple http server that emulates the OBS api for a static release.

To start this 'fake obs server' (hence the original name for the packages):

python tools/fakeobs.py 8001 &

Note: You should do this under "release-tools" directory, since fakeobs.py will read mappings.xml under "release-tools" directory.

[edit] Add the MDS project as a remote link

A local OBS can now add a 'remote link' to the MDS API service (FakeOBS) port 8001, /public, note, http, not-https. So assuming the MDS is running on a machine called 'mer' then this would work.

<project name="MerDS">
 <title>Mer Delivery System</title>
 <description>A 'remote link' to the MDS API service running against a local copy of Mer
 </description>
 <remoteurl>http://mer:8001/public</remoteurl>
 <person userid="Admin" role="maintainer"/>
 <person userid="Admin" role="bugowner"/>
</project>

[edit] MDS Build Targets

Mer supports a number of build targets for various release points and architectures.

These are organised as:

<link>:<area>:<arch>:<release>

So the build target:

MerDS:Core:armv6l:0.20120120.1

would build against the collection of packages that forms 'Core' built for the armv6l architecture.

[edit] Use MDS repos as a build target

Once your project is set-up, you can add Mer build targets to the OBS and they should start building automatically. Given a setup as above then a project meta would contain repositories similar to this:

<project name="MyMerUx">
  <title>MyMerUx</title>
  <description>
  A UX building against Mer
  </description>
  <person userid="Admin" role="maintainer"/>
  <person userid="Admin" role="bugowner"/>
  <repository name="MyMerUX_Mer_Core_i586">
    <path repository="Core_i586" project="MerDS:Core:i586"/>
    <arch>i586</arch>
  </repository>
</project>

[edit] Running under Systemd

MDS can (and should) be run as a service. This sample systemd service file may be useful:

in /etc/systemd/system/mds.service

[Unit]
Description=Mer Delivery System
After=multi-user.target

[Service]
User=mds
Group=mds
WorkingDirectory=/srv/mer/mds/release-tools/
ExecStart=/usr/bin/python /srv/mer/mds/release-tools/tools/fakeobs.py 8001

[Install]
WantedBy=multi-user.target


Then:

systemctl enable mds.service
systemctl start mds.service
systemctl status mds.service

[edit] Limitations

There is no way to limit MDS to only certain architectures.

There is no way to see the list of build target releases or architectures.


You can log a bug (against the .Release-tools component) or show up in #mer and poke Stskeeps if there's any questions.

[edit] MDS2

DRAFT

Mer Delivery System 2.0 is a new approach to the MDS idea. Instead of utilizing checked out (in the git sense) project cores to know what OBS projects & packages it has to present, it now has the ability to access any commit/tag/git reference inside a git repository.

[edit] mappings.xml

Instead of a

 <mapping project="Core:i586" path="obs-projects/Core/i586" binaries="obs-repos/Core:i586:latest" reponame="Core_i586" /> 

a different form is used:

 <mapping project="Core" path="packages-git/mer/mer-core" />

When querying MDS through it's OBS API, it will now respond to requests for Core:GIT_REFERENCE:SUBDIR. This means, to find which packages to respond with, it will look in mappings.xml for "Core", find the GIT_REFERENCE commit in packages-git/mer/mer-core and extract packages.xml from subdir/ in the tree.

Personal tools