The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)

MDS2

From Mer Wiki
(Difference between revisions)
Jump to: navigation, search
(Created page with " = MDS2 = MDS2 uses a different manner of accessing repositories and source code than MDS1. This means that instead of accessing Mer:MDS:Core:i586, you access Mer:MDS:Core:i586:…")
 
 
(4 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
 
−
= MDS2 =
 
−
 
 
MDS2 uses a different manner of accessing repositories and source code than MDS1. This means that instead of accessing Mer:MDS:Core:i586, you access Mer:MDS:Core:i586:<version|tag|branch>
 
MDS2 uses a different manner of accessing repositories and source code than MDS1. This means that instead of accessing Mer:MDS:Core:i586, you access Mer:MDS:Core:i586:<version|tag|branch>
  
 
Instead of using a individual checkout, MDS2 uses the git repositories directly and allows individual downloads of binary releases.
 
Instead of using a individual checkout, MDS2 uses the git repositories directly and allows individual downloads of binary releases.
 +
 +
= Installation =
 +
 +
== Requirements ==
 +
 +
This guide has been tested with:
 +
 +
* a fresh minimal OpenSuse 12.1 install with network connection
 +
* python 2.7
 +
* git, gitdb, GitPython
 +
* recent lxml (2.3 was tested)
 +
 +
== Setup ==
 +
 +
=== As user: root ===
 +
 +
==== Create mer user ====
 +
 +
<pre>
 +
useradd -m -d /home/mer -g users -k /etc/skel -r -s /bin/bash mer
 +
passwd mer  # set the password for the new user
 +
</pre>
 +
 +
==== Install packages ====
 +
 +
If zypper tells you about conflicting packages, remove them.
 +
 +
<pre>
 +
zypper install zlib-devel python-devel gcc gcc-g++ python-pip git-core python-xml libxml2-devel libxslt-devel
 +
 +
pip install gitdb
 +
pip install GitPython
 +
pip install elementtree
 +
pip install lxml
 +
pip install PDIS-XPath
 +
</pre>
 +
 +
=== As user: mer ===
 +
 +
==== Clone MDS repo from Github ====
 +
 +
<pre>
 +
cd /home/mer
 +
ssh-keygen -t rsa
 +
git clone git://github.com/mer-tools/mds.git
 +
cd mds
 +
git checkout mds2
 +
</pre>
 +
 +
==== Check mappings.xml file ====
 +
 +
<pre>
 +
<mappings>
 +
<mapping project="Core" path="packages-git/mer/project-core"
 +
                        binaries="obs-repos/"
 +
                        packages-path="packages-git/"
 +
                        packages-upstream="rsync://releases.merproject.org/mer-releases/packages-git/"
 +
                        binaries-upstream="rsync://releases.merproject.org/mer-releases/obs-repos/"/>
 +
</mappings>
 +
</pre>
 +
 +
==== Launch daemon ====
 +
 +
<pre>
 +
python2.7 tools/mds2.py 7000
 +
</pre>
 +
 +
== Systemd integration ==
 +
 +
For systemd create a service file to run MDS as a daemon. Note the value of the port, and uncomment the rsync proxy if needed.
 +
 +
<pre>
 +
[Unit]
 +
Description=Mer Delivery System
 +
After=multi-user.target
 +
 +
[Service]
 +
User=mer
 +
Group=users
 +
WorkingDirectory=/home/mer/mds
 +
#Environment=RSYNC_PROXY=proxy:8080
 +
ExecStart=/usr/bin/python2.7 /home/mer/mds/tools/mds2.py 7000
 +
Restart=always
 +
 +
[Install]
 +
WantedBy=multi-user.target
 +
</pre>
 +
 +
== Initial download of packages and repos ==
 +
 +
Choose the initial Mer release that you want to download, for example 0.20130528.1, and pull packages and repos using the REST API.
 +
 +
'''WARNING:''' It will take a long time with no output, just be patient.
 +
 +
<pre>
 +
curl http://127.0.0.1:7000/update/packages/Core
 +
curl http://127.0.0.1:7000/update/repo/Core/0.20130528.1
 +
</pre>
  
 
= Administration commands =
 
= Administration commands =
Line 10: Line 105:
 
In the following, HOST:PORT is the host and port of the MDS2 server:
 
In the following, HOST:PORT is the host and port of the MDS2 server:
  
 +
<pre>
 
curl http://HOST:PORT/update/packages/<project>
 
curl http://HOST:PORT/update/packages/<project>
 +
</pre>
  
 
What this will do is look in mappings.xml, find the relevant project mapping, and then rsync update from packages-upstream to packages-path. This will bring the packages-path directory up to date, and make the relevant version/tag/branches available through OBS protocol.
 
What this will do is look in mappings.xml, find the relevant project mapping, and then rsync update from packages-upstream to packages-path. This will bring the packages-path directory up to date, and make the relevant version/tag/branches available through OBS protocol.
  
 +
<pre>
 
curl http://HOST:PORT/update/repo/<project>/<version|tag|branch>
 
curl http://HOST:PORT/update/repo/<project>/<version|tag|branch>
 +
</pre>
  
 
What this will do is look in mappings.xml, find the relevant project mapping, then rsync from the binaries-upstream the binary repositories named <project>:*:<version|tag|branch> into 'binaries' directory. This makes it available for binary repository access through OBS protocol.
 
What this will do is look in mappings.xml, find the relevant project mapping, then rsync from the binaries-upstream the binary repositories named <project>:*:<version|tag|branch> into 'binaries' directory. This makes it available for binary repository access through OBS protocol.
Line 24: Line 123:
 
When a new Mer release or prerelease comes in, you need to first sync the local copy of the git repositories, which includes the Mer Core in git.
 
When a new Mer release or prerelease comes in, you need to first sync the local copy of the git repositories, which includes the Mer Core in git.
  
 +
<pre>
 
curl http://cmds:8002/update/packages/Core
 
curl http://cmds:8002/update/packages/Core
−
 
+
</pre>
 
This will synchronise the git repositories and as one of the effects, make the sources for the new release available in mer:mds2:Core:ARCH:<version>
 
This will synchronise the git repositories and as one of the effects, make the sources for the new release available in mer:mds2:Core:ARCH:<version>
  
 
Next up, you need to import the individual release:
 
Next up, you need to import the individual release:
  
 +
<pre>
 
curl http://cmds:8002/update/repo/Core/<version>
 
curl http://cmds:8002/update/repo/Core/<version>
 +
</pre>
  
 
This will rsync down the individual binary release locally. Now over OBS api this version has binary repositories available for mer:mds2:Core:ARCH:<version>
 
This will rsync down the individual binary release locally. Now over OBS api this version has binary repositories available for mer:mds2:Core:ARCH:<version>
Line 36: Line 138:
 
So how do we update so that people take the new version into use?
 
So how do we update so that people take the new version into use?
  
−
mer:devel serves as "This project serves as a convenient pointer to the mer core project in the mds2 instance".
+
mer:devel project serves as "This project serves as a convenient pointer to the mer core project in the mds2 instance".
  
−
You then modify the 'next' pointers in that meta prj to the version you'd like it to within MDS2.
+
You then modify the 'next' or 'latest' pointers in that meta prj, depending on which type of release, to the version you'd like it to within MDS2. Current maintainers of mer:devel are lbt, stskeeps and sage

Latest revision as of 10:09, 14 June 2013

MDS2 uses a different manner of accessing repositories and source code than MDS1. This means that instead of accessing Mer:MDS:Core:i586, you access Mer:MDS:Core:i586:<version|tag|branch>

Instead of using a individual checkout, MDS2 uses the git repositories directly and allows individual downloads of binary releases.

Contents

[edit] Installation

[edit] Requirements

This guide has been tested with:

  • a fresh minimal OpenSuse 12.1 install with network connection
  • python 2.7
  • git, gitdb, GitPython
  • recent lxml (2.3 was tested)

[edit] Setup

[edit] As user: root

[edit] Create mer user 

useradd -m -d /home/mer -g users -k /etc/skel -r -s /bin/bash mer
passwd mer   # set the password for the new user

[edit] Install packages

If zypper tells you about conflicting packages, remove them. 

zypper install zlib-devel python-devel gcc gcc-g++ python-pip git-core python-xml libxml2-devel libxslt-devel

pip install gitdb
pip install GitPython
pip install elementtree
pip install lxml
pip install PDIS-XPath

[edit] As user: mer

[edit] Clone MDS repo from Github 

cd /home/mer
ssh-keygen -t rsa
git clone git://github.com/mer-tools/mds.git
cd mds
git checkout mds2

[edit] Check mappings.xml file 

<mappings>
<mapping project="Core" path="packages-git/mer/project-core"
                        binaries="obs-repos/"
                        packages-path="packages-git/"
                        packages-upstream="rsync://releases.merproject.org/mer-releases/packages-git/"
                        binaries-upstream="rsync://releases.merproject.org/mer-releases/obs-repos/"/>
</mappings>

[edit] Launch daemon 

python2.7 tools/mds2.py 7000

[edit] Systemd integration

For systemd create a service file to run MDS as a daemon. Note the value of the port, and uncomment the rsync proxy if needed. 

[Unit]
Description=Mer Delivery System
After=multi-user.target

[Service]
User=mer
Group=users
WorkingDirectory=/home/mer/mds
#Environment=RSYNC_PROXY=proxy:8080
ExecStart=/usr/bin/python2.7 /home/mer/mds/tools/mds2.py 7000
Restart=always

[Install]
WantedBy=multi-user.target

[edit] Initial download of packages and repos

Choose the initial Mer release that you want to download, for example 0.20130528.1, and pull packages and repos using the REST API.

WARNING: It will take a long time with no output, just be patient. 

curl http://127.0.0.1:7000/update/packages/Core
curl http://127.0.0.1:7000/update/repo/Core/0.20130528.1

[edit] Administration commands

In the following, HOST:PORT is the host and port of the MDS2 server: 

curl http://HOST:PORT/update/packages/<project>

What this will do is look in mappings.xml, find the relevant project mapping, and then rsync update from packages-upstream to packages-path. This will bring the packages-path directory up to date, and make the relevant version/tag/branches available through OBS protocol. 

curl http://HOST:PORT/update/repo/<project>/<version|tag|branch>

What this will do is look in mappings.xml, find the relevant project mapping, then rsync from the binaries-upstream the binary repositories named <project>:*:<version|tag|branch> into 'binaries' directory. This makes it available for binary repository access through OBS protocol.

[edit] MDS2 in Mer Community OBS

The MDS2 sits on cmds:8002 on the cmds host, within /data/mds

When a new Mer release or prerelease comes in, you need to first sync the local copy of the git repositories, which includes the Mer Core in git. 

curl http://cmds:8002/update/packages/Core

This will synchronise the git repositories and as one of the effects, make the sources for the new release available in mer:mds2:Core:ARCH:<version>

Next up, you need to import the individual release: 

curl http://cmds:8002/update/repo/Core/<version>

This will rsync down the individual binary release locally. Now over OBS api this version has binary repositories available for mer:mds2:Core:ARCH:<version>

So how do we update so that people take the new version into use?

mer:devel project serves as "This project serves as a convenient pointer to the mer core project in the mds2 instance".

You then modify the 'next' or 'latest' pointers in that meta prj, depending on which type of release, to the version you'd like it to within MDS2. Current maintainers of mer:devel are lbt, stskeeps and sage

Retrieved from "https://wiki.merproject.org/wiki/index.php?title=MDS2&oldid=3671"
Personal tools