The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)


OBS Setup

From Mer Wiki
(Difference between revisions)
Jump to: navigation, search
(With thanks to contributors from http://wiki.meego.com/Build_Infrastructure/Sysadmin_Distro/OBS_setup_openSUSE114)
 
(note new scripts)
 
(15 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Please check https://github.com/openSUSE/open-build-service  for the latest setup instructions!
+
These steps show how to set up a Mer Open Build Service (OBS) on an openSuSE 11.4 installation. An OBS installation will usually consist of an OBS web server, a main OBS server and one or more OBS workers. This document describes how to setup a basic evaluation installation and is not meant as production ready or hardened setup.
 
+
These steps show how to set up an Open Build Service (OBS) on an openSuSE 11.4 installation. An OBS installation will usually consist of an OBS web server, a main OBS server and one or more OBS workers. This document describes how to setup a basic evaluation installation and is not meant as production ready or hardened setup.
+
  
 
PLEASE NOTE : Most users won't need to install their own OBS and can just use an account on a publicly accessible OBS.
 
PLEASE NOTE : Most users won't need to install their own OBS and can just use an account on a publicly accessible OBS.
Line 7: Line 5:
 
Vendors may need to have full control of their builds and keep source private due to licensing restrictions, etc.  This guide addresses these special cases.
 
Vendors may need to have full control of their builds and keep source private due to licensing restrictions, etc.  This guide addresses these special cases.
  
== Add Repositories ==
+
The instructions on this page are being superseded by the scripts currently available here :
 +
  https://github.com/lbt/obs-setup
  
First, we have to add the openSUSE Tools repositories to the OBS servers
 
  
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.4/openSUSE:Tools.repo
+
== Pre-requisites ==
zypper ar http://repo.pub.meego.com/Mer:/Tools/openSUSE_11.4/Mer:Tools.repo
+
 
 +
* One or more physical or virtual machine with a minimal installation of OpenSuse 11.4 (12.1 is not recommended yet). It should be sysvinit based, not systemd.
 +
* Local DNS (or setup /etc/hosts on all relevant machines)
 +
* amd64 is normal but 686 should work
 +
 
 +
== Add Repositories ==
  
The following repo is needed for the API/WebUI server (possibly not once 2.3 is released)
+
Mer has a special version of OBS that it uses to enable SB2 and other
 +
features. Thes patches
  
zypper ar http://download.opensuse.org/repositories/devel:/languages:/ruby:/extensions/openSUSE_11.4/devel:languages:ruby:extensions.repo
+
First, we have to add the Mer OBS repository to the OBS servers
  
The following repo is needed until 2.3 is released
+
=== Mer OBS ===
  
  zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.4/openSUSE:Tools:Unstable.repo
+
  zypper ar http://repo.pub.meego.com//Mer:/OBS:/Testing/openSUSE_11.4/Mer:OBS:Testing.repo
  
To activate them:
+
To activate it:
  
 
  zypper refresh
 
  zypper refresh
Line 85: Line 89:
 
  zypper install obs-api mysql memcached apache2
 
  zypper install obs-api mysql memcached apache2
  
 +
If you installed a minimal openSUSE installation then python packages may conflict with the minimal configuration package patterns-openSUSE-minimal_base. This package can be safely removed if it causes dependency problems.
  
  chkconfig --add memcached obsapidelayed apache2 mysql
+
Remove runlevel 2 from the runlevels for mysql:
 +
  sed -i -e's/^# Default-Start:  2 3 5$/# Default-Start:  3 5/' /etc/init.d/mysql  
  
If you installed a minimal openSUSE installation then python packages may conflict with the minimal configuration package patterns-openSUSE-minimal_base. This package can be safely removed if it causes dependency problems.
+
to prevent the following error:
 +
insserv: FATAL: service network is missed in the runlevels 2 to use service mysql
 +
 
 +
Then:
 +
chkconfig --add memcached obsapidelayed apache2 mysql
  
 
=== MySQL Setup ===
 
=== MySQL Setup ===
Line 167: Line 177:
  
 
Only needs editing if you are using LDAP or if your backend is not on the same machine as the api (frontend) when you should change the SOURCE_HOST.
 
Only needs editing if you are using LDAP or if your backend is not on the same machine as the api (frontend) when you should change the SOURCE_HOST.
 +
 +
=== Set options.yml ===
 +
You may also need to edit /srv/www/obs/{webui,api}/config/options.yml to set
 +
; use_xforward: true
 +
: if you have a large setup and enabled X-Forward
 +
: NOTE: As of 4 Apr 2012 X-Forward is known not to work with the Mer build of OBS.
 +
: https://bugs.merproject.org/show_bug.cgi?id=278
 +
; new_user_registration: deny
 +
: If you use LDAP and create users elsewhere
 +
; source_server_url: http://src-*:5352
 +
: If you have setup the bs_srcsrver on another host
  
 
=== Setup apache for webui ===
 
=== Setup apache for webui ===
Line 191: Line 212:
 
  vi /etc/apache2/vhosts.d/obs.conf
 
  vi /etc/apache2/vhosts.d/obs.conf
  
The packages install to /srv/obs, not /obs as the appliance uses so change <VirtualHost *:82> to have
+
You also need to create the directory for the repos (TODO: What should be the permissions?)
  
  DocumentRoot "/srv/obs/repos/"
+
  mkdir -p /srv/obs/repos
 
+
chown -R obsrun.obsrun /srv/obs/repos
and
+
 
+
  <Directory /srv/obs/repos>
+
  
 
Also enable https by adding:
 
Also enable https by adding:
Line 208: Line 226:
 
  rcobsapidelayed start
 
  rcobsapidelayed start
 
  rcapache2 start
 
  rcapache2 start
 
  
 
At this point you should be able to open a browser and go to your server and login to OBS.  Normal https should be the webui, https on port 444 should be the api, and http on port 82 should be the repository directory.  The default username/password is Admin/opensuse. Admin user should be able to create a new project in the '''List of All Projects -> Create a new project''' and filling in a project name, title and description.
 
At this point you should be able to open a browser and go to your server and login to OBS.  Normal https should be the webui, https on port 444 should be the api, and http on port 82 should be the repository directory.  The default username/password is Admin/opensuse. Admin user should be able to create a new project in the '''List of All Projects -> Create a new project''' and filling in a project name, title and description.
Line 216: Line 233:
 
=== Install the Worker Packages ===
 
=== Install the Worker Packages ===
  
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools/openSUSE_11.4/openSUSE:Tools.repo
+
  zypper ar http://repo.pub.meego.com/Mer:/Tools:/Testing/openSUSE_11.4_standard/Mer:Tools:Testing.repo
  zypper ar http://repo.pub.meego.com/Mer:/Tools/openSUSE_11.4/Mer:Tools.repo
+
 
+
The following repo is needed until 2.3 is released
+
 
+
zypper ar http://download.opensuse.org/repositories/openSUSE:/Tools:/Unstable/openSUSE_11.4/openSUSE:Tools:Unstable.repo
+
zypper refresh
+
 
+
 
+
 
+
 
  zypper install obs-worker qemu
 
  zypper install obs-worker qemu
  
Line 234: Line 242:
 
  vi /etc/sysconfig/obs-worker
 
  vi /etc/sysconfig/obs-worker
  
Most values in this file are well commented. One that needs to be set for ARM builds is:
+
Most values in /etc/sysconfig/obs-server are well commented. One that needs to be set for ARM builds is:
  
 
  OBS_SCHEDULER_ARCHITECTURES="i586 x86_64 armv7el armv8el“
 
  OBS_SCHEDULER_ARCHITECTURES="i586 x86_64 armv7el armv8el“
Line 274: Line 282:
 
----
 
----
  
[[Category:OBS]]
+
[[Category:obs]]

Latest revision as of 13:56, 24 April 2012

These steps show how to set up a Mer Open Build Service (OBS) on an openSuSE 11.4 installation. An OBS installation will usually consist of an OBS web server, a main OBS server and one or more OBS workers. This document describes how to setup a basic evaluation installation and is not meant as production ready or hardened setup.

PLEASE NOTE : Most users won't need to install their own OBS and can just use an account on a publicly accessible OBS.

Vendors may need to have full control of their builds and keep source private due to licensing restrictions, etc. This guide addresses these special cases.

The instructions on this page are being superseded by the scripts currently available here :

 https://github.com/lbt/obs-setup


Contents

[edit] Pre-requisites

  • One or more physical or virtual machine with a minimal installation of OpenSuse 11.4 (12.1 is not recommended yet). It should be sysvinit based, not systemd.
  • Local DNS (or setup /etc/hosts on all relevant machines)
  • amd64 is normal but 686 should work

[edit] Add Repositories

Mer has a special version of OBS that it uses to enable SB2 and other features. Thes patches

First, we have to add the Mer OBS repository to the OBS servers

[edit] Mer OBS

zypper ar http://repo.pub.meego.com//Mer:/OBS:/Testing/openSUSE_11.4/Mer:OBS:Testing.repo

To activate it:

zypper refresh

When zypper asks, accept the trust key.

[edit] Setup OBS Backend Server

For high load situations this can be split over multiple machines:

  • Scheduler Server
  • Source Server
  • Repository Server
  • Signing Server

alternatively this can all be done on one backend server.

[edit] Install the Server packages

zypper install obs-server obs-signd

The repository server should have

chkconfig --add obsrepserver

The source server should have

chkconfig --add obssrcserver

The scheduler should have

chkconfig --add obsscheduler obsdispatcher obspublisher obswarden

Finally, on the signer

chkconfig --add obssigner

Configure access control to the reposerver : default is 127.0.0.1 Need to modify the ipaccess values. (see https://github.com/lbt/open-build-service/commit/477eeefb90c620333775c6be9b733329fbbdff7e for a fix that should make it upstream)

vi /usr/lib/obs/server/BSConfig.pm

On the repository server:

rcobsrepserver start

On the source server:

rcobssrcserver start

On the scheduler server

rcobsscheduler start
rcobsdispatcher start
rcobspublisher start
rcobswarden start

On the signer:

rcobssigner start

[edit] Setup OBS API/Webserver

zypper install obs-api mysql memcached apache2

If you installed a minimal openSUSE installation then python packages may conflict with the minimal configuration package patterns-openSUSE-minimal_base. This package can be safely removed if it causes dependency problems.

Remove runlevel 2 from the runlevels for mysql:

sed -i -e's/^# Default-Start:  2 3 5$/# Default-Start:  3 5/' /etc/init.d/mysql 

to prevent the following error:

insserv: FATAL: service network is missed in the runlevels 2 to use service mysql

Then:

chkconfig --add memcached obsapidelayed apache2 mysql

[edit] MySQL Setup

Start mysql so we can configure it:

rcmysql start

Secure the installation:

/usr/bin/mysql_secure_installation

Create the databases:

mysql -u root -p

At the mysql prompt create the databases and add the obs user to them (substitute ********** with your password):

create database api_production;
create database webui_production;
GRANT all privileges
     ON api_production.* 
     TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
GRANT all privileges
     ON webui_production.* 
     TO 'obs'@'%', 'obs'@'localhost' IDENTIFIED BY '************';
FLUSH PRIVILEGES;
quit

Now, configure OBS to use MySQL. First the API server:

vi /srv/www/obs/api/config/database.yml

To have:

     production:
       adapter: mysql
       database: api_production
       username: obs
       password: ************

And setup the webui to use MySQL:

vi /srv/www/obs/webui/config/database.yml
     production:
       adapter: mysql
       database: webui_production
       username: obs
       password: ************

Populate the database:

cd /srv/www/obs/api/
RAILS_ENV="production"  rake db:setup
cd /srv/www/obs/webui/
RAILS_ENV="production" rake db:setup

If this fails, check /srv/www/obs/api/config/environments/production.rb and /srv/www/obs/webui/config/environments/production.rb for configuration of the SOURCE_HOST and FRONTEND_HOST.

[edit] Setup Web UI Configuration

New edit the webui config:

vi /srv/www/obs/webui/config/environments/production.rb

and set:

FRONTEND_PORT = 444
FRONTEND_PROTOCOL = 'https'
DOWNLOAD_URL = "http://obs:82/"

[edit] Setup API Configuration

The api config:

/srv/www/obs/api/config/environments/production.rb

Only needs editing if you are using LDAP or if your backend is not on the same machine as the api (frontend) when you should change the SOURCE_HOST.

[edit] Set options.yml

You may also need to edit /srv/www/obs/{webui,api}/config/options.yml to set

use_xforward
true
if you have a large setup and enabled X-Forward
NOTE: As of 4 Apr 2012 X-Forward is known not to work with the Mer build of OBS.
https://bugs.merproject.org/show_bug.cgi?id=278
new_user_registration
deny
If you use LDAP and create users elsewhere
source_server_url
http://src-*:5352
If you have setup the bs_srcsrver on another host

[edit] Setup apache for webui

Edit the apache config:

vi /etc/sysconfig/apache2

Then add "passenger xforward headers" ( proxy? ) to the APACHE_MODULES= line

OBS and API uses SSL so you may need to create a certificate

mkdir -p /srv/obs/certs/
openssl genrsa -out /srv/obs/certs/server.key 2048
openssl req -new -x509 -key /srv/obs/certs/server.key -out /srv/obs/certs/server.crt -days 3650

And verify log directory ownership

chown -R wwwrun:www /srv/www/obs/webui/log/
chown -R wwwrun:www /srv/www/obs/api/log/

Edit the apache virtual host configuration. This can be tweaked a lot to setup sane named vhosts if you have DNS.

vi /etc/apache2/vhosts.d/obs.conf

You also need to create the directory for the repos (TODO: What should be the permissions?)

mkdir -p /srv/obs/repos
chown -R obsrun.obsrun /srv/obs/repos

Also enable https by adding:

 Listen 443

Now start the server:

rcmemcached start
rcobsapidelayed start
rcapache2 start

At this point you should be able to open a browser and go to your server and login to OBS. Normal https should be the webui, https on port 444 should be the api, and http on port 82 should be the repository directory. The default username/password is Admin/opensuse. Admin user should be able to create a new project in the List of All Projects -> Create a new project and filling in a project name, title and description.

[edit] Setup OBS Worker

[edit] Install the Worker Packages

zypper ar http://repo.pub.meego.com/Mer:/Tools:/Testing/openSUSE_11.4_standard/Mer:Tools:Testing.repo
zypper install obs-worker qemu

[edit] Configure OBS Worker

Edit the file /etc/sysconfig/obs-worker in order to point to correct repository server which is one of the main OBS server services. In this example the server is 192.168.1.1.

vi /etc/sysconfig/obs-worker

Most values in /etc/sysconfig/obs-server are well commented. One that needs to be set for ARM builds is:

OBS_SCHEDULER_ARCHITECTURES="i586 x86_64 armv7el armv8el“

[edit] Start OBS Worker

Start the worker service:

chkconfig --add obsworker
rcobsworker start


[edit] Troubleshooting

Note that logs live in:

  1. /srv/obs/log/ for the backend systems
  2. /srv/www/obs/api/log/ for the api and authentication interface
  3. /srv/www/obs/webui/log/ for the web interface

Here are common issues that have been seen and how to resolve them.

[edit] HTTP Error 500

Check /srv/www/obs/api/config/environments/production.rb and /srv/www/obs/webui/config/environments/production.rb for configuration of the SOURCE_HOST and FRONTEND_HOST.

[edit] 404 while trying to see the package repository

Check, if the /srv/obs/repos directory is owned by obsrun.obsrun - if not, chown it. Failing to do so will prevent publisher from publishing your repositories in repository directory, and when you try accessing them you will get the 404.

[edit] OBS Web Interface Error:

When you get the following error:

OBS Web Interface Error:
Error Details:

Errorcode: unknown
Message: <?xml version="1.0" encoding="UTF-8"?> <status code="unknown"> <summary>uncaught exception: Failed to establish connect

check all the modified OBS configuration files (.pm files) for missing semicolons (;). The trace should also be visible in /srv/obs/log/*.log files.


Personal tools