Latest revision as of 11:51, 22 October 2014

[edit] SB2

In Mer, we have switched to a better and more flexible cross compilation method. This is instructions to help you understand what changes has been made to the components in order to use this, and develop using it. The approach is not in any way Mer-specific and can be applied to other OBS-using distributions. This will only work with SB2 2.3_development branch

[edit] SB2 Documentation

If you are looking for more info, here are some relevant locations:

• SB2 git repository: https://maemo.gitorious.org/scratchbox2 (has a good README)
• Very detailed doc about internals: http://www.merproject.org/~lbt/SB2_internals_1st_ed_20120425.pdf

[edit] What does SB2-OBS do differently than the old approach

The result of the SB2-OBS work compared to the old approach is that it's much easier to maintain and easier to understand. The old approach was using 40+ injected x86 binaries / specific cross compilation acceleration packages which was a nightmare to maintain.

The new SB2-OBS approach also allows much easier porting to different architectures, as we don't have to add in every single cross acceleration package a new architecture.

The old approach was similar to what was developed in the MeeGo project for ARM cross compilation, injecting X86 binaries into the build environment. With SB2 we gain a sane separation between tools and target.

When a SB2flags: configuration item is encountered, 'build' switches behaviour. When it is preinstalling packages, it will install packages listed in SB2install: to BUILD_ROOT and rest to BUILD_TARGET (typically /target). The packages in SB2install are typically a X86 tools/bootstrap, which includes SB2, QEMU and cross compilers.

Upon booting the VM, it will set up an abuild user and chown -R abuild /target. It will then set up a SB2 target in /target and begin to install target RPM packages utilizing an accelerated X86 rpm package manager.

[edit] State

The setup is now in Mer production, get an account on http://webui-ci.tspre.org and view:

http://webui-ci.tspre.org/project/show?project=Core%3Aarmv6l

http://webui-ci.tspre.org/project/show?project=Core%3Aarmv7l

http://webui-ci.tspre.org/project/show?project=Core%3Aarmv7hl

which successfully compiles all the Mer core packages for these ARM architectures.

There has also been a succesful attempt with similar results for MIPS architecture, allowing it to boot up to qmlviewer within QEMU emulator without any reported/noticable errors, see the following image:

http://releases.merproject.org/~carsten/mer-on-mipsel-qemu.png

[edit] Steps

It is assumed you already have a functioning osc+build install on your computer.

• Install modified osc client
  • git clone git://github.com/stskeeps/osc.git
  • cd osc
  • (as root) python setup.py install --prefix=/usr
  • If you are on ubuntu, you might need to install into dist-packages instead of site-packages
• Install modified 'build'
  • git clone git://github.com/stskeeps/obs-build.git
  • cd obs-build
  • (as root) make install

[edit] Trying it out

• Register an account on http://webui-ci.tspre.org
• osc -A http://api-ci.tspre.org ls
• (type in your username/password)
• osc -A http://api-ci.tspre.org checkout Core:armv7l acl
• cd Core:armv7l/acl
• osc build --clean --no-verify Core_armv7l armv7el

[edit] In a KVM

• You need to set up so you can do KVM builds
• osc build --clean --no-verify --vm-type=kvm Core_armv7l armv7el

[edit] OBS project configuration settings

In the fork of 'build' and Carsten Munk's https://github.com/stskeeps/open-build-service OBS fork, it is now possible to do the following configuration settings

 %ifarch armv7el
 SB2install: packages-to-be-installed-in-root
 SB2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm --debug
 %endif

The sb2flags available:

 --toolchain /path/to/your/crosscompiler-gcc/in/tools
 --qemu /path/to/your/dynamic/qemu
 --installmode name-of-sb2-mode-used-for-buildsystem-setup
 --defaultmode name-of-sb2-mode-used-for-rpmbuild
 --debug, will enable SB2 debugging/logs

As an example, current home:Admin:sb2:noaccel project configuration is:

 %ifarch armv7el
 sb2install: sb2-tools-arm
 sb2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm --debug
 %endif

[edit] SB2 enablement details of Mer Core

[edit] Current setup

QEMU usermode emulation, dynamic and statically linked binaries: http://gitweb.merproject.org/gitweb?p=mer-core/qemu-usermode.git;a=tree

Scratchbox2 packaging: http://gitweb.merproject.org/gitweb?p=mer-core/scratchbox2.git;a=tree

Fakeroot packaging: http://gitweb.merproject.org/gitweb?p=mer-core/fakeroot.git;a=tree

SB2-Tools cross compilation tools set: http://gitweb.merproject.org/gitweb?p=mer-crosshelpers/sb2-tools-template.git;a=tree

Enablement configuration:

http://gitweb.merproject.org/gitweb?p=mer/project-core.git;a=blob;f=crosshelpers/_config;h=8c232dce8b2c3b23030f1b821c4cb0f5c3269c41;hb=HEAD (see lines 5-8, rest is old cross approach and will be removed)

http://gitweb.merproject.org/gitweb?p=mer/project-core.git;a=blob;f=armv6l/_config.in;h=348b4a0636ee84b4cac37573d8acaa5161c32280;hb=HEAD - ARMv6 enablement configuration

[edit] OBS server and clients

First off, deploy the SB2-OBS patches for obs-build, opensuse-build-service patches to OBS and osc and obs-build for developer's clients from http://github.com/stskeeps/

It is important to deploy obs-build -and- opensuse-build-service to your OBS backend as these are interconnected and the OBS is what publishes 'build' code to the OBS workers.

For opensuse-build-service, it might be easier simply to grab the patch from https://github.com/stskeeps/open-build-service/commit/d54b286f821954f0938fb19a1e7f710e66ebf5ae and patch it directly to your installed setup, in /usr/lib/obs/server.

[edit] Foundation

You must have a repository that contains the built RPMs from a project you intend to enable with SB2 cross compilation, both X86 and ARM packages.

In http://webui-ci.tspre.org , we have those in Core:i586 and Core:armv7l. But the ARM port currently is utilizing our old cross compilation approach. So we need to somehow remove that. For non-Mer projects this is probably not needed.

To do this, we have created Core:armv7l:noaccel project, with this meta configuration:

 <project name="Core:armv7l:noaccel">
   <title></title>
   <description></description>
   <person role="maintainer" userid="Admin"/>
   <person role="bugowner" userid="Admin"/>
   <repository name="Core_armv7l">
     <arch>armv7el</arch>
     <arch>i586</arch>
   </repository>
  </project>

The reader may notice that we don't have any <path> directories in this repository. This is because we will be utilizing aggregatepac ( http://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#link_and_aggregate ) to transfer the binaries as-is from our Core:armv7l without having to drag in the project configurations from there.

For aggregatepac to work, you need to have similar <repository> as the origin project in the target project.

What I then did was:

  osc -A http://api-ci.tspre.org ls Core:armv7l | xargs -L1 -Ixxx osc -A http://api-ci.tspre.org aggregatepac Core:armv7l xxx Core:armv7l:noaccel

Which will cause all the built RPMs from Core:armv7l to be represented in Core:armv7l:noaccel.

Then I created our Core:armv7l:sb2 repository with the following meta prj:

 <project name="Core:armv7l:sb2">
   <title></title>
   <description></description>
   <person role="maintainer" userid="Admin"/>
   <person role="bugowner" userid="Admin"/>
   <build>
      <enable/>
   </build>
   <repository name="Core_armv7l">
     <path repository="Core_i586" project="Core:i586"/>
     <path repository="Core_armv7l" project="Core:armv7l:noaccel"/>
     <arch>armv7el</arch>
     <arch>i586</arch>
   </repository>
  </project>

This means, that when searching for RPMs for building, it will first look in Core_i586 repository of Core:i586 and then in Core_armv7l repository of Core:armv7l:noaccel.

You need to copy the Core:armv7l project configuration (see http://webui-ci.tspre.org/project/prjconf?project=Core%3Aarmv7l) and remove the cross enablement, and apply this to Core:armv7l:sb2 project configuration (osc meta prjconf -e Core:armv7l:sb2)

[edit] scratchbox2 package

Naturally we need a scratchbox2 package in the repository and it was uploaded with OBS (create package, upload sources), see it at http://webui-ci.tspre.org/package/show?package=scratchbox2&project=Core%3Aarmv7l%3Asb2

[edit] qemu-usermode

Compared to typical OBS setups where a static qemu-arm/etc binary is utilized, we are utilizing a dynamically linked qemu. This package can be seen at http://webui-ci.tspre.org/package/files?package=qemu-usermode&project=Core%3Aarmv7l%3Asb2

[edit] cross compiler

We need a cross compiler built, in Mer, this is done by the gcc source package, but has a cross-armv7l-gcc.spec. For other non-Mer projects this may be done differently, but you need to have a working cross compiler.

For this to work, you need to have the glibc and kernel headers from the ARM side exported into X86. In Mer, this is handled by these packages:

http://webui-ci.tspre.org/package/files?package=kernel-headers-arm&project=Core%3Aarmv7l%3Asb2

http://webui-ci.tspre.org/package/files?package=glibc-arm&project=Core%3Aarmv7l%3Asb2

http://webui-ci.tspre.org/package/files?package=glibc-devel-arm&project=Core%3Aarmv7l%3Asb2

http://webui-ci.tspre.org/package/files?package=glibc-headers-arm&project=Core%3Aarmv7l%3Asb2

The magic happens in baselibs.conf.

In Mer, we had to drop some bad dependencies in the cross-armv7l-gcc that linked to a certain version/release of libgcc/libmudflap, making it uninstallable.

[edit] fakeroot

Scratchbox2 relies on fakeroot, so this package should be installed into the tree, see http://webui-ci.tspre.org/package/files?package=fakeroot&project=Core%3Aarmv7l%3Asb2

It is very very very important that you use the -sysv variant, as this is much much faster.

[edit] sb2-tools

We have one package that is exported from X86 to ARM, containing the X86 tools, QEMU, SB2, cross compiler, etc.

This package can be seen at http://webui-ci.tspre.org/package/files?package=sb2-tools&project=Core%3Aarmv7l%3Asb2 and it basically takes the contents of RPM packages from the filesystem that it can read, and dumps them into one big RPM for tools install.

This package is then, for ARM/targets, installed into / through the use of the SB2install: package project configuration statement, whereas rest of packages are installed into /target

A possible evolution could be SB2install: being able to install packages from other architectures than the target, making the need for the sb2-tools package disappear.

The magic happens in the in-spec creation of baselibs.conf

[edit] glibc

SB2 requires some patches to glibc in order to function well, for X86 side. We include this package to the OBS project by a osc linkpac -C copy Core:armv7l glibc Core:armv7l:sb2 glibc , check it out and modify it with these patches:

For glibc 2.13/Mer, the patches are available at http://webui-ci.tspre.org/package/files?package=glibc&project=Core%3Aarmv7l%3Asb2

 glibc-2.14.1-elf-rtld.c.1.diff
 glibc-2.14.1-ldso-nodefaultdirs-option.5.diff	 
 glibc-2.14.1-ldso-rpath-prefix-option.2.diff	  
 glibc-2.14.1-nscd-socket-location.4.diff	  
 glibc-2.14.1-nsswitchconf-location.3.diff

[edit] bringing in the rest of the sources

Now, go into Repositories tab of the core and disable i586 builds, and only enable (for i586) glibc/sb2-tools/fakeroot/cross-armv7l-gcc/scratchbox/qemu-usermode within their respective Repositories tabs inside their packages.

The reason for this that we'll be bringing in a lot of source packages we don't really want to also build for x86.

Go and add this to the Core:armv7l:sb2 project configuration (osc meta prj -e):

 %ifarch armv7el
 sb2install: sb2-tools-arm
 sb2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm
 %endif

In Mer, we have all the source packages in Core:armv7l project. Now we want to bring them all in, as we want to start building :)

 osc -A http://api.tspre.org ls Core:armv7l | xargs -L1 -Ixxx osc -A http://api.tspre.org linkpac -C copy Core:armv7l xxx Core:armv7l:sb2

[edit] Roadmap

• To enable autoconf and automake (autoreconf as well) acceleration within Mer core, proved to work in sample runs
• Remove old cross compilation approach + packages from Mer
• Provide automatically built OBS packages (of OBS itself) with SB2-OBS patches included, for deployment on Community OBS and on vendor OBSes. This has been a bit delayed due to build.opensuse.org disk failure.
• Provide additional statistics showing performance differences between old cross approach and SB2-OBS, on this page

[edit] Coffee break

Enjoy while packages build :)