The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)

SB2

From Mer Wiki
(Difference between revisions)
Jump to: navigation, search
(→‎sb2-tools)
(→‎State)
Line 13: Line 13:
 
== State ==
 
== State ==
  
−
The setup can currently build 308 out of 320 packages in Mer Core
+
The setup is now in Mer production
  
 
== Steps ==
 
== Steps ==

Revision as of 09:09, 1 February 2012

Contents

SB2

In Mer, we're currently experimenting with integrating SB2 into OBS to provide a better and more flexible cross compilation method. This is instructions to help you get started testing/developing this project.

WARNING: this is in-development code and may cause damage

What does SB2-OBS do differently than current approach

When a SB2flags: configuration item is encountered, 'build' switches behaviour. When it is preinstalling packages, it will install packages listed in SB2install: to BUILD_ROOT and rest to BUILD_TARGET (typically /target). The packages in SB2install are typically a X86 tools/bootstrap, which includes SB2, QEMU and cross compilers.

Upon booting the VM, it will set up an abuild user and chown -R abuild /target. It will then set up a SB2 target in /target and begin to install RPM packages utilizing RPM.

State

The setup is now in Mer production

Steps

It is assumed you already have a functioning osc+build install on your computer.

Trying it out

In a KVM

  • You need to set up so you can do KVM builds
  • osc build --clean --no-verify --vm-type=kvm Core_armv7l armv7el

OBS project configuration settings

In the fork of 'build' and Carsten Munk's https://github.com/stskeeps/open-build-service OBS fork, it is now possible to do the following configuration settings 

 %ifarch armv7el
 SB2install: packages-to-be-installed-in-root
 SB2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm --debug
 %endif

The sb2flags available: 

 --toolchain /path/to/your/crosscompiler-gcc/in/tools
 --qemu /path/to/your/dynamic/qemu
 --installmode name-of-sb2-mode-used-for-buildsystem-setup
 --defaultmode name-of-sb2-mode-used-for-rpmbuild
 --debug, will enable SB2 debugging/logs

As an example, current home:Admin:sb2:noaccel project configuration is: 

 %ifarch armv7el
 sb2install: sb2-tools-arm
 sb2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm --debug
 %endif

SB2 enablement details of Mer Core

OBS server and clients

First off, deploy the SB2-OBS patches for obs-build, opensuse-build-service patches to OBS and osc and obs-build for developer's clients from http://github.com/stskeeps/

It is important to deploy obs-build -and- opensuse-build-service to your OBS backend as these are interconnected and the OBS is what publishes 'build' code to the OBS workers.

For opensuse-build-service, it might be easier simply to grab the patch from https://github.com/stskeeps/open-build-service/commit/d54b286f821954f0938fb19a1e7f710e66ebf5ae and patch it directly to your installed setup, in /usr/lib/obs/server.

Foundation

You must have a repository that contains the built RPMs from a project you intend to enable with SB2 cross compilation, both X86 and ARM packages.

In http://webui-ci.tspre.org , we have those in Core:i586 and Core:armv7l. But the ARM port currently is utilizing our old cross compilation approach. So we need to somehow remove that. For non-Mer projects this is probably not needed.

To do this, we have created Core:armv7l:noaccel project, with this meta configuration: 

 <project name="Core:armv7l:noaccel">
   <title></title>
   <description></description>
   <person role="maintainer" userid="Admin"/>
   <person role="bugowner" userid="Admin"/>
   <repository name="Core_armv7l">
     <arch>armv7el</arch>
     <arch>i586</arch>
   </repository>
  </project>

The reader may notice that we don't have any <path> directories in this repository. This is because we will be utilizing aggregatepac ( http://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#link_and_aggregate ) to transfer the binaries as-is from our Core:armv7l without having to drag in the project configurations from there.

For aggregatepac to work, you need to have similar <repository> as the origin project in the target project.

What I then did was: 

  osc -A http://api-ci.tspre.org ls Core:armv7l | xargs -L1 -Ixxx osc -A http://api-ci.tspre.org aggregatepac Core:armv7l xxx Core:armv7l:noaccel

Which will cause all the built RPMs from Core:armv7l to be represented in Core:armv7l:noaccel.

Then I created our Core:armv7l:sb2 repository with the following meta prj: 

 <project name="Core:armv7l:sb2">
   <title></title>
   <description></description>
   <person role="maintainer" userid="Admin"/>
   <person role="bugowner" userid="Admin"/>
   <build>
      <enable/>
   </build>
   <repository name="Core_armv7l">
     <path repository="Core_i586" project="Core:i586"/>
     <path repository="Core_armv7l" project="Core:armv7l:noaccel"/>
     <arch>armv7el</arch>
     <arch>i586</arch>
   </repository>
  </project>

This means, that when searching for RPMs for building, it will first look in Core_i586 repository of Core:i586 and then in Core_armv7l repository of Core:armv7l:noaccel.

You need to copy the Core:armv7l project configuration (see http://webui-ci.tspre.org/project/prjconf?project=Core%3Aarmv7l) and remove the cross enablement, and apply this to Core:armv7l:sb2 project configuration (osc meta prjconf -e Core:armv7l:sb2)

scratchbox2 package

Naturally we need a scratchbox2 package in the repository and it was uploaded with OBS (create package, upload sources), see it at http://webui-ci.tspre.org/package/show?package=scratchbox2&project=Core%3Aarmv7l%3Asb2

qemu-usermode

Compared to typical OBS setups where a static qemu-arm/etc binary is utilized, we are utilizing a dynamically linked qemu. This package can be seen at http://webui-ci.tspre.org/package/files?package=qemu-usermode&project=Core%3Aarmv7l%3Asb2

cross compiler

We need a cross compiler built, in Mer, this is done by the gcc source package, but has a cross-armv7l-gcc.spec. For other non-Mer projects this may be done differently, but you need to have a working cross compiler.

For this to work, you need to have the glibc and kernel headers from the ARM side exported into X86. In Mer, this is handled by these packages:

http://webui-ci.tspre.org/package/files?package=kernel-headers-arm&project=Core%3Aarmv7l%3Asb2

http://webui-ci.tspre.org/package/files?package=glibc-arm&project=Core%3Aarmv7l%3Asb2

http://webui-ci.tspre.org/package/files?package=glibc-devel-arm&project=Core%3Aarmv7l%3Asb2

http://webui-ci.tspre.org/package/files?package=glibc-headers-arm&project=Core%3Aarmv7l%3Asb2

The magic happens in baselibs.conf.

In Mer, we had to drop some bad dependencies in the cross-armv7l-gcc that linked to a certain version/release of libgcc/libmudflap, making it uninstallable.

fakeroot

Scratchbox2 relies on fakeroot, so this package should be installed into the tree, see http://webui-ci.tspre.org/package/files?package=fakeroot&project=Core%3Aarmv7l%3Asb2

It is very very very important that you use the -sysv variant, as this is much much faster.

sb2-tools

We have one package that is exported from X86 to ARM, containing the X86 tools, QEMU, SB2, cross compiler, etc.

This package can be seen at http://webui-ci.tspre.org/package/files?package=sb2-tools&project=Core%3Aarmv7l%3Asb2 and it basically takes the contents of RPM packages from the filesystem that it can read, and dumps them into one big RPM for tools install.

This package is then, for ARM/targets, installed into / through the use of the SB2install: package project configuration statement, whereas rest of packages are installed into /target

A possible evolution could be SB2install: being able to install packages from other architectures than the target, making the need for the sb2-tools package disappear.

The magic happens in the in-spec creation of baselibs.conf

glibc

SB2 requires some patches to glibc in order to function well, for X86 side. We include this package to the OBS project by a osc linkpac -C copy Core:armv7l glibc Core:armv7l:sb2 glibc , check it out and modify it with these patches:

For glibc 2.13/Mer, the patches are available at http://webui-ci.tspre.org/package/files?package=glibc&project=Core%3Aarmv7l%3Asb2 

 glibc-2.14.1-elf-rtld.c.1.diff
 glibc-2.14.1-ldso-nodefaultdirs-option.5.diff	 
 glibc-2.14.1-ldso-rpath-prefix-option.2.diff	  
 glibc-2.14.1-nscd-socket-location.4.diff	  
 glibc-2.14.1-nsswitchconf-location.3.diff

bringing in the rest of the sources

Now, go into Repositories tab of the core and disable i586 builds, and only enable (for i586) glibc/sb2-tools/fakeroot/cross-armv7l-gcc/scratchbox/qemu-usermode within their respective Repositories tabs inside their packages.

The reason for this that we'll be bringing in a lot of source packages we don't really want to also build for x86.

Go and add this to the Core:armv7l:sb2 project configuration (osc meta prj -e): 

 %ifarch armv7el
 sb2install: sb2-tools-arm
 sb2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm
 %endif


In Mer, we have all the source packages in Core:armv7l project. Now we want to bring them all in, as we want to start building :) 

 osc -A http://api.tspre.org ls Core:armv7l | xargs -L1 -Ixxx osc -A http://api.tspre.org linkpac -C copy Core:armv7l xxx Core:armv7l:sb2

Known issues

Coffee break

Enjoy while packages build :)

Retrieved from "https://wiki.merproject.org/wiki/index.php?title=SB2&oldid=884"
Personal tools