The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)
SB2
Contents |
SB2
In Mer, we're currently experimenting with integrating SB2 into OBS to provide a better and more flexible cross compilation method. This is instructions to help you get started testing/developing this project.
WARNING: this is in-development code and may cause damage
What does SB2-OBS do differently than current approach
When a SB2flags: configuration item is encountered, 'build' switches behaviour. When it is preinstalling packages, it will install packages listed in SB2install: to BUILD_ROOT and rest to BUILD_TARGET (typically /target). The packages in SB2install are typically a X86 tools/bootstrap, which includes SB2, QEMU and cross compilers.
Upon booting the VM, it will set up an abuild user and chown -R abuild /target. It will then set up a SB2 target in /target and begin to install RPM packages utilizing RPM.
State
The setup can currently build 308 out of 320 packages in Mer Core
Steps
It is assumed you already have a functioning osc+build install on your computer.
- Install modified osc client
- git clone git://github.com/stskeeps/osc.git
- cd osc
- (as root) python setup.py install --prefix=/usr
- If you are on ubuntu, you might need to install into dist-packages instead of site-packages
- Install modified 'build'
- git clone git://github.com/stskeeps/obs-build.git
- cd obs-build
- (as root) make install
Trying it out
- Register an account on http://webui-ci.tspre.org
- osc -A http://api-ci.tspre.org ls
- (type in your username/password)
- osc -A http://api-ci.tspre.org checkout Core:armv7l:sb2 acl
- cd Core:armv7l:sb2/acl
- osc build --clean --no-verify Core_armv7l armv7el
In a KVM
- You need to set up so you can do KVM builds
- osc build --clean --no-verify --vm-type=kvm Core_armv7l armv7el
OBS project configuration settings
In the fork of 'build' and Carsten Munk's https://github.com/stskeeps/open-build-service OBS fork, it is now possible to do the following configuration settings
%ifarch armv7el SB2install: packages-to-be-installed-in-root SB2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm --debug %endif
The sb2flags available:
--toolchain /path/to/your/crosscompiler-gcc/in/tools --qemu /path/to/your/dynamic/qemu --installmode name-of-sb2-mode-used-for-buildsystem-setup --defaultmode name-of-sb2-mode-used-for-rpmbuild --debug, will enable SB2 debugging/logs
As an example, current home:Admin:sb2:noaccel project configuration is:
%ifarch armv7el sb2install: sb2-tools-arm sb2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm --debug %endif
SB2 enablement details of Mer Core
OBS server and clients
First off, deploy the SB2-OBS patches for obs-build, opensuse-build-service patches to OBS and osc and obs-build for developer's clients from http://github.com/stskeeps/
It is important to deploy obs-build -and- opensuse-build-service to your OBS backend as these are interconnected and the OBS is what publishes 'build' code to the OBS workers.
For opensuse-build-service, it might be easier simply to grab the patch from https://github.com/stskeeps/open-build-service/commit/d54b286f821954f0938fb19a1e7f710e66ebf5ae and patch it directly to your installed setup, in /usr/lib/obs/server.
Foundation
You must have a repository that contains the built RPMs from a project you intend to enable with SB2 cross compilation, both X86 and ARM packages.
In http://webui-ci.tspre.org , we have those in Core:i586 and Core:armv7l. But the ARM port currently is utilizing our old cross compilation approach. So we need to somehow remove that. For non-Mer projects this is probably not needed.
To do this, we have created Core:armv7l:noaccel project, with this meta configuration:
<project name="Core:armv7l:noaccel"> <title></title> <description></description> <person role="maintainer" userid="Admin"/> <person role="bugowner" userid="Admin"/> <repository name="Core_armv7l"> <arch>armv7el</arch> <arch>i586</arch> </repository> </project>
The reader may notice that we don't have any <path> directories in this repository. This is because we will be utilizing aggregatepac ( http://en.opensuse.org/openSUSE:Build_Service_Tips_and_Tricks#link_and_aggregate ) to transfer the binaries as-is from our Core:armv7l without having to drag in the project configurations from there.
For aggregatepac to work, you need to have similar <repository> as the origin project in the target project.
What I then did was:
osc -A http://api-ci.tspre.org ls Core:armv7l | xargs -L1 -Ixxx osc -A http://api-ci.tspre.org aggregatepac Core:armv7l xxx Core:armv7l:noaccel
Which will cause all the built RPMs from Core:armv7l to be represented in Core:armv7l:noaccel.
Then I created our Core:armv7l:sb2 repository with the following meta prj:
<project name="Core:armv7l:sb2"> <title></title> <description></description> <person role="maintainer" userid="Admin"/> <person role="bugowner" userid="Admin"/> <build> <enable/> </build> <repository name="Core_armv7l"> <path repository="Core_i586" project="Core:i586"/> <path repository="Core_armv7l" project="Core:armv7l:noaccel"/> <arch>armv7el</arch> <arch>i586</arch> </repository> </project>
This means, that when searching for RPMs for building, it will first look in Core_i586 repository of Core:i586 and then in Core_armv7l repository of Core:armv7l:noaccel.
You need to copy the Core:armv7l project configuration (see http://webui-ci.tspre.org/project/prjconf?project=Core%3Aarmv7l) and remove the cross enablement, and apply this to Core:armv7l:sb2 project configuration (osc meta prjconf -e Core:armv7l:sb2)
scratchbox2 package
Naturally we need a scratchbox2 package in the repository and it was uploaded with OBS (create package, upload sources), see it at http://webui-ci.tspre.org/package/show?package=scratchbox2&project=Core%3Aarmv7l%3Asb2
qemu-usermode
Compared to typical OBS setups where a static qemu-arm/etc binary is utilized, we are utilizing a dynamically linked qemu. This package can be seen at http://webui-ci.tspre.org/package/files?package=qemu-usermode&project=Core%3Aarmv7l%3Asb2
cross compiler
We need a cross compiler built, in Mer, this is done by the gcc source package, but has a cross-armv7l-gcc.spec. For other non-Mer projects this may be done differently, but you need to have a working cross compiler.
For this to work, you need to have the glibc and kernel headers from the ARM side exported into X86. In Mer, this is handled by these packages:
http://webui-ci.tspre.org/package/files?package=kernel-headers-arm&project=Core%3Aarmv7l%3Asb2
http://webui-ci.tspre.org/package/files?package=glibc-arm&project=Core%3Aarmv7l%3Asb2
http://webui-ci.tspre.org/package/files?package=glibc-devel-arm&project=Core%3Aarmv7l%3Asb2
http://webui-ci.tspre.org/package/files?package=glibc-headers-arm&project=Core%3Aarmv7l%3Asb2
The magic happens in baselibs.conf.
In Mer, we had to drop some bad dependencies in the cross-armv7l-gcc that linked to a certain version/release of libgcc/libmudflap, making it uninstallable.
fakeroot
Scratchbox2 relies on fakeroot, so this package should be installed into the tree, see http://webui-ci.tspre.org/package/files?package=fakeroot&project=Core%3Aarmv7l%3Asb2
It is very very very important that you use the -sysv variant, as this is much much faster.
sb2-tools
We have one package that is exported from X86 to ARM, containing the X86 tools, QEMU, SB2, cross compiler, etc.
This package can be seen at http://webui-ci.tspre.org/package/files?package=sb2-tools&project=Core%3Aarmv7l%3Asb2
This package is then, for ARM/targets, installed into / through the use of the SB2install: package project configuration statement, whereas rest of packages are installed into /target
A possible evolution could be SB2install: being able to install packages from other architectures than the target, making the need for the sb2-tools package disappear.
glibc
SB2 requires some patches to glibc in order to function well, for X86 side. We include this package to the OBS project by a osc linkpac -C copy Core:armv7l glibc Core:armv7l:sb2 glibc , check it out and modify it with these patches:
For glibc 2.13/Mer, the patches are available at http://webui-ci.tspre.org/package/files?package=glibc&project=Core%3Aarmv7l%3Asb2
glibc-2.14.1-elf-rtld.c.1.diff glibc-2.14.1-ldso-nodefaultdirs-option.5.diff glibc-2.14.1-ldso-rpath-prefix-option.2.diff glibc-2.14.1-nscd-socket-location.4.diff glibc-2.14.1-nsswitchconf-location.3.diff
bringing in the rest of the sources
Now, go into Repositories tab of the core and disable i586 builds, and only enable (for i586) glibc/sb2-tools/fakeroot/cross-armv7l-gcc/scratchbox/qemu-usermode within their respective Repositories tabs inside their packages.
The reason for this that we'll be bringing in a lot of source packages we don't really want to also build for x86.
Go and add this to the Core:armv7l:sb2 project configuration (osc meta prj -e):
%ifarch armv7el sb2install: sb2-tools-arm sb2flags: --toolchain /opt/cross/bin/armv7l-meego-linux-gnueabi-gcc --qemu /usr/bin/qemu-arm %endif
In Mer, we have all the source packages in Core:armv7l project. Now we want to bring them all in, as we want to start building :)
osc -A http://api.tspre.org ls Core:armv7l | xargs -L1 -Ixxx osc -A http://api.tspre.org linkpac -C copy Core:armv7l xxx Core:armv7l:sb2
Known issues
- 'cc' doesn't work right now, but 'gcc' does
- RPM packages cannot contain device nodes, such as 'filesystem' used to in Mer, see changes in http://webui-ci.tspre.org/package/files?package=filesystem&project=Core%3Aarmv7l
Coffee break
Enjoy while packages build :)