Revision as of 10:45, 20 August 2012

The platform SDK aims to offer a installable disk image that can be run on virtual machines (typically) or actual hardware, which contains tools like Scratchbox2, MIC (image creator), spectacle, osc, qemu, etc preinstalled, to make it easier for a developer to work with Mer.

In a hurry? Here's the tl;dr; version :

cd $HOME; curl -k -O https://img.merproject.org/images/sdk/mer-sb2sdk-i486-chroot-latest.tar.bz2
sudo mkdir -p /srv/mer/sdks/sdk
cd /srv/mer/sdks/sdk
sudo tar --numeric-owner -p -xjf $HOME/mer-sb2sdk-i486-chroot-latest.tar.bz2
/srv/mer/sdks/sdk/mer-sdk-chroot mount
/srv/mer/sdks/sdk/mer-sdk-chroot enter

Read on for details, WARNINGS, pre-requisites etc...

Mer platform SDK

To avoid any confusion, there are two types of SDKs: Application SDKs and Platform SDKs. Application SDK's are the ones that deal with Qt applications and libraries, HTML5, etc, while Platform SDK's are for developing the platform, compiling native code and other things revolving around the platform developer/hacker needs.

The platform SDK aims to offer a installable disk image that can be run on virtual machines (typically) or actual hardware, which contains tools like Scratchbox2, MIC (image creator), spectacle, osc, qemu, etc preinstalled, to make it easier for a developer to work with Mer.

You may also like to contribute to developing the SDK.

Using the platform SDK

SDK Requirements

The chroot SDK will run on most modern linux machines. It needs:

Linux distribution (one in a virtual machine works well)
about 400Mb free space to install
hundreds of Mb for rpm caches for osc and mic
Generic x86 CPU
user must have sudo rights

WARNING

The SDK consists of a directory tree and when you 'mount' the SDK you will link your home directory, root directory and any data directories within this tree.

Because of this you *MUST* 'umount' the SDK before deleting it - if you don't then you *WILL* lose data.

If you are unfamiliar with bind mounts, the safest way to uninstall is to reboot your machine and remove the SDK *BEFORE* using the sdk 'mount' command.

Please do not be alarmed by this - we're simply letting you know that running an "rm -rf" command in the wrong place is dangerous. It's better to explain this in big letters than risk someone losing data by mistake :)

Obviously people working on SDK development and testing should take particular care.

Installation / setup

The platform SDK is provided as a rootfs tarball that contains essential tools for Mer platform development along with a helper script to enter the rootfs.

Filesystem requirements:

The SDK can be installed to any location with enough space - we'll use /srv/ as per the Linux FHS (feel free to adapt the commands to use any other location).
The installation path must contain an intermediate directory called 'sdks' which only has Mer SDKs inside. eg /srv/mer/sdks/sdk/...
The SDK must not be on a nosuid filesystem - note automounted usb drives typically have this set

To setup the SDK:

Download the latest stable SDK rootfs tarball.

cd $HOME
curl -k -O https://img.merproject.org/images/sdk/mer-sb2sdk-i486-chroot-latest.tar.bz2

Create directory for the SDK rootfs and extract the tarball as root or with sudo

sudo mkdir -p /srv/mer/sdks/sdk
cd /srv/mer/sdks/sdk
sudo tar --numeric-owner -p -xjf $HOME/mer-sb2sdk-i486-chroot-latest.tar.bz2

SDK control script

The platform SDK rootfs contains a helper script to enter the chroot named 'mer-sdk-chroot'. The helper script is located the the root directory (/) of the rootfs. It requires you to have sudo ability.

As mentioned, the SDK is location independent so it uses the location of the helper script to determine which SDK to enter.

Connect the SDK using 'mount'

You need to connect the SDK before using it. This only needs to be done once (well, once each time you boot your machine) and it connects things like /proc and $HOME in the SDK. It also links /parentroot of the SDK to your normal / directory.

Under normal use the SDK does not need to be disconnected. As per the warning above, you should disconnect using 'umount' before removing the SDK (see remove/disconnect info below)

To connect run:

/srv/mer/sdks/sdk/mer-sdk-chroot mount

You can now use the SDK from multiple terminals at once.

Entering chroot

Before entering the SDK you may want to make an SDK equivalent of ".profile" to give you a nice prompt to remind you that you are in the SDK:

echo 'PS1="MerSDK $PS1"' >> ~/.mersdk.profile

To enter the rootfs with the helper script run

/srv/mer/sdks/sdk/mer-sdk-chroot enter

You should find that you are operating under your normal username and that your home directory is available as /home/<username> and any other mountpoints are mounted under /parentroot/*

You have sudo rights automatically. If sudo fails within the sdk, make sure that the filesystem the sdk is on is not mounted with the "nosuid" parameter. "mount" on the host system gives you this information, add "suid" as parameter in fstab, if necessary.

Basic tasks

Building an image

Mer images are build using mic image creator. Mic takes a kickstart file that defines the image to be created as input. To create a Mer image with the platform SDK simply call mic as you would do on any other system. For example

As a test this should make a new local SDK rootfs image:

Check: https://img.merproject.org/images/sdk/latest/images/mer-sb2sdk-i486-chroot/ for the name of the latest KickStart file and use it in the first command below:

curl -k -o mer-sdk-x86-chroot-latest.ks <INSERT URL OF LATEST KICKSTART HERE>
sudo mic create fs mer-sdk-x86-chroot-latest.ks -o . --pkgmgr=yum --arch i486 --compress-disk-image=tar.bz2

If you get an error through the build process like so:

    MerSDK sivan@truck:~/MER$ sudo mic create raw nemo.ks -o nemo.raw --pkgmgr=yum --arch i686
    Info: Retrieving repo metadata:
    Info: Retrieving 8767e9370d0c347ce2fb1add49e0e06a63b8fc7a7ad19a8c7f1ab58effc193b4-group.xml.gz ... DONE
    Info: Loading dm_snapshot...
    Error <mount>: Error creating ext3 filesystem on disk /dev/loop101
    Exception AttributeError: "'NoneType' object has no attribute 'px_proxy_factory_free'" in <bound method ProxyFactory.__del__ of <libproxy.ProxyFactory object at     0xa00924c>> ignored

Then, exit the SDK chroot , do this:

    sudo rm /etc/mtab; sudo ln -s /proc/self/mounts /etc/mtab

Then, re-enter the SDK chroot and retry image building.

Running osc

(This section needs to be improved)

Since your home directory has been linked into the SDK then osc commands should work as normal.

osc -A API_URL ls

Where API_URL can be:

 https://api.pub.meego.com     MeeGo community OBS
 https://api.ci.merproject.org   Mer's continous integration OBS (not for regular use)

or the API url for your own, or your company's OBS.

Compiling with the SDK

The SDK is available with Scratchbox2; it is pre-installed in some images or can be installed into a minimal SDK by adding a the Mer 'cross-tools' repository and installing a compiler suitable for your project.

Once the SDK is setup, see the documentation on using the Platform SDK and SB2

Packaging

The SDK includes tools needed to create rpm packages for Mer.

See https://build.pub.meego.com/package/show?package=sdk-kickstarter-configs&project=Mer%3ATools%3ATesting

Hosting repositories

To make installing created packages easier you can host your own rpm repository with the SDK and install packages to your target device from there.

TODO

SDK contents

The platform SDK's rootfs has some useful packages preinstalled that are listed below. If you're missing something you can use zypper to install the needed extra packages or if not available in the repositories compile them yourself.

sb2 cross compilers
osc and build
mic
spectacle
vim

Adding New Tools

New tools can be installed using

sudo zypper in <package>

Upgrading the SDK

Don't just do zypper up!

The SDK image is (as of 5 June 2012) pinned to specific static repositories and upgrades are carried out using "sdk-upgrade" which allows the repos to be updated to latest, next or set to point to any Mer Core/SDK release pairing.

Upgrading Older SDKs

(ie one without sdk-upgrade binary)

You may be prompted to remove the unstable Tools:Testing repo - and you really should as that's now going to be used by the Tools team for sharing experimental packages. Then follow the steps indicated.

Old SDKs which don't have sdk-upgrade should execute these steps before moving on to the appropriate upgrade process:

sudo zypper ref
sudo zypper in sdk-chroot
# There is now an sdk-upgrade binary

Special 6.1.0 Upgrade

This release needs to install the correct script and then exit/re-enter the SDK:

sudo sdk-upgrade --sdk 6.0.2  # <-- this hardcode will be removed
sudo zypper ref
sudo zypper in sdk-chroot
# Now you have the latest SDK scripts which will clean up some old issues
exit

# unmount, re-mount and re-enter the SDK to clean up
/srv/mer/sdks/sdk/mer-sdk-chroot umount
/srv/mer/sdks/sdk/mer-sdk-chroot mount
/srv/mer/sdks/sdk/mer-sdk-chroot enter

Now move on to the normal upgrade

Normal Upgrade

Simply run:

sudo sdk-upgrade --sdk 6.0.2  # <-- this hardcode will be removed

and you will then be told what steps to take to upgrade (usually just zypper ref/up).

Documentation for sdk-upgrade

    usage: sdk-upgrade [--next] [--core <release>] [--sdk <release>]

    Update the zypper repos to point to the latest, next or specified
    Mer / SDK releases. It does not run zypper ref or zypper up.

    By default checks
       http://releases.merproject.org/releases/latest-release
       http://repo.pub.meego.com/releases/Mer-Tools/latest-release

    --next : Check for the 'next' release instead of 'latest'

    --core <release> : Use Mer Core <release> instead of checking for latest/next
                       eg: --core 0.20120517.1
    --sdk  <release> : Use SDK <release> instead of checking for latest/next
                       eg: --sdk 0.5.1

Advanced details

Documentation for mer-sdk-chroot

The usage for the helper script is:

   usage: mer-sdk-chroot [<enter>] [-u <user>] [-r <SDK root path>]
          mer-sdk-chroot mount [-u <user>] [-m <all|none|root|home>] [-r <SDK root path>]
          mer-sdk-chroot umount [-r <SDK root path>]

      This is the Mer chroot SDK.
      For information see http://wiki.merproject.org/wiki/Platform_SDK

      The SDK has 3 commands:
         enter (default)
         mount
         umount

     enter
        Used to enter the SDK and begin working. The SDK bash shell is a
        login shell. See below for .profile handling
        Must be preceded by a 'mount' to setup the SDK.
        May be used in multiple terminals and simply enters the
        chroot

     mount
        Used to setup the SDK mountpoints. Only needed once per login
        session

     umount
        Used to clean up the SDK mountpoints before removing the SDK.

     Options:

      -u  System user to link into SDK (not needed if using sudo)
      -m  Devices to bind mount from host: none, all (default)
          root, home
      -r The root of the SDK to use - normally derived from the
         pathname of mer-sdk-chroot

     Profile

     Entering the SDK runs the user's normal .profile and any (SDK)
     system profile entries. It will not execute the host's system
     profile entries.

     The environment variable MERSDK is set to 1 to allow .profile to
     detect the SDK.

     If the user has a ~/.mersdk.profile then it is sourced after the
     normal .profile handling (this allows the common use case of
     setting a profile to be handled).

     Hooks

     If the user specified has a .mersdkrc in their $HOME, it will be
     sourced to allow hook functions to be defined. Hooks are run as
     root. No commands should be executed immediately.

     These hooks are usually used to define symbolic links from any
     /parentroot/data type filesystems into the SDK root to setup
     system specific shared caches or filesystem layouts etc

Sharing mic and osc caches

You can use the ~/.mersdkrc file to check and link the standard mic and osc cache directories to those in parentroot.

Note that multiple *concurrent* instances of mic should not share the same cache (it's OK to share from different SDKs and different terminals provided they're not running and accessing the cache at the same time).

Here's some code you can use in ~/.mersdkrc to share a mic cache:

   if [[ ! -e ${sdkroot}/var/tmp/mic ]]; then
       mkdir -p ${sdkroot}/var/tmp/mic
   fi
   if [[ ! -e ${sdkroot}/var/tmp/mic/cache && ! -h ${sdkroot}/var/tmp/mic/cache  ]]; then
       ln -s /parentroot/var/tmp/mic/cache ${sdkroot}/var/tmp/mic/cache
   fi

In your ~/.oscrc file there is a line:

 packagecachedir = <something>/build-pkg-cache

If this is under $HOME then it will be shared anyway; otherwise do something similar to the mic cache code above.

Multiple SDKs

You should be able to install and connect to multiple SDKs at the same time.

Removing and disconnecting the SDK

Under normal use the SDK does not need to be disconnected; it's only usually needed when you want to remove an instance of the SDK.

To disconnect run:

/srv/mer/sdks/sdk/mer-sdk-chroot umount

Be sure that the bind-mounts(especialy the home directory) are all umounted after you disconnect the SDK. You can use 'cat /proc/self/mountstats' to check it. If it is not umounted cleanly, you should umount it manually after you exit the chroot environment.

The SDK does warn you if umounts fail. It also uses 'lazy' umounts which means that the mountpoints will go away as soon as the kernel can clean them (typically when processes exit or terminals cd out of the mounted directories.)

Once the SDK has been cleanly unmounted, it may be removed manually.

Note that the mountpoint:

 /srv/mer/sdks

will stay in the mount list all the time - this is intentional. Also note that it can sometimes appear to show a mount to the same device as another mountpoint (eg /) depending on your distro's configuration. (See bug 328 for details)

Platform SDK