The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)

Nemo/Development

From Mer Wiki
< Nemo(Difference between revisions)
Jump to: navigation, search
(→‎Setting up a webhook to github)
(→‎Changelogs: Examples for bug references)
 
(42 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 +
= Repositories =
 +
 +
Nemo Mobile has three organisations in the github: ''nemomobile'', ''nemobile-ux'', and ''nemobile-packages''.
 +
 +
''nemomobile'' contains packages/components/configs/etc. where Nemo is the upstream project (''nemomobile-ux'' will contain all Glacier UX packages). This organization can be found from https://github.com/nemomobile Each repo project contains packaging information which is using the [[Nemo/Development#With_tar_git_service|tar_git service]]
 +
 +
The ''nemomobile-packages'' is meant only for packaging information for packages which contain upstream somewhere else and can be found from https://github.com/nemomobile-packages These are packaged either with [[Nemo/Development#With_git_pkg_service|git_pkg]] or [[Nemo/Development#Using_submodules_with_tar_git|tar_git with submodules]].
 +
 +
Also there are various [[upstream repositories]] around the internet which are used.
 +
 +
As Nemo is using Mer as its core, one should also remember to look into those repositories at http://gitweb.merproject.org/gitweb/
 +
 
= Automated Packaging from git =
 
= Automated Packaging from git =
  
 
== Setting up a webhook to github ==
 
== Setting up a webhook to github ==
  
−
Open the git tree you have. Go to '''Settings->Service Hooks->WebHook URLs''' and enter https://webhook.merproject.org/webhook/ to the URL field. Click "Update Settings" and "Test Hook".  
+
Open the git tree you have. Go to '''Settings->Service Hooks->WebHook URLs''' ([https://help.github.com/articles/post-receive-hooks  Guide at Github])and enter https://webhook.merproject.org/webhook/ to the URL field. Click "Update Settings"<s> and "Test Hook"</s>.  
  
−
After clicking "Test Hook" the webhooks admin interface will show to the admins that you have active git tree. Contact Sage or w00t at #nemomobile IRC channel and they can help you to setup the rest of the process. Here the admin asks you what will be the package name and the target obs project where it should go and saves the information to the webhook admin interface. After this you can check the that the information is valid from https://webhook.merproject.org/webhook/
+
'''NOTE:''' If you don't see the "Settings" tab as shown [https://help.github.com/articles/post-receive-hooks here], contact your project admin to give you more rights or ask them to do it for you.
 +
 
 +
<s>After clicking "Test Hook" the webhooks ''admin interface'' will show to the admins that you have active git tree.</s> Now contact on the admins sledges, Sage, or w00t at #nemomobile IRC channel and they will help you to setup the rest of the webhook process. Here the admin asks you what will be the package name and the target obs project where it should go and saves the information to the webhook admin interface. After this you can check the that the information is valid from https://webhook.merproject.org/webhook/
 +
 
 +
== Preparing the target project in OBS ==
 +
 
 +
'''NOTE:''' To user OBS for building you need to have account to the OBS and created a project e.g., ''home:nemo:xyz''. To get an account follow instructions at https://build.merproject.org/user/register_user
 +
 
 +
'''NOTE:''' This is valid only when you are using webhooks outside ''nemo:*'' OBS projects, e.g., building with webhooks at ''home:*''.
 +
 
 +
To make sure that services have rights add the package content to the OBS the obs target project needs to have '''cibot''' added as the project maintainer. This can be done with osc:
 +
osc -A https://api.merproject.org/ maintainer --add=cibot --role=maintainer '''<TARGET_PROJECT>'''
 +
 
 +
Other than above you don't need to do anything else in OBS, as cibot automatically sets up the package and creates the needed files to obs project
  
 
== With tar_git service ==
 
== With tar_git service ==
 +
 +
tar_git is the service that is used to package packages where nemomobile is the upstream, which means packages that are located at https://github.com/nemomobile/
 +
 +
=== File structure ===
  
 
With tar_git service '''packaging information is carried on the same git branch with the code in rpm/ subdirectory'''. This directory can contain any files which all are exported to packaging in OBS. Usually the content is just plain spec file:
 
With tar_git service '''packaging information is carried on the same git branch with the code in rpm/ subdirectory'''. This directory can contain any files which all are exported to packaging in OBS. Usually the content is just plain spec file:
Line 14: Line 43:
  
 
If more than one .spec, .yaml or .changes is present the one that matches to the "Package" in webhooks will be taken. This allows one to create multiple packages with same source and different .spec file.
 
If more than one .spec, .yaml or .changes is present the one that matches to the "Package" in webhooks will be taken. This allows one to create multiple packages with same source and different .spec file.
 +
 +
tar_git also supports [https://www.kernel.org/pub/software/scm/git/docs/git-submodule.html Git submodules].
 +
 +
=== Changelogs ===
  
 
Changelogs are automatically generated from git commit messages, when ever git commit contains line that contains the following format
 
Changelogs are automatically generated from git commit messages, when ever git commit contains line that contains the following format
−
  [sample text] Longer sample text
+
  [text] Longer messages
−
it is added automatically to the next changelog entry when new version is tagged.
+
  
−
'''NOTE:''' If you have existing changelog that isn't already in proper format in git you need to add the old .changes file to the rpm/ directory. This will be is used as a base of the changelog generation and automated changelogs are appended to that. Should be noted that this changes file should not be updated after adding it to the git tree.
+
In some cases (e.g. required by CI), you might also need to have a bug reference, e.g. for Mer Bugzilla, use:
 +
 
 +
[text] Longer messages (Contributes to MER#123)
 +
 
 +
Or if the change actually closes/fixes some bug:
 +
 
 +
[text] Longer message (Fixes MER#123)
 +
 
 +
Example:
 +
commit 1efda9cf6a950509b26b69c599fbdc89f70104ae
 +
Author: Captain Nemo <captain@nemomobile.org>
 +
Date:  Fri Apr 26 15:21:34 2013 +0300
 +
 +
    This is a sample commit message that contains two entries
 +
    that are picked to the .changes file.
 +
 +
    [packaging] Fixed issues with packaging.
 +
    [binary_x] fix crash when binary_x is called with param -y
 +
   
 +
    Signed-off-by: Captain Nemo <captain@nemomobile.org>
 +
 
 +
it is added automatically to the next changelog entry when new version is tagged. '''NOTE:''' Also should be noted that there ''MUST'' be at least one commit between each tag that contains a line shown above so that the changelog will be generated properly. Here is example how the lines above would correspond in .changes file. If tag 0.1 would point to commit 1efda9cf6a950509b26b69c599fbdc89f70104ae above.
 +
* Fri Apr 26 2013 Captain Nemo <captain@nemomobile.org> - 0.1
 +
- [packaging] Fixed issues with packaging.
 +
- [binary_x] fix crash when binary_x is called with param -y
 +
 
 +
'''NOTE:''' If you have existing changelog that isn't already in proper format, i.e., git tags and commit messages, you need to add the old .changes file to the rpm/ directory. This .changes file will be is used as a base of the changelog generation and automated changelogs are appended to that. ''Should be noted that this changes file '''MUST NOT''' be updated after adding it to the git tree''.
 +
 
 +
'''NOTE:''' If you start from scratch with new package and you start following the changelog convention of tar_git '''no .changes file''' should be added to the rpm/ directory.
 +
 
 +
=== Tagging ===
 +
 
 +
Tags with tar_git should always be version number that was tagged. For example:
 +
0.1
 +
1.2.4
 +
There is also possibility to add prefix if one wants to sort tags or if you have multiple branches from which packages are build and you want to separate those with names.
 +
prefix/0.1
  
 
When ever you do tag on the branch enabled with the webhook, it will automatically launch the tar_git service and the new sources will be downloaded and built on the target project.
 
When ever you do tag on the branch enabled with the webhook, it will automatically launch the tar_git service and the new sources will be downloaded and built on the target project.
 +
 +
Please refer to the tagging format at:
 +
https://wiki.merproject.org/wiki/Nemo/Development/Releasing
 +
 +
=== Logs/Process ===
  
 
To follow the behaviour of your webhooks you can join to #mer-boss IRC channel @ Freenode. Here is couple of examples of those messages:
 
To follow the behaviour of your webhooks you can join to #mer-boss IRC channel @ Freenode. Here is couple of examples of those messages:
Line 30: Line 103:
 
If you tagged a package you can click the link and check that everything is ok.
 
If you tagged a package you can click the link and check that everything is ok.
  
−
== FAQ ==
+
=== Using submodules with tar_git ===
 +
 
 +
When handling upstream packages where one can't use the changlog format in commits or can't push commits as wanted, submodules and tar_git comes very handy. With submodule one can use tar_git easily just by adding packaging and changelog entries in commits. Adding new submodule is easy, for example:
 +
git submodule add https://github.com/libhybris/libhybris.git libhybris
 +
 
 +
Which in this case adds ''https://github.com/libhybris/libhybris.git'' as submodule to ''libhybris'' directory. Dir structure with submodule looks like https://github.com/nemomobile-packages/libhybris
 +
 
 +
If one wants to checkout certain commit from the specific tree it can be done with normal git commands
 +
cd libhybris
 +
git checkout 2a86e842e9daee1d7a222285cd2b29649e63a0b6
 +
 
 +
After this commit the change with proper commit message that contains the lines that are picked to changes file, which looks like https://github.com/nemomobile-packages/libhybris/commit/390677499e44cb044afaed760d353fb2bf413346 and after this tag as usual and tar_git handles the rest.
 +
 
 +
=== Testing rpm package build before committing ===
 +
To test sources that have tar_git packaging there is tool called ''mb2'', this can be installed to [[Platform_SDK|Mer Platform SDK]] and comes in package called ''sdk-utils''. The usage is following:
 +
mb2 -t SB2_TARGET_NAME build
 +
This automatically downloads and installs the dependencies to the sb2 target you specified and then builds and rpm package from the sources within that sb2 target.
 +
 
 +
After this you can move the rpm to the target device and install it and test that everything is ok. The rpm filename can be seen in the end of mb2 command if it was successfull. Example below.
 +
...
 +
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/sage/rpmbuild/BUILDROOT/nemo-mobile-session-13-0.arm
 +
'''Wrote: /home/sage/rpmbuild/RPMS/armv7hl/nemo-mobile-session-13-0.armv7hl.rpm'''
 +
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.YyhUi6
 +
+ umask 022
 +
+ cd /home/sage/rpmbuild/BUILD
 +
+ cd nemo-mobile-session
 +
+ /bin/rm -rf /home/sage/rpmbuild/BUILDROOT/nemo-mobile-session-13-0.arm
 +
+ exit 0
 +
 
 +
In case you are building dependency packages and need them for sb2 target you can also install it to your target easily
 +
sb2 -t mer-core -R -m sdk-install rpm -Uvh --force /home/sage/rpmbuild/RPMS/armv7hl/nemo-mobile-session-13-0.armv7hl.rpm
 +
 
 +
=== FAQ ===
 
  Q: I removed a tag remotely and pushed it again, but this didn't trigger the webhook at github?
 
  Q: I removed a tag remotely and pushed it again, but this didn't trigger the webhook at github?
 
  A: Retag locally and then push again.
 
  A: Retag locally and then push again.
  
 
== With git_pkg service ==
 
== With git_pkg service ==
−
TODO
+
For packages where Nemo isn't upstream there is plan to use git_pkg instead of tar_git. See https://github.com/lbt/git-pkg for the code. This integration work has not started yet.
 +
 
 +
Before starting to use git_pkg you need to set following environment variables if using gitpkg <= 0.0.5
 +
export EDITOR=vim
 +
export GIT_AUTHOR_NAME="Developer"
 +
export GIT_AUTHOR_EMAIL="developer@my-dev.dev"
 +
 
 +
=== Structure ===
 +
With gitpkg there is the master branch which contains the code, but only the code and no packaging information. The packaging information is located in separate ''pkg-mer'' branch.
 +
 
 +
=== Create new release ===
 +
Before doing release you need to tag the version in master branch, e.g.,
 +
git tag 0.26
 +
After this you can do the release
 +
gp_release --ver=0.26
 +
After calling this command EDITOR is opened and you are asked to fill in the changelog information.
  
 +
=== FAQ ===
 +
'''Q:''' I screwed up something when doing gp_release --ver=0.1.1 how can I revert?
  
 +
'''A:'''
 +
git tag -d pkg-mer-0.1.1-1 mer-0.1.1-1;
 +
git checkout pkg-mer && git reset --hard HEAD~1 && git checkout master;
  
 
= Other stuff =
 
= Other stuff =

Latest revision as of 16:32, 1 June 2015

Contents

[edit] Repositories

Nemo Mobile has three organisations in the github: nemomobile, nemobile-ux, and nemobile-packages.

nemomobile contains packages/components/configs/etc. where Nemo is the upstream project (nemomobile-ux will contain all Glacier UX packages). This organization can be found from https://github.com/nemomobile Each repo project contains packaging information which is using the tar_git service

The nemomobile-packages is meant only for packaging information for packages which contain upstream somewhere else and can be found from https://github.com/nemomobile-packages These are packaged either with git_pkg or tar_git with submodules.

Also there are various upstream repositories around the internet which are used.

As Nemo is using Mer as its core, one should also remember to look into those repositories at http://gitweb.merproject.org/gitweb/

[edit] Automated Packaging from git

[edit] Setting up a webhook to github

Open the git tree you have. Go to Settings->Service Hooks->WebHook URLs (Guide at Github)and enter https://webhook.merproject.org/webhook/ to the URL field. Click "Update Settings" and "Test Hook".

NOTE: If you don't see the "Settings" tab as shown here, contact your project admin to give you more rights or ask them to do it for you.

After clicking "Test Hook" the webhooks admin interface will show to the admins that you have active git tree. Now contact on the admins sledges, Sage, or w00t at #nemomobile IRC channel and they will help you to setup the rest of the webhook process. Here the admin asks you what will be the package name and the target obs project where it should go and saves the information to the webhook admin interface. After this you can check the that the information is valid from https://webhook.merproject.org/webhook/

[edit] Preparing the target project in OBS

NOTE: To user OBS for building you need to have account to the OBS and created a project e.g., home:nemo:xyz. To get an account follow instructions at https://build.merproject.org/user/register_user

NOTE: This is valid only when you are using webhooks outside nemo:* OBS projects, e.g., building with webhooks at home:*.

To make sure that services have rights add the package content to the OBS the obs target project needs to have cibot added as the project maintainer. This can be done with osc: 

osc -A https://api.merproject.org/ maintainer --add=cibot --role=maintainer <TARGET_PROJECT>

Other than above you don't need to do anything else in OBS, as cibot automatically sets up the package and creates the needed files to obs project

[edit] With tar_git service

tar_git is the service that is used to package packages where nemomobile is the upstream, which means packages that are located at https://github.com/nemomobile/

[edit] File structure

With tar_git service packaging information is carried on the same git branch with the code in rpm/ subdirectory. This directory can contain any files which all are exported to packaging in OBS. Usually the content is just plain spec file: 

rpm/
  example.spec

If more than one .spec, .yaml or .changes is present the one that matches to the "Package" in webhooks will be taken. This allows one to create multiple packages with same source and different .spec file.

tar_git also supports Git submodules.

[edit] Changelogs

Changelogs are automatically generated from git commit messages, when ever git commit contains line that contains the following format 

[text] Longer messages

In some cases (e.g. required by CI), you might also need to have a bug reference, e.g. for Mer Bugzilla, use: 

[text] Longer messages (Contributes to MER#123)

Or if the change actually closes/fixes some bug: 

[text] Longer message (Fixes MER#123)

Example: 

commit 1efda9cf6a950509b26b69c599fbdc89f70104ae
Author: Captain Nemo <captain@nemomobile.org>
Date:   Fri Apr 26 15:21:34 2013 +0300

    This is a sample commit message that contains two entries 
    that are picked to the .changes file.

    [packaging] Fixed issues with packaging.
    [binary_x] fix crash when binary_x is called with param -y
   
    Signed-off-by: Captain Nemo <captain@nemomobile.org>

it is added automatically to the next changelog entry when new version is tagged. NOTE: Also should be noted that there MUST be at least one commit between each tag that contains a line shown above so that the changelog will be generated properly. Here is example how the lines above would correspond in .changes file. If tag 0.1 would point to commit 1efda9cf6a950509b26b69c599fbdc89f70104ae above. 

* Fri Apr 26 2013 Captain Nemo <captain@nemomobile.org> - 0.1
- [packaging] Fixed issues with packaging.
- [binary_x] fix crash when binary_x is called with param -y

NOTE: If you have existing changelog that isn't already in proper format, i.e., git tags and commit messages, you need to add the old .changes file to the rpm/ directory. This .changes file will be is used as a base of the changelog generation and automated changelogs are appended to that. Should be noted that this changes file MUST NOT be updated after adding it to the git tree.

NOTE: If you start from scratch with new package and you start following the changelog convention of tar_git no .changes file should be added to the rpm/ directory.

[edit] Tagging

Tags with tar_git should always be version number that was tagged. For example: 

0.1
1.2.4

There is also possibility to add prefix if one wants to sort tags or if you have multiple branches from which packages are build and you want to separate those with names. 

prefix/0.1

When ever you do tag on the branch enabled with the webhook, it will automatically launch the tar_git service and the new sources will be downloaded and built on the target project.

Please refer to the tagging format at: https://wiki.merproject.org/wiki/Nemo/Development/Releasing

[edit] Logs/Process

To follow the behaviour of your webhooks you can join to #mer-boss IRC channel @ Freenode. Here is couple of examples of those messages: 

< Merbot`> Tag(s) by Developer X in master branch of https://github.com/nemomobile/nemo-qml-plugins.git, which will trigger build in project nemo:devel:mw package nemo-qml-plugins (https://build.merproject.org//package/show?package=nemo-qml-plugins&project=nemo:devel:mw)
< Merbot`> 1 commit(s) pushed by Developer Y to master branch of https://github.com/tswindell/telepathy-ring.git

If you tagged a package you can click the link and check that everything is ok.

[edit] Using submodules with tar_git

When handling upstream packages where one can't use the changlog format in commits or can't push commits as wanted, submodules and tar_git comes very handy. With submodule one can use tar_git easily just by adding packaging and changelog entries in commits. Adding new submodule is easy, for example: 

git submodule add https://github.com/libhybris/libhybris.git libhybris

Which in this case adds https://github.com/libhybris/libhybris.git as submodule to libhybris directory. Dir structure with submodule looks like https://github.com/nemomobile-packages/libhybris

If one wants to checkout certain commit from the specific tree it can be done with normal git commands 

cd libhybris
git checkout 2a86e842e9daee1d7a222285cd2b29649e63a0b6

After this commit the change with proper commit message that contains the lines that are picked to changes file, which looks like https://github.com/nemomobile-packages/libhybris/commit/390677499e44cb044afaed760d353fb2bf413346 and after this tag as usual and tar_git handles the rest.

[edit] Testing rpm package build before committing

To test sources that have tar_git packaging there is tool called mb2, this can be installed to Mer Platform SDK and comes in package called sdk-utils. The usage is following: 

mb2 -t SB2_TARGET_NAME build

This automatically downloads and installs the dependencies to the sb2 target you specified and then builds and rpm package from the sources within that sb2 target.

After this you can move the rpm to the target device and install it and test that everything is ok. The rpm filename can be seen in the end of mb2 command if it was successfull. Example below. 

...
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/sage/rpmbuild/BUILDROOT/nemo-mobile-session-13-0.arm
Wrote: /home/sage/rpmbuild/RPMS/armv7hl/nemo-mobile-session-13-0.armv7hl.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.YyhUi6
+ umask 022
+ cd /home/sage/rpmbuild/BUILD
+ cd nemo-mobile-session
+ /bin/rm -rf /home/sage/rpmbuild/BUILDROOT/nemo-mobile-session-13-0.arm
+ exit 0

In case you are building dependency packages and need them for sb2 target you can also install it to your target easily 

sb2 -t mer-core -R -m sdk-install rpm -Uvh --force /home/sage/rpmbuild/RPMS/armv7hl/nemo-mobile-session-13-0.armv7hl.rpm

[edit] FAQ 

Q: I removed a tag remotely and pushed it again, but this didn't trigger the webhook at github?
A: Retag locally and then push again.

[edit] With git_pkg service

For packages where Nemo isn't upstream there is plan to use git_pkg instead of tar_git. See https://github.com/lbt/git-pkg for the code. This integration work has not started yet.

Before starting to use git_pkg you need to set following environment variables if using gitpkg <= 0.0.5 

export EDITOR=vim
export GIT_AUTHOR_NAME="Developer"
export GIT_AUTHOR_EMAIL="developer@my-dev.dev"

[edit] Structure

With gitpkg there is the master branch which contains the code, but only the code and no packaging information. The packaging information is located in separate pkg-mer branch.

[edit] Create new release

Before doing release you need to tag the version in master branch, e.g., 

git tag 0.26

After this you can do the release 

gp_release --ver=0.26

After calling this command EDITOR is opened and you are asked to fill in the changelog information.

[edit] FAQ

Q: I screwed up something when doing gp_release --ver=0.1.1 how can I revert?

A: 

git tag -d pkg-mer-0.1.1-1 mer-0.1.1-1;
git checkout pkg-mer && git reset --hard HEAD~1 && git checkout master;

[edit] Other stuff

This section is a bit of a guide about how to perform unified development on the software that we (in Nemo) write and maintain.

Retrieved from "https://wiki.merproject.org/wiki/index.php?title=Nemo/Development&oldid=5351"
Personal tools