The Mer Wiki now uses your Mer user account and password (create account on https://bugs.merproject.org/)

Nemo/Audio

From Mer Wiki
Jump to: navigation, search

Contents

Audio in Nemo

This page tries to document all relevant information regarding audio handling in Nemo, what pieces are relevant for which purpose and why and list places for further documentation.

Relevant Components

Configuration and adaptation

Audio modes

Audio modes are in practice audio routings as well as mode specific role-volumes. Audio mode configuration is always adaptation specific, all modes are not necessarily present in every adaptation.

Examples of audio modes,

  • ihf (Internal Hands-Free, or the device loudspeaker)
  • lineout (Normal stereo-device connected to audio jacket, usually 3.5mm stereo jack)
  • hp (Head Phone, small loudspeaker in device used for call audio)
  • bta2dp (Bluetooth A2DP, wireless A2DP device)

Audio routing targets are configured using PulseAudio port-configurations, each routing target has its own port-configuration. When audio mode changes, alsa-sink's active port is changed for routing related to given mode. Port configurations are in device at /usr/share/pulseaudio/alsa-mixer/{paths,profiles}

stream-restore-nemo contains patches to enable role-volumes, or route-volumes, which means every audio mode has individual volume level stored. When audio mode changes stream-restore-nemo applies mode's stored volume values to streams. Stream-restore-nemo also exports volume-proxy interface to communicate volume changes with Mainvolume.

pulseaudio-policy-enforcement-module enforces routing changes and other relevant audio policy rules (enforcing volume limits, corking streams that are not allowed to play for example during call, etc.) to PulseAudio and policy-enforcement-module communicates with OHM.

Mainvolume

Nemo/Audio/MainVolume

The purpose of mainvolume-module in PulseAudio is to simplify volume control by providing single entrypoint to control role volumes. Volume controlling of streams should not normally be done by hand (few important exceptions exist). Mainvolume is controlled using DBus API.

How volume works

Volume levels are stored in stream-restore-nemo database. There are two kinds of volumes, "static" volumes, or the normal stream volumes, which stay the same regardless of the audio mode, and "role-volumes or route-volumes" which are stored for each individual mode.

For example, role-volumes can be thought as

  • sink-input-by-media-role:x-maemo:ihf
  • sink-input-by-media-role:x-maemo:lineout
  • sink-input-by-media-role:x-maemo:earpiece

and so on. When volume for sink-input-by-media-role:x-maemo is queried, stream-restore-nemo only replies currently active role-volume. Applying role-specific volume to this entry is done behind the scenes when audio mode changes. 

       PA
   |-----------------------|
   | stream-restore-nemo --|----- Stream volume database, contains volumes for role-volumes as well as
   |       |          |    |      "static" stream-volumes.
   | mainvolume   DBus API |
   |       |          ^    |
   | DBus API         |    |
   |       ^          |    |
   |-------|----------|----|
           |          |
           |        applications changing static volumes (eg ngfd)
           |
   volume controlling application
   
   Mainvolume client -> Mainvolume -> Volume proxy -> Stream Restore Nemo

So, for example if ihf-route is selected, volume is set to -6dB, when user connects headset to device lineout-route is selected and its stored volume applied, and volume is for example at level -16dB.

Role-volumes are defined in /etc/pulse/x-maemo-route.table. Syntax is 

   <route-stream-name> <default volume in dB> [minimum volume in dB]

for example: 

   sink-input-by-media-role:x-maemo -15
   sink-input-by-media-role:phone -10 -20

Default volume is set to stream when entering mode that doesn't have any stream volume stored yet. If minimum volume is set and volume is below minimum volume level, next time route with that volume is entered default volume is applied instead. This is useful in phone call case to prevent audio volume to be inaudible if for some reason volume is set to really low value in previous call.

When mainvolume-module is used for volume controlling, it controls precisely these predefined route-volumes. When in-call, it controls media-role:phone stream volume, and otherwise media-role:x-maemo. Also mainvolume module has configurable steps for volume levels, for example if five steps are used for in-call case they could be -15dB, -10dB, -7dB, -5dB, -2dB, 0dB. Then mainvolume exposes 5 steps in call-mode.

module-meego-parameters handles mode specific preferences and passes Mainvolume step configuration to Mainvolume as well.

Few useful command line applications for volumes (mainvolume-volume.py pasr.py)

Parameters Module

Module parameters: 

   cache true/false    Cache parameters, when tuning new algorithm parameters this should be set to false, so that changing modes also reloads parameters from disk.
   directory <PATH>    Directory where algorithm- and mode-sets are located. Defaults to /var/lib/pulse-nokia
   initial_mode <MODE> String of default mode parameters module loads before getting proper mode from OHM. Defaults to ihf

module-meego-parameters handles parameter sets for each mode, passes new parameters for modules that need them when mode changes etc. Parameters are currently stored in /var/lib/pulse-nokia

Parameters consist of algorithm-sets and mode-sets. Algorithm-sets contain all possible configurations for all algorithms, mode-sets combine mode-specific parameters from available algorithm-sets.

For example, ihf mode could consist of mainvolume and xprot parameters, found in 

   algs/mainvolume/set-suitable-for-ihf
   algs/xprot/set-suitable-for-ihf

Mode ihf contains these two sets, 

   modes/ihf/mainvolume -> ../../algs/mainvolume/set-suitable-for-ihf
   modes/ihf/xprot -> ../../algs/xprot/set-suitable-for-ihf

Sets in modes are symbolic links to contents in algorithm-sets.

Current setup with N9xx doesn't anymore contain binary algorithms due to incompatibility to updated PulseAudio, so only mainvolume sets are effective as of now.

Stream Restore Nemo (module-stream-restore-nemo)

DBus API is identical to upstream stream-restore: http://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/Developer/Clients/DBus/StreamRestore

stream-restore-nemo is upstream stream-restore module with patches to enable role-volumes and small proxy which is used by mainvolume to communicate to stream-restore-nemo. Stream volumes can be changed directly with stream-restore-nemo as well. If volumes are set for streams other than route-volume-streams, everything works fine as well, but the volumes are not updated according to audio mode.

ngfd communicates directly with stream-restore-nemo to update volumes for event sounds, ringtone volume, etc.

Audio resource policy

Audio policy manager is part of Policy Framework and governs audio (and video) resources.

ohmd is the central daemon which is running plugins that do the various work needed. It also has prolog engine which does policy decisions based on device state. 

   ohmd            -> policy-enforcement-module      -> PulseAudio streams etc.
   - telephony        - static configuration
   - accessories      - dynamic based on ohmd control
   - media
   - resource

Audio resources, groups and rules are defined in policy settings, accessory_rules.pl, audio_cork_rules.pl, audio_groups.pl, audio_route_rules.pl, audio_volume_rules.pl, resource_classes.pl, resource_rules.pl, compiled into plc file and installed in /usr/share/policy/rules/current/. Part of audio stream classification is done statically in xpolicy.conf, also from policy settings package, installed in /etc/pulse/xpolicy.conf.

qt-mobility gstreamer multimedia plugins MediaPlayer and CameraBin have built-in support for resource policy, so if you are using those components for media applications you don't need to (and actually you should not) implement your own resource handling.

Static system wide streams, or streams from applications that are a set of "basic Nemo experience" (ringtone, alerts, UI feedback sounds and like) can be classified statically in the xpolicy.conf file.

Other audio applications, media player, games, etc. should acquire resources in the way explained in the examples. As done in the example, audio applications should ask for resources immediately, but not make the requirement mandatory (isOptional true, which is also default). That way you can call acquire() for the ResourceSet immediately and get resourcesGranted signal whenever the resources are available.

If/when audio enforcing works properly, most audio is routed to null-sink or corked during calls anyway, but it's good practice to pause playback when resources are lost.

To encourage developers to implement proper audio resource handling in Nemo, playback of unknown/unclassified streams is not allowed. If you need to test your audio application before resource handling is implemented, you can add your binary to xpolicy.conf. But correct way is to request audio resources dynamically and it is usually easier to take resource handling into account early in development.

Libresource-qt API documentation is still hosted at harmattan-dev as well..

http://harmattan-dev.nokia.com/docs/library/html/libresourceqt/main.html?tab=0

Example

https://gitorious.org/libresourceqt-usage-demos/libresourceqt-demo-player/blobs/master/playerwidget.cpp

Adding stream to xpolicy.conf for testing

If you need to test audio application that doesn't have resource handling implemented yet, you can add static configuration to local xpolicy.conf.

Create directory /etc/pulse/xpolicy.conf.d

Create file for configuration, for example

/etc/pulse/xpolicy.conf.d/application.conf: 

   # match using binary name
   [stream]
   exe = my-binary-name
   group = player

   # or match using proplist entries
   [stream]
   property = application.name@equals:"Music Shelf"
   group = player

Groups are listed in Group section in /etc/pulse/xpolicy.conf, but usually only player or game is relevant.

After changing xpolicy.conf restart PulseAudio to get new configuration in use 

   # As nemo user
   systemctl --user restart pulseaudio.service

When your application is outputting audio, issuing pactl list sink-inputs will show properties of the stream. There is property policy.group which will tell the group of the stream. If this shows "othermedia", it means audio is not allowed to play. For above example policy.group = "player"

For example running paplay with static group player will show: 

   $ pactl list sink-inputs
   Sink Input #2
           [...snip...]
           Properties:
                   media.format = "WAV (Microsoft)"
                   application.name = "paplay"
                   media.name = "/test_clips/short_stereo_48000.wav"
                   native-protocol.peer = "UNIX socket client"
                   native-protocol.version = "29"
                   application.process.id = "15828"
                   application.process.user = "nemo"
                   application.process.host = "Jolla"
                   application.process.binary = "pacat"
                   application.language = "en_US.UTF-8"
                   application.process.machine_id = "d9080909d80e4d5a91ae964d3af2d038"
                   application.process.session_id = "c42"
                   application.process.arg0 = "paplay"
                   policy.group = "player"
                   policy.stream_flags = hex:00000000
                   media.role = "x-maemo"
                   module-stream-restore.id = "sink-input-by-media-role:x-maemo"

Stream creation with Nemo

This is relevant if you are writing audio application using PulseAudio API directly. Usually it's better to use some higher level library.

http://freedesktop.org/software/pulseaudio/doxygen/streams.html

In normal use cases new streams should be created without volume (or PA_PROP_MEDIA_ROLE proplist value) defined so that module-match and module-stream-restore-nemo can match, classify and restore volume for newly created stream. Streams without a role (PA_PROP_MEDIA_ROLE) are defaulted to "x-maemo", default is defined in /etc/pulse/x-maemo-match.table 

   pa_stream *stream;
   pa_proplist *proplist;
   
   proplist = pa_proplist_new();
   pa_proplist_sets(proplist, PA_PROP_MEDIA_ROLE, "x-maemo");
   stream = pa_stream_new_with_proplist(context, "my stream", &sample_spec, &channel_map, proplist);
   
   // Connect default stream
   pa_stream_connect_playback(stream, NULL, NULL, 0, NULL, NULL);

If volume is defined for new stream module-stream-restore-nemo doesn't change the value but the stream passes through untouched. Also role (PA_PROP_MEDIA_ROLE) should be defined for the stream so that the new stream doesn't interfere with role streams. PulseAudio in Nemo runs in flat volume volume which means that volume passed to pa_stream_connect_playback() is absolute volume. Thus if volume is wanted for a stream; (usually playback stream is either mono or stereo) 

   pa_stream *stream;
   pa_cvolume volume;
   pa_proplist *proplist;
   
   double dB_volume = -5.0;
   
   pa_cvolume_set(&volume, sample_spec.channels, pa_sw_volume_from_dB(dB_volume));
   
   proplist = pa_proplist_new();
   pa_proplist_sets(proplist, PA_PROP_MEDIA_ROLE, "event");
   
   // Create new stream with defined role
   stream = pa_stream_new_with_proplist(context, "my-stream", &sample_spec, &channel_map, proplist);
   pa_stream_connect_playback(stream, NULL, NULL, 0, &volume, NULL);
   
   pa_proplist_free(proplist);

Now in ideal world the device Nemo is running in has proper ALSA dB info and/or tuned volume levels and you will get playback with true absolute volume -5dB.

Life of a new stream in Nemo

  1. Stream is created with NULL device, NULL name and NULL volume
  2. Stream is connected to context
  3. New corresponding sink-input is created in server-side
  4. Default sink is set for new sink-input (sink.music)
  5. Policy enforcement module classifies stream
  6. module-match renames unnamed stream to default name
  7. stream-restore-nemo classifies stream and applies current role volume for the stream
  8. Client starts to transfer data to stream

libresource

resource.h API example

To classify as a let's say "game", 

   #include <sys/types.h>
   #include <unistd.h> /* for getpid() */
   
   #include <resource.h>
   
   resource_set_t *resource_set;
   
   resource_set = resource_set_create ("game", RESOURCE_AUDIO_PLAYBACK, 0, 0, grant_callback, NULL);
   resource_set_configure_advice_callback (resource_set, advice_callback, NULL);
   resource_set_configure_audio (resource_set, "game", getpid(), "*");
   resource_set_acquire (resource_set);
   
   /* implement both grant_callback and advice_callback */
   static void grant_callback (resource_set_t *resource_set, uint32_t resources, void *userdata) {
       if (resources & RESOURCE_AUDIO_PLAYBACK) {
           /* our resources are granted, play audio */
       } else {
           /* we lost our resources, (some higher priority client has the resources), be nice and stop playing
       }
   }
   
   static void advice_callback (resource_set_t *resource_set, uint32_t resources, void *userdata) {
   }
Retrieved from "https://wiki.merproject.org/wiki/index.php?title=Nemo/Audio&oldid=4913"
Personal tools